faster improvement with adaptive implementation research
TRANSCRIPT
Faster Improvement with
Adaptive Implementation
Research
1
John Oslashvretveit Director of Research Professor of Health Innovation and
Evaluation Karolinska Institutet Stockholm Sweden
232016
Help with these challenges Changing clinical practice and service delivery
eg use this test in these situations not these establish team-based service
Effective methods for change under-used by improvement projects
Frustration of practical improvers research not answering their questions
how do we implement this costs and savings conditions we need to get the same
results
Researcherrsquos challenges reducing time and cost
Time and funding to publish
Role for working on implementation ndash for trial or spread ndash time and funding
documenting changes
attributing outcomes (esp which part most impactful)
generalising take-up of our research 2
232016
Does not help with hellip
Changing academic research criteria Not improverrsquos questions
Do not reward ldquo3R researchrdquo relevant responsive
rigorous
Rather ldquo3P review criteriardquo for proposals publications
and promotion using RCT ideally
Settings where implementation is not
possible
but will know why 3
Relevance to researchers
if evaluating an intervention (eg new treatment or service delivery model)
knowledge of non-experimental methods which data to gather to document and evaluate
designing effective intervention (eg protocol for establishing an intervention to test)
Issues in translating research into practice and
scale-up ndash researcherrsquos role
4 232016
It was effective therehellip but different herehellip
5 232016
We donrsquot have a
problem here
Our patientsservice
is different
We have other changes
and priorities
Session covers
Case example - problems
Proven solutions ndash why not implemented
ldquoFitrdquo of Solution in the ldquoContextrdquo
Does the adaption work Adaptive implementation research
Research to answer improverrsquos questions
Implications for you
6 232016
What I mean byhellip
ldquoInterventionrdquo covers both
a)Do hand hygiene consistently (beforeafter change)
b) Training and feedback (actions to enable change)
Words
a) the ldquonew better wayrdquo - hand hygiene eg wash hands between patients use bar code reader to reduce medication errors rapid response
team to prevent avoidable deterioration on nursing units (The Improvement-change)
b) What we do directly to enable ldquotake uprdquo by staff of the
new better way (ldquoimplementationrdquo methods - training feedback rewards punishment for not using etc)
7 232016
8 232016
B Better hand
hygiene
A Poor hand
hygiene
Intervention Plan Wash hands between
patients Reduce barriers
Feedback
Implementation actions (phases) 1Education 2Gell dispensers 3Patient expectations 4Feedback compliance (infection rates)
- Patient
- Close carers
- Providers
How
Surrounding ldquocontextrdquo helps and hinders
Intervention concept
Wash hands between patients
Reduce barriers Feedback
What I mean byhellip
Conditions or context for the intervention
= influences which indirectly help or hinder the
intervention
high workload affects hand hygiene
disruption to supplies
IT system
how the hospital or physicians are paid
9 232016
232016 10
Mary 84 yrs Obstructive airways (COPD)
heart disease mild depression
Stable at home on meds
very independent
Unpaid motivational coach
and security-guard - ldquoMattyrdquo
Healthcare experience
Emergency hospital admission
Delays and meds changed
HAI
Sent home ndash no comms or support
Readmission
Beforeafter
11
232016
Mary - six weeks later
After hospitalisation
Avoidable cost 4600$
12
232016
Name a proven interventions for any problems
Changes of medswrong meds
Hospital acquired infection
After care transition
Medication reconcilliation
Hand hygiene - bundle of interventions
After-care - Coleman CTI ndash Transition coach and
education 13
232016
Your experience
We have implemented solutions to prevent this
- Yes No sometimes
We have effective Med Rec - Medications changes
unlikely
HAI from Hand Hygiene non-compliance unlikely
After care information to PCP ensured within 2
days of discharge
Effective after-care support for older multiple
morbidity patients
14
232016
Why not implemented - your vote
No problem (one-off event)
Not aware of these proven effective solutions
Know how to implement - donrsquot have time or
resources to implement
To implement effectively - donrsquot have knowledge
and skills
Know solutions - uncertain if we would get same
results - would it work here
Donrsquot have research to show how to implement in
different settings
15
Johns Questhellip
Why not implemented
What would help fast amp widespread
implementation
How can research best help
16
232016
Conclusions so farhellip
hellipin 2 slides
232016 17
Success depends onhellip
Seed Gardenerplanting amp nurture Soil climate
Idea 10
AdaptionImplementation 30
Personalities 20
The 10203040 change success rule
Soil receptive ndash staff
readiness
amp wider Climate 40
No intervention survives first contact with context
18
232016
Implemented as planned
Intervention plan
Adapt to survive
Adapt the improvement change tested
elsewhere
Eg turn at risk patients every 4hrs not 2hrs
(Pressure ulcer intervention)
Adapt the context
Increase staffing by 05 FTE
Both - Adapt the change amp context
Need ldquoGoldilocks Improvement-Fit for
Take Uprdquo GIFTU
19
Context ndash innovation ldquoFitrdquo
20
232016
Fidelity and adaptive implementation
t
21
232016
Adapt to ldquoFitrdquo to setting and subjects
Adjust over time ndash dynamicgtgtgtgtgt
To do this-Resources data and skills
Research questionshellip
Is the adaptation more or less effective
How can we find effective adaptations in
different settings
How can we do lower-cost faster research
on questions which will help take up
22
232016
Research ldquoCaserdquo example
Improving physicians hand hygiene compliance
ndash most effective intervention
Pilot study 8 hospitals using QI methods Identified 24 causes
Different in each hospital
Interventions tailored to local cause
2nd study ndash web based tool Collect data about your causes (from 24) and used these
interventions to fix your local causes
8 Pilots compliance 48 81
Vs 174 organisations 58 84 (769
projects in
23
Variation reported in a UK study
24
232016
One lesson for research from case
Providers supplied data
Method enables easy local adaptation
But adaptations made are unknown
Effectiveness on average not the only knowledge needed by implementers
Need data to explain variations Not implemented
Context hinders
Adapt successfully or unsuccessfully
25
232016
More informative research
Experimental Trials Can and should we explain high and low outcomes
performers
Can we do subgroup analysis
Focus on non-experimental observational
studies
26
232016
Do you have any of these ldquoResearch
challengesrdquo Documenting
Use taxonomies of implementation methods
Use adaptation typology
Attributing Use Implementation outcome model
Use Logic Modelprogramme theory
Generalising ndash guidance Find which adaptations in which context get which results
Use (take up by practitioners)
= need Pre-study planning to get data for
27
Exercise
You are the head of a clinic
To enable physicians to follow antibiotic
prescribing guidelineshellip
Which behavioral change techniques would
be most effective
Would you use the same in FQC PHC and
Regan paediatrics outpatients 28
232016
Selection from 93 behavior change techniques (Michie 2013)
1 Instruction on how to perform a behavior
2 Punishment
3 Anticipation of future rewards or removal of punishment
4 Practice with Graded tasks
5 Mental rehearsal of successful performance
6 Examine Anticipated regret
7 Feedback on behavior
8 Other(s) monitoring with awareness
9 Social comparison
10 Behavioral contract
11 Social support
12 Identification of self as role model
29
232016
See also Powell 2015 73 implementation
strategies
Mixes direct interventions and indirect context
changes
Education
Audit and provide feedback
Create or change credentialing andor licensure
standards
Develop disincentives
Use capitated payments
Do we need to distinguish 30
232016
Document
Adaptions
1Who made the
modification
2What was modified
(Stirman 2009)
31
232016
Documenting critical context factors -
framworks CFIR (Damschroder 2009)
PHARIS (Rycroft-Malone 2002)
ORCA readiness - based on PHARIS
MUSIQ (Kaplan et al 2010) - QI
French et al 2009 (review of context measures
for evidence-based practice (EBP))
32
232016
Recommended practical tools ORCA (or CFIR httpwwwcfirguideorg)
33
change readiness (HRET)
4 areas the innovation target audience the organisation
the environment
34
Attributing ndash use causal chain frameworks Theory informed Case evaluation
35
232016
Programme theory models or logic models
36
232016
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Help with these challenges Changing clinical practice and service delivery
eg use this test in these situations not these establish team-based service
Effective methods for change under-used by improvement projects
Frustration of practical improvers research not answering their questions
how do we implement this costs and savings conditions we need to get the same
results
Researcherrsquos challenges reducing time and cost
Time and funding to publish
Role for working on implementation ndash for trial or spread ndash time and funding
documenting changes
attributing outcomes (esp which part most impactful)
generalising take-up of our research 2
232016
Does not help with hellip
Changing academic research criteria Not improverrsquos questions
Do not reward ldquo3R researchrdquo relevant responsive
rigorous
Rather ldquo3P review criteriardquo for proposals publications
and promotion using RCT ideally
Settings where implementation is not
possible
but will know why 3
Relevance to researchers
if evaluating an intervention (eg new treatment or service delivery model)
knowledge of non-experimental methods which data to gather to document and evaluate
designing effective intervention (eg protocol for establishing an intervention to test)
Issues in translating research into practice and
scale-up ndash researcherrsquos role
4 232016
It was effective therehellip but different herehellip
5 232016
We donrsquot have a
problem here
Our patientsservice
is different
We have other changes
and priorities
Session covers
Case example - problems
Proven solutions ndash why not implemented
ldquoFitrdquo of Solution in the ldquoContextrdquo
Does the adaption work Adaptive implementation research
Research to answer improverrsquos questions
Implications for you
6 232016
What I mean byhellip
ldquoInterventionrdquo covers both
a)Do hand hygiene consistently (beforeafter change)
b) Training and feedback (actions to enable change)
Words
a) the ldquonew better wayrdquo - hand hygiene eg wash hands between patients use bar code reader to reduce medication errors rapid response
team to prevent avoidable deterioration on nursing units (The Improvement-change)
b) What we do directly to enable ldquotake uprdquo by staff of the
new better way (ldquoimplementationrdquo methods - training feedback rewards punishment for not using etc)
7 232016
8 232016
B Better hand
hygiene
A Poor hand
hygiene
Intervention Plan Wash hands between
patients Reduce barriers
Feedback
Implementation actions (phases) 1Education 2Gell dispensers 3Patient expectations 4Feedback compliance (infection rates)
- Patient
- Close carers
- Providers
How
Surrounding ldquocontextrdquo helps and hinders
Intervention concept
Wash hands between patients
Reduce barriers Feedback
What I mean byhellip
Conditions or context for the intervention
= influences which indirectly help or hinder the
intervention
high workload affects hand hygiene
disruption to supplies
IT system
how the hospital or physicians are paid
9 232016
232016 10
Mary 84 yrs Obstructive airways (COPD)
heart disease mild depression
Stable at home on meds
very independent
Unpaid motivational coach
and security-guard - ldquoMattyrdquo
Healthcare experience
Emergency hospital admission
Delays and meds changed
HAI
Sent home ndash no comms or support
Readmission
Beforeafter
11
232016
Mary - six weeks later
After hospitalisation
Avoidable cost 4600$
12
232016
Name a proven interventions for any problems
Changes of medswrong meds
Hospital acquired infection
After care transition
Medication reconcilliation
Hand hygiene - bundle of interventions
After-care - Coleman CTI ndash Transition coach and
education 13
232016
Your experience
We have implemented solutions to prevent this
- Yes No sometimes
We have effective Med Rec - Medications changes
unlikely
HAI from Hand Hygiene non-compliance unlikely
After care information to PCP ensured within 2
days of discharge
Effective after-care support for older multiple
morbidity patients
14
232016
Why not implemented - your vote
No problem (one-off event)
Not aware of these proven effective solutions
Know how to implement - donrsquot have time or
resources to implement
To implement effectively - donrsquot have knowledge
and skills
Know solutions - uncertain if we would get same
results - would it work here
Donrsquot have research to show how to implement in
different settings
15
Johns Questhellip
Why not implemented
What would help fast amp widespread
implementation
How can research best help
16
232016
Conclusions so farhellip
hellipin 2 slides
232016 17
Success depends onhellip
Seed Gardenerplanting amp nurture Soil climate
Idea 10
AdaptionImplementation 30
Personalities 20
The 10203040 change success rule
Soil receptive ndash staff
readiness
amp wider Climate 40
No intervention survives first contact with context
18
232016
Implemented as planned
Intervention plan
Adapt to survive
Adapt the improvement change tested
elsewhere
Eg turn at risk patients every 4hrs not 2hrs
(Pressure ulcer intervention)
Adapt the context
Increase staffing by 05 FTE
Both - Adapt the change amp context
Need ldquoGoldilocks Improvement-Fit for
Take Uprdquo GIFTU
19
Context ndash innovation ldquoFitrdquo
20
232016
Fidelity and adaptive implementation
t
21
232016
Adapt to ldquoFitrdquo to setting and subjects
Adjust over time ndash dynamicgtgtgtgtgt
To do this-Resources data and skills
Research questionshellip
Is the adaptation more or less effective
How can we find effective adaptations in
different settings
How can we do lower-cost faster research
on questions which will help take up
22
232016
Research ldquoCaserdquo example
Improving physicians hand hygiene compliance
ndash most effective intervention
Pilot study 8 hospitals using QI methods Identified 24 causes
Different in each hospital
Interventions tailored to local cause
2nd study ndash web based tool Collect data about your causes (from 24) and used these
interventions to fix your local causes
8 Pilots compliance 48 81
Vs 174 organisations 58 84 (769
projects in
23
Variation reported in a UK study
24
232016
One lesson for research from case
Providers supplied data
Method enables easy local adaptation
But adaptations made are unknown
Effectiveness on average not the only knowledge needed by implementers
Need data to explain variations Not implemented
Context hinders
Adapt successfully or unsuccessfully
25
232016
More informative research
Experimental Trials Can and should we explain high and low outcomes
performers
Can we do subgroup analysis
Focus on non-experimental observational
studies
26
232016
Do you have any of these ldquoResearch
challengesrdquo Documenting
Use taxonomies of implementation methods
Use adaptation typology
Attributing Use Implementation outcome model
Use Logic Modelprogramme theory
Generalising ndash guidance Find which adaptations in which context get which results
Use (take up by practitioners)
= need Pre-study planning to get data for
27
Exercise
You are the head of a clinic
To enable physicians to follow antibiotic
prescribing guidelineshellip
Which behavioral change techniques would
be most effective
Would you use the same in FQC PHC and
Regan paediatrics outpatients 28
232016
Selection from 93 behavior change techniques (Michie 2013)
1 Instruction on how to perform a behavior
2 Punishment
3 Anticipation of future rewards or removal of punishment
4 Practice with Graded tasks
5 Mental rehearsal of successful performance
6 Examine Anticipated regret
7 Feedback on behavior
8 Other(s) monitoring with awareness
9 Social comparison
10 Behavioral contract
11 Social support
12 Identification of self as role model
29
232016
See also Powell 2015 73 implementation
strategies
Mixes direct interventions and indirect context
changes
Education
Audit and provide feedback
Create or change credentialing andor licensure
standards
Develop disincentives
Use capitated payments
Do we need to distinguish 30
232016
Document
Adaptions
1Who made the
modification
2What was modified
(Stirman 2009)
31
232016
Documenting critical context factors -
framworks CFIR (Damschroder 2009)
PHARIS (Rycroft-Malone 2002)
ORCA readiness - based on PHARIS
MUSIQ (Kaplan et al 2010) - QI
French et al 2009 (review of context measures
for evidence-based practice (EBP))
32
232016
Recommended practical tools ORCA (or CFIR httpwwwcfirguideorg)
33
change readiness (HRET)
4 areas the innovation target audience the organisation
the environment
34
Attributing ndash use causal chain frameworks Theory informed Case evaluation
35
232016
Programme theory models or logic models
36
232016
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Does not help with hellip
Changing academic research criteria Not improverrsquos questions
Do not reward ldquo3R researchrdquo relevant responsive
rigorous
Rather ldquo3P review criteriardquo for proposals publications
and promotion using RCT ideally
Settings where implementation is not
possible
but will know why 3
Relevance to researchers
if evaluating an intervention (eg new treatment or service delivery model)
knowledge of non-experimental methods which data to gather to document and evaluate
designing effective intervention (eg protocol for establishing an intervention to test)
Issues in translating research into practice and
scale-up ndash researcherrsquos role
4 232016
It was effective therehellip but different herehellip
5 232016
We donrsquot have a
problem here
Our patientsservice
is different
We have other changes
and priorities
Session covers
Case example - problems
Proven solutions ndash why not implemented
ldquoFitrdquo of Solution in the ldquoContextrdquo
Does the adaption work Adaptive implementation research
Research to answer improverrsquos questions
Implications for you
6 232016
What I mean byhellip
ldquoInterventionrdquo covers both
a)Do hand hygiene consistently (beforeafter change)
b) Training and feedback (actions to enable change)
Words
a) the ldquonew better wayrdquo - hand hygiene eg wash hands between patients use bar code reader to reduce medication errors rapid response
team to prevent avoidable deterioration on nursing units (The Improvement-change)
b) What we do directly to enable ldquotake uprdquo by staff of the
new better way (ldquoimplementationrdquo methods - training feedback rewards punishment for not using etc)
7 232016
8 232016
B Better hand
hygiene
A Poor hand
hygiene
Intervention Plan Wash hands between
patients Reduce barriers
Feedback
Implementation actions (phases) 1Education 2Gell dispensers 3Patient expectations 4Feedback compliance (infection rates)
- Patient
- Close carers
- Providers
How
Surrounding ldquocontextrdquo helps and hinders
Intervention concept
Wash hands between patients
Reduce barriers Feedback
What I mean byhellip
Conditions or context for the intervention
= influences which indirectly help or hinder the
intervention
high workload affects hand hygiene
disruption to supplies
IT system
how the hospital or physicians are paid
9 232016
232016 10
Mary 84 yrs Obstructive airways (COPD)
heart disease mild depression
Stable at home on meds
very independent
Unpaid motivational coach
and security-guard - ldquoMattyrdquo
Healthcare experience
Emergency hospital admission
Delays and meds changed
HAI
Sent home ndash no comms or support
Readmission
Beforeafter
11
232016
Mary - six weeks later
After hospitalisation
Avoidable cost 4600$
12
232016
Name a proven interventions for any problems
Changes of medswrong meds
Hospital acquired infection
After care transition
Medication reconcilliation
Hand hygiene - bundle of interventions
After-care - Coleman CTI ndash Transition coach and
education 13
232016
Your experience
We have implemented solutions to prevent this
- Yes No sometimes
We have effective Med Rec - Medications changes
unlikely
HAI from Hand Hygiene non-compliance unlikely
After care information to PCP ensured within 2
days of discharge
Effective after-care support for older multiple
morbidity patients
14
232016
Why not implemented - your vote
No problem (one-off event)
Not aware of these proven effective solutions
Know how to implement - donrsquot have time or
resources to implement
To implement effectively - donrsquot have knowledge
and skills
Know solutions - uncertain if we would get same
results - would it work here
Donrsquot have research to show how to implement in
different settings
15
Johns Questhellip
Why not implemented
What would help fast amp widespread
implementation
How can research best help
16
232016
Conclusions so farhellip
hellipin 2 slides
232016 17
Success depends onhellip
Seed Gardenerplanting amp nurture Soil climate
Idea 10
AdaptionImplementation 30
Personalities 20
The 10203040 change success rule
Soil receptive ndash staff
readiness
amp wider Climate 40
No intervention survives first contact with context
18
232016
Implemented as planned
Intervention plan
Adapt to survive
Adapt the improvement change tested
elsewhere
Eg turn at risk patients every 4hrs not 2hrs
(Pressure ulcer intervention)
Adapt the context
Increase staffing by 05 FTE
Both - Adapt the change amp context
Need ldquoGoldilocks Improvement-Fit for
Take Uprdquo GIFTU
19
Context ndash innovation ldquoFitrdquo
20
232016
Fidelity and adaptive implementation
t
21
232016
Adapt to ldquoFitrdquo to setting and subjects
Adjust over time ndash dynamicgtgtgtgtgt
To do this-Resources data and skills
Research questionshellip
Is the adaptation more or less effective
How can we find effective adaptations in
different settings
How can we do lower-cost faster research
on questions which will help take up
22
232016
Research ldquoCaserdquo example
Improving physicians hand hygiene compliance
ndash most effective intervention
Pilot study 8 hospitals using QI methods Identified 24 causes
Different in each hospital
Interventions tailored to local cause
2nd study ndash web based tool Collect data about your causes (from 24) and used these
interventions to fix your local causes
8 Pilots compliance 48 81
Vs 174 organisations 58 84 (769
projects in
23
Variation reported in a UK study
24
232016
One lesson for research from case
Providers supplied data
Method enables easy local adaptation
But adaptations made are unknown
Effectiveness on average not the only knowledge needed by implementers
Need data to explain variations Not implemented
Context hinders
Adapt successfully or unsuccessfully
25
232016
More informative research
Experimental Trials Can and should we explain high and low outcomes
performers
Can we do subgroup analysis
Focus on non-experimental observational
studies
26
232016
Do you have any of these ldquoResearch
challengesrdquo Documenting
Use taxonomies of implementation methods
Use adaptation typology
Attributing Use Implementation outcome model
Use Logic Modelprogramme theory
Generalising ndash guidance Find which adaptations in which context get which results
Use (take up by practitioners)
= need Pre-study planning to get data for
27
Exercise
You are the head of a clinic
To enable physicians to follow antibiotic
prescribing guidelineshellip
Which behavioral change techniques would
be most effective
Would you use the same in FQC PHC and
Regan paediatrics outpatients 28
232016
Selection from 93 behavior change techniques (Michie 2013)
1 Instruction on how to perform a behavior
2 Punishment
3 Anticipation of future rewards or removal of punishment
4 Practice with Graded tasks
5 Mental rehearsal of successful performance
6 Examine Anticipated regret
7 Feedback on behavior
8 Other(s) monitoring with awareness
9 Social comparison
10 Behavioral contract
11 Social support
12 Identification of self as role model
29
232016
See also Powell 2015 73 implementation
strategies
Mixes direct interventions and indirect context
changes
Education
Audit and provide feedback
Create or change credentialing andor licensure
standards
Develop disincentives
Use capitated payments
Do we need to distinguish 30
232016
Document
Adaptions
1Who made the
modification
2What was modified
(Stirman 2009)
31
232016
Documenting critical context factors -
framworks CFIR (Damschroder 2009)
PHARIS (Rycroft-Malone 2002)
ORCA readiness - based on PHARIS
MUSIQ (Kaplan et al 2010) - QI
French et al 2009 (review of context measures
for evidence-based practice (EBP))
32
232016
Recommended practical tools ORCA (or CFIR httpwwwcfirguideorg)
33
change readiness (HRET)
4 areas the innovation target audience the organisation
the environment
34
Attributing ndash use causal chain frameworks Theory informed Case evaluation
35
232016
Programme theory models or logic models
36
232016
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Relevance to researchers
if evaluating an intervention (eg new treatment or service delivery model)
knowledge of non-experimental methods which data to gather to document and evaluate
designing effective intervention (eg protocol for establishing an intervention to test)
Issues in translating research into practice and
scale-up ndash researcherrsquos role
4 232016
It was effective therehellip but different herehellip
5 232016
We donrsquot have a
problem here
Our patientsservice
is different
We have other changes
and priorities
Session covers
Case example - problems
Proven solutions ndash why not implemented
ldquoFitrdquo of Solution in the ldquoContextrdquo
Does the adaption work Adaptive implementation research
Research to answer improverrsquos questions
Implications for you
6 232016
What I mean byhellip
ldquoInterventionrdquo covers both
a)Do hand hygiene consistently (beforeafter change)
b) Training and feedback (actions to enable change)
Words
a) the ldquonew better wayrdquo - hand hygiene eg wash hands between patients use bar code reader to reduce medication errors rapid response
team to prevent avoidable deterioration on nursing units (The Improvement-change)
b) What we do directly to enable ldquotake uprdquo by staff of the
new better way (ldquoimplementationrdquo methods - training feedback rewards punishment for not using etc)
7 232016
8 232016
B Better hand
hygiene
A Poor hand
hygiene
Intervention Plan Wash hands between
patients Reduce barriers
Feedback
Implementation actions (phases) 1Education 2Gell dispensers 3Patient expectations 4Feedback compliance (infection rates)
- Patient
- Close carers
- Providers
How
Surrounding ldquocontextrdquo helps and hinders
Intervention concept
Wash hands between patients
Reduce barriers Feedback
What I mean byhellip
Conditions or context for the intervention
= influences which indirectly help or hinder the
intervention
high workload affects hand hygiene
disruption to supplies
IT system
how the hospital or physicians are paid
9 232016
232016 10
Mary 84 yrs Obstructive airways (COPD)
heart disease mild depression
Stable at home on meds
very independent
Unpaid motivational coach
and security-guard - ldquoMattyrdquo
Healthcare experience
Emergency hospital admission
Delays and meds changed
HAI
Sent home ndash no comms or support
Readmission
Beforeafter
11
232016
Mary - six weeks later
After hospitalisation
Avoidable cost 4600$
12
232016
Name a proven interventions for any problems
Changes of medswrong meds
Hospital acquired infection
After care transition
Medication reconcilliation
Hand hygiene - bundle of interventions
After-care - Coleman CTI ndash Transition coach and
education 13
232016
Your experience
We have implemented solutions to prevent this
- Yes No sometimes
We have effective Med Rec - Medications changes
unlikely
HAI from Hand Hygiene non-compliance unlikely
After care information to PCP ensured within 2
days of discharge
Effective after-care support for older multiple
morbidity patients
14
232016
Why not implemented - your vote
No problem (one-off event)
Not aware of these proven effective solutions
Know how to implement - donrsquot have time or
resources to implement
To implement effectively - donrsquot have knowledge
and skills
Know solutions - uncertain if we would get same
results - would it work here
Donrsquot have research to show how to implement in
different settings
15
Johns Questhellip
Why not implemented
What would help fast amp widespread
implementation
How can research best help
16
232016
Conclusions so farhellip
hellipin 2 slides
232016 17
Success depends onhellip
Seed Gardenerplanting amp nurture Soil climate
Idea 10
AdaptionImplementation 30
Personalities 20
The 10203040 change success rule
Soil receptive ndash staff
readiness
amp wider Climate 40
No intervention survives first contact with context
18
232016
Implemented as planned
Intervention plan
Adapt to survive
Adapt the improvement change tested
elsewhere
Eg turn at risk patients every 4hrs not 2hrs
(Pressure ulcer intervention)
Adapt the context
Increase staffing by 05 FTE
Both - Adapt the change amp context
Need ldquoGoldilocks Improvement-Fit for
Take Uprdquo GIFTU
19
Context ndash innovation ldquoFitrdquo
20
232016
Fidelity and adaptive implementation
t
21
232016
Adapt to ldquoFitrdquo to setting and subjects
Adjust over time ndash dynamicgtgtgtgtgt
To do this-Resources data and skills
Research questionshellip
Is the adaptation more or less effective
How can we find effective adaptations in
different settings
How can we do lower-cost faster research
on questions which will help take up
22
232016
Research ldquoCaserdquo example
Improving physicians hand hygiene compliance
ndash most effective intervention
Pilot study 8 hospitals using QI methods Identified 24 causes
Different in each hospital
Interventions tailored to local cause
2nd study ndash web based tool Collect data about your causes (from 24) and used these
interventions to fix your local causes
8 Pilots compliance 48 81
Vs 174 organisations 58 84 (769
projects in
23
Variation reported in a UK study
24
232016
One lesson for research from case
Providers supplied data
Method enables easy local adaptation
But adaptations made are unknown
Effectiveness on average not the only knowledge needed by implementers
Need data to explain variations Not implemented
Context hinders
Adapt successfully or unsuccessfully
25
232016
More informative research
Experimental Trials Can and should we explain high and low outcomes
performers
Can we do subgroup analysis
Focus on non-experimental observational
studies
26
232016
Do you have any of these ldquoResearch
challengesrdquo Documenting
Use taxonomies of implementation methods
Use adaptation typology
Attributing Use Implementation outcome model
Use Logic Modelprogramme theory
Generalising ndash guidance Find which adaptations in which context get which results
Use (take up by practitioners)
= need Pre-study planning to get data for
27
Exercise
You are the head of a clinic
To enable physicians to follow antibiotic
prescribing guidelineshellip
Which behavioral change techniques would
be most effective
Would you use the same in FQC PHC and
Regan paediatrics outpatients 28
232016
Selection from 93 behavior change techniques (Michie 2013)
1 Instruction on how to perform a behavior
2 Punishment
3 Anticipation of future rewards or removal of punishment
4 Practice with Graded tasks
5 Mental rehearsal of successful performance
6 Examine Anticipated regret
7 Feedback on behavior
8 Other(s) monitoring with awareness
9 Social comparison
10 Behavioral contract
11 Social support
12 Identification of self as role model
29
232016
See also Powell 2015 73 implementation
strategies
Mixes direct interventions and indirect context
changes
Education
Audit and provide feedback
Create or change credentialing andor licensure
standards
Develop disincentives
Use capitated payments
Do we need to distinguish 30
232016
Document
Adaptions
1Who made the
modification
2What was modified
(Stirman 2009)
31
232016
Documenting critical context factors -
framworks CFIR (Damschroder 2009)
PHARIS (Rycroft-Malone 2002)
ORCA readiness - based on PHARIS
MUSIQ (Kaplan et al 2010) - QI
French et al 2009 (review of context measures
for evidence-based practice (EBP))
32
232016
Recommended practical tools ORCA (or CFIR httpwwwcfirguideorg)
33
change readiness (HRET)
4 areas the innovation target audience the organisation
the environment
34
Attributing ndash use causal chain frameworks Theory informed Case evaluation
35
232016
Programme theory models or logic models
36
232016
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
It was effective therehellip but different herehellip
5 232016
We donrsquot have a
problem here
Our patientsservice
is different
We have other changes
and priorities
Session covers
Case example - problems
Proven solutions ndash why not implemented
ldquoFitrdquo of Solution in the ldquoContextrdquo
Does the adaption work Adaptive implementation research
Research to answer improverrsquos questions
Implications for you
6 232016
What I mean byhellip
ldquoInterventionrdquo covers both
a)Do hand hygiene consistently (beforeafter change)
b) Training and feedback (actions to enable change)
Words
a) the ldquonew better wayrdquo - hand hygiene eg wash hands between patients use bar code reader to reduce medication errors rapid response
team to prevent avoidable deterioration on nursing units (The Improvement-change)
b) What we do directly to enable ldquotake uprdquo by staff of the
new better way (ldquoimplementationrdquo methods - training feedback rewards punishment for not using etc)
7 232016
8 232016
B Better hand
hygiene
A Poor hand
hygiene
Intervention Plan Wash hands between
patients Reduce barriers
Feedback
Implementation actions (phases) 1Education 2Gell dispensers 3Patient expectations 4Feedback compliance (infection rates)
- Patient
- Close carers
- Providers
How
Surrounding ldquocontextrdquo helps and hinders
Intervention concept
Wash hands between patients
Reduce barriers Feedback
What I mean byhellip
Conditions or context for the intervention
= influences which indirectly help or hinder the
intervention
high workload affects hand hygiene
disruption to supplies
IT system
how the hospital or physicians are paid
9 232016
232016 10
Mary 84 yrs Obstructive airways (COPD)
heart disease mild depression
Stable at home on meds
very independent
Unpaid motivational coach
and security-guard - ldquoMattyrdquo
Healthcare experience
Emergency hospital admission
Delays and meds changed
HAI
Sent home ndash no comms or support
Readmission
Beforeafter
11
232016
Mary - six weeks later
After hospitalisation
Avoidable cost 4600$
12
232016
Name a proven interventions for any problems
Changes of medswrong meds
Hospital acquired infection
After care transition
Medication reconcilliation
Hand hygiene - bundle of interventions
After-care - Coleman CTI ndash Transition coach and
education 13
232016
Your experience
We have implemented solutions to prevent this
- Yes No sometimes
We have effective Med Rec - Medications changes
unlikely
HAI from Hand Hygiene non-compliance unlikely
After care information to PCP ensured within 2
days of discharge
Effective after-care support for older multiple
morbidity patients
14
232016
Why not implemented - your vote
No problem (one-off event)
Not aware of these proven effective solutions
Know how to implement - donrsquot have time or
resources to implement
To implement effectively - donrsquot have knowledge
and skills
Know solutions - uncertain if we would get same
results - would it work here
Donrsquot have research to show how to implement in
different settings
15
Johns Questhellip
Why not implemented
What would help fast amp widespread
implementation
How can research best help
16
232016
Conclusions so farhellip
hellipin 2 slides
232016 17
Success depends onhellip
Seed Gardenerplanting amp nurture Soil climate
Idea 10
AdaptionImplementation 30
Personalities 20
The 10203040 change success rule
Soil receptive ndash staff
readiness
amp wider Climate 40
No intervention survives first contact with context
18
232016
Implemented as planned
Intervention plan
Adapt to survive
Adapt the improvement change tested
elsewhere
Eg turn at risk patients every 4hrs not 2hrs
(Pressure ulcer intervention)
Adapt the context
Increase staffing by 05 FTE
Both - Adapt the change amp context
Need ldquoGoldilocks Improvement-Fit for
Take Uprdquo GIFTU
19
Context ndash innovation ldquoFitrdquo
20
232016
Fidelity and adaptive implementation
t
21
232016
Adapt to ldquoFitrdquo to setting and subjects
Adjust over time ndash dynamicgtgtgtgtgt
To do this-Resources data and skills
Research questionshellip
Is the adaptation more or less effective
How can we find effective adaptations in
different settings
How can we do lower-cost faster research
on questions which will help take up
22
232016
Research ldquoCaserdquo example
Improving physicians hand hygiene compliance
ndash most effective intervention
Pilot study 8 hospitals using QI methods Identified 24 causes
Different in each hospital
Interventions tailored to local cause
2nd study ndash web based tool Collect data about your causes (from 24) and used these
interventions to fix your local causes
8 Pilots compliance 48 81
Vs 174 organisations 58 84 (769
projects in
23
Variation reported in a UK study
24
232016
One lesson for research from case
Providers supplied data
Method enables easy local adaptation
But adaptations made are unknown
Effectiveness on average not the only knowledge needed by implementers
Need data to explain variations Not implemented
Context hinders
Adapt successfully or unsuccessfully
25
232016
More informative research
Experimental Trials Can and should we explain high and low outcomes
performers
Can we do subgroup analysis
Focus on non-experimental observational
studies
26
232016
Do you have any of these ldquoResearch
challengesrdquo Documenting
Use taxonomies of implementation methods
Use adaptation typology
Attributing Use Implementation outcome model
Use Logic Modelprogramme theory
Generalising ndash guidance Find which adaptations in which context get which results
Use (take up by practitioners)
= need Pre-study planning to get data for
27
Exercise
You are the head of a clinic
To enable physicians to follow antibiotic
prescribing guidelineshellip
Which behavioral change techniques would
be most effective
Would you use the same in FQC PHC and
Regan paediatrics outpatients 28
232016
Selection from 93 behavior change techniques (Michie 2013)
1 Instruction on how to perform a behavior
2 Punishment
3 Anticipation of future rewards or removal of punishment
4 Practice with Graded tasks
5 Mental rehearsal of successful performance
6 Examine Anticipated regret
7 Feedback on behavior
8 Other(s) monitoring with awareness
9 Social comparison
10 Behavioral contract
11 Social support
12 Identification of self as role model
29
232016
See also Powell 2015 73 implementation
strategies
Mixes direct interventions and indirect context
changes
Education
Audit and provide feedback
Create or change credentialing andor licensure
standards
Develop disincentives
Use capitated payments
Do we need to distinguish 30
232016
Document
Adaptions
1Who made the
modification
2What was modified
(Stirman 2009)
31
232016
Documenting critical context factors -
framworks CFIR (Damschroder 2009)
PHARIS (Rycroft-Malone 2002)
ORCA readiness - based on PHARIS
MUSIQ (Kaplan et al 2010) - QI
French et al 2009 (review of context measures
for evidence-based practice (EBP))
32
232016
Recommended practical tools ORCA (or CFIR httpwwwcfirguideorg)
33
change readiness (HRET)
4 areas the innovation target audience the organisation
the environment
34
Attributing ndash use causal chain frameworks Theory informed Case evaluation
35
232016
Programme theory models or logic models
36
232016
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Session covers
Case example - problems
Proven solutions ndash why not implemented
ldquoFitrdquo of Solution in the ldquoContextrdquo
Does the adaption work Adaptive implementation research
Research to answer improverrsquos questions
Implications for you
6 232016
What I mean byhellip
ldquoInterventionrdquo covers both
a)Do hand hygiene consistently (beforeafter change)
b) Training and feedback (actions to enable change)
Words
a) the ldquonew better wayrdquo - hand hygiene eg wash hands between patients use bar code reader to reduce medication errors rapid response
team to prevent avoidable deterioration on nursing units (The Improvement-change)
b) What we do directly to enable ldquotake uprdquo by staff of the
new better way (ldquoimplementationrdquo methods - training feedback rewards punishment for not using etc)
7 232016
8 232016
B Better hand
hygiene
A Poor hand
hygiene
Intervention Plan Wash hands between
patients Reduce barriers
Feedback
Implementation actions (phases) 1Education 2Gell dispensers 3Patient expectations 4Feedback compliance (infection rates)
- Patient
- Close carers
- Providers
How
Surrounding ldquocontextrdquo helps and hinders
Intervention concept
Wash hands between patients
Reduce barriers Feedback
What I mean byhellip
Conditions or context for the intervention
= influences which indirectly help or hinder the
intervention
high workload affects hand hygiene
disruption to supplies
IT system
how the hospital or physicians are paid
9 232016
232016 10
Mary 84 yrs Obstructive airways (COPD)
heart disease mild depression
Stable at home on meds
very independent
Unpaid motivational coach
and security-guard - ldquoMattyrdquo
Healthcare experience
Emergency hospital admission
Delays and meds changed
HAI
Sent home ndash no comms or support
Readmission
Beforeafter
11
232016
Mary - six weeks later
After hospitalisation
Avoidable cost 4600$
12
232016
Name a proven interventions for any problems
Changes of medswrong meds
Hospital acquired infection
After care transition
Medication reconcilliation
Hand hygiene - bundle of interventions
After-care - Coleman CTI ndash Transition coach and
education 13
232016
Your experience
We have implemented solutions to prevent this
- Yes No sometimes
We have effective Med Rec - Medications changes
unlikely
HAI from Hand Hygiene non-compliance unlikely
After care information to PCP ensured within 2
days of discharge
Effective after-care support for older multiple
morbidity patients
14
232016
Why not implemented - your vote
No problem (one-off event)
Not aware of these proven effective solutions
Know how to implement - donrsquot have time or
resources to implement
To implement effectively - donrsquot have knowledge
and skills
Know solutions - uncertain if we would get same
results - would it work here
Donrsquot have research to show how to implement in
different settings
15
Johns Questhellip
Why not implemented
What would help fast amp widespread
implementation
How can research best help
16
232016
Conclusions so farhellip
hellipin 2 slides
232016 17
Success depends onhellip
Seed Gardenerplanting amp nurture Soil climate
Idea 10
AdaptionImplementation 30
Personalities 20
The 10203040 change success rule
Soil receptive ndash staff
readiness
amp wider Climate 40
No intervention survives first contact with context
18
232016
Implemented as planned
Intervention plan
Adapt to survive
Adapt the improvement change tested
elsewhere
Eg turn at risk patients every 4hrs not 2hrs
(Pressure ulcer intervention)
Adapt the context
Increase staffing by 05 FTE
Both - Adapt the change amp context
Need ldquoGoldilocks Improvement-Fit for
Take Uprdquo GIFTU
19
Context ndash innovation ldquoFitrdquo
20
232016
Fidelity and adaptive implementation
t
21
232016
Adapt to ldquoFitrdquo to setting and subjects
Adjust over time ndash dynamicgtgtgtgtgt
To do this-Resources data and skills
Research questionshellip
Is the adaptation more or less effective
How can we find effective adaptations in
different settings
How can we do lower-cost faster research
on questions which will help take up
22
232016
Research ldquoCaserdquo example
Improving physicians hand hygiene compliance
ndash most effective intervention
Pilot study 8 hospitals using QI methods Identified 24 causes
Different in each hospital
Interventions tailored to local cause
2nd study ndash web based tool Collect data about your causes (from 24) and used these
interventions to fix your local causes
8 Pilots compliance 48 81
Vs 174 organisations 58 84 (769
projects in
23
Variation reported in a UK study
24
232016
One lesson for research from case
Providers supplied data
Method enables easy local adaptation
But adaptations made are unknown
Effectiveness on average not the only knowledge needed by implementers
Need data to explain variations Not implemented
Context hinders
Adapt successfully or unsuccessfully
25
232016
More informative research
Experimental Trials Can and should we explain high and low outcomes
performers
Can we do subgroup analysis
Focus on non-experimental observational
studies
26
232016
Do you have any of these ldquoResearch
challengesrdquo Documenting
Use taxonomies of implementation methods
Use adaptation typology
Attributing Use Implementation outcome model
Use Logic Modelprogramme theory
Generalising ndash guidance Find which adaptations in which context get which results
Use (take up by practitioners)
= need Pre-study planning to get data for
27
Exercise
You are the head of a clinic
To enable physicians to follow antibiotic
prescribing guidelineshellip
Which behavioral change techniques would
be most effective
Would you use the same in FQC PHC and
Regan paediatrics outpatients 28
232016
Selection from 93 behavior change techniques (Michie 2013)
1 Instruction on how to perform a behavior
2 Punishment
3 Anticipation of future rewards or removal of punishment
4 Practice with Graded tasks
5 Mental rehearsal of successful performance
6 Examine Anticipated regret
7 Feedback on behavior
8 Other(s) monitoring with awareness
9 Social comparison
10 Behavioral contract
11 Social support
12 Identification of self as role model
29
232016
See also Powell 2015 73 implementation
strategies
Mixes direct interventions and indirect context
changes
Education
Audit and provide feedback
Create or change credentialing andor licensure
standards
Develop disincentives
Use capitated payments
Do we need to distinguish 30
232016
Document
Adaptions
1Who made the
modification
2What was modified
(Stirman 2009)
31
232016
Documenting critical context factors -
framworks CFIR (Damschroder 2009)
PHARIS (Rycroft-Malone 2002)
ORCA readiness - based on PHARIS
MUSIQ (Kaplan et al 2010) - QI
French et al 2009 (review of context measures
for evidence-based practice (EBP))
32
232016
Recommended practical tools ORCA (or CFIR httpwwwcfirguideorg)
33
change readiness (HRET)
4 areas the innovation target audience the organisation
the environment
34
Attributing ndash use causal chain frameworks Theory informed Case evaluation
35
232016
Programme theory models or logic models
36
232016
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
What I mean byhellip
ldquoInterventionrdquo covers both
a)Do hand hygiene consistently (beforeafter change)
b) Training and feedback (actions to enable change)
Words
a) the ldquonew better wayrdquo - hand hygiene eg wash hands between patients use bar code reader to reduce medication errors rapid response
team to prevent avoidable deterioration on nursing units (The Improvement-change)
b) What we do directly to enable ldquotake uprdquo by staff of the
new better way (ldquoimplementationrdquo methods - training feedback rewards punishment for not using etc)
7 232016
8 232016
B Better hand
hygiene
A Poor hand
hygiene
Intervention Plan Wash hands between
patients Reduce barriers
Feedback
Implementation actions (phases) 1Education 2Gell dispensers 3Patient expectations 4Feedback compliance (infection rates)
- Patient
- Close carers
- Providers
How
Surrounding ldquocontextrdquo helps and hinders
Intervention concept
Wash hands between patients
Reduce barriers Feedback
What I mean byhellip
Conditions or context for the intervention
= influences which indirectly help or hinder the
intervention
high workload affects hand hygiene
disruption to supplies
IT system
how the hospital or physicians are paid
9 232016
232016 10
Mary 84 yrs Obstructive airways (COPD)
heart disease mild depression
Stable at home on meds
very independent
Unpaid motivational coach
and security-guard - ldquoMattyrdquo
Healthcare experience
Emergency hospital admission
Delays and meds changed
HAI
Sent home ndash no comms or support
Readmission
Beforeafter
11
232016
Mary - six weeks later
After hospitalisation
Avoidable cost 4600$
12
232016
Name a proven interventions for any problems
Changes of medswrong meds
Hospital acquired infection
After care transition
Medication reconcilliation
Hand hygiene - bundle of interventions
After-care - Coleman CTI ndash Transition coach and
education 13
232016
Your experience
We have implemented solutions to prevent this
- Yes No sometimes
We have effective Med Rec - Medications changes
unlikely
HAI from Hand Hygiene non-compliance unlikely
After care information to PCP ensured within 2
days of discharge
Effective after-care support for older multiple
morbidity patients
14
232016
Why not implemented - your vote
No problem (one-off event)
Not aware of these proven effective solutions
Know how to implement - donrsquot have time or
resources to implement
To implement effectively - donrsquot have knowledge
and skills
Know solutions - uncertain if we would get same
results - would it work here
Donrsquot have research to show how to implement in
different settings
15
Johns Questhellip
Why not implemented
What would help fast amp widespread
implementation
How can research best help
16
232016
Conclusions so farhellip
hellipin 2 slides
232016 17
Success depends onhellip
Seed Gardenerplanting amp nurture Soil climate
Idea 10
AdaptionImplementation 30
Personalities 20
The 10203040 change success rule
Soil receptive ndash staff
readiness
amp wider Climate 40
No intervention survives first contact with context
18
232016
Implemented as planned
Intervention plan
Adapt to survive
Adapt the improvement change tested
elsewhere
Eg turn at risk patients every 4hrs not 2hrs
(Pressure ulcer intervention)
Adapt the context
Increase staffing by 05 FTE
Both - Adapt the change amp context
Need ldquoGoldilocks Improvement-Fit for
Take Uprdquo GIFTU
19
Context ndash innovation ldquoFitrdquo
20
232016
Fidelity and adaptive implementation
t
21
232016
Adapt to ldquoFitrdquo to setting and subjects
Adjust over time ndash dynamicgtgtgtgtgt
To do this-Resources data and skills
Research questionshellip
Is the adaptation more or less effective
How can we find effective adaptations in
different settings
How can we do lower-cost faster research
on questions which will help take up
22
232016
Research ldquoCaserdquo example
Improving physicians hand hygiene compliance
ndash most effective intervention
Pilot study 8 hospitals using QI methods Identified 24 causes
Different in each hospital
Interventions tailored to local cause
2nd study ndash web based tool Collect data about your causes (from 24) and used these
interventions to fix your local causes
8 Pilots compliance 48 81
Vs 174 organisations 58 84 (769
projects in
23
Variation reported in a UK study
24
232016
One lesson for research from case
Providers supplied data
Method enables easy local adaptation
But adaptations made are unknown
Effectiveness on average not the only knowledge needed by implementers
Need data to explain variations Not implemented
Context hinders
Adapt successfully or unsuccessfully
25
232016
More informative research
Experimental Trials Can and should we explain high and low outcomes
performers
Can we do subgroup analysis
Focus on non-experimental observational
studies
26
232016
Do you have any of these ldquoResearch
challengesrdquo Documenting
Use taxonomies of implementation methods
Use adaptation typology
Attributing Use Implementation outcome model
Use Logic Modelprogramme theory
Generalising ndash guidance Find which adaptations in which context get which results
Use (take up by practitioners)
= need Pre-study planning to get data for
27
Exercise
You are the head of a clinic
To enable physicians to follow antibiotic
prescribing guidelineshellip
Which behavioral change techniques would
be most effective
Would you use the same in FQC PHC and
Regan paediatrics outpatients 28
232016
Selection from 93 behavior change techniques (Michie 2013)
1 Instruction on how to perform a behavior
2 Punishment
3 Anticipation of future rewards or removal of punishment
4 Practice with Graded tasks
5 Mental rehearsal of successful performance
6 Examine Anticipated regret
7 Feedback on behavior
8 Other(s) monitoring with awareness
9 Social comparison
10 Behavioral contract
11 Social support
12 Identification of self as role model
29
232016
See also Powell 2015 73 implementation
strategies
Mixes direct interventions and indirect context
changes
Education
Audit and provide feedback
Create or change credentialing andor licensure
standards
Develop disincentives
Use capitated payments
Do we need to distinguish 30
232016
Document
Adaptions
1Who made the
modification
2What was modified
(Stirman 2009)
31
232016
Documenting critical context factors -
framworks CFIR (Damschroder 2009)
PHARIS (Rycroft-Malone 2002)
ORCA readiness - based on PHARIS
MUSIQ (Kaplan et al 2010) - QI
French et al 2009 (review of context measures
for evidence-based practice (EBP))
32
232016
Recommended practical tools ORCA (or CFIR httpwwwcfirguideorg)
33
change readiness (HRET)
4 areas the innovation target audience the organisation
the environment
34
Attributing ndash use causal chain frameworks Theory informed Case evaluation
35
232016
Programme theory models or logic models
36
232016
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
8 232016
B Better hand
hygiene
A Poor hand
hygiene
Intervention Plan Wash hands between
patients Reduce barriers
Feedback
Implementation actions (phases) 1Education 2Gell dispensers 3Patient expectations 4Feedback compliance (infection rates)
- Patient
- Close carers
- Providers
How
Surrounding ldquocontextrdquo helps and hinders
Intervention concept
Wash hands between patients
Reduce barriers Feedback
What I mean byhellip
Conditions or context for the intervention
= influences which indirectly help or hinder the
intervention
high workload affects hand hygiene
disruption to supplies
IT system
how the hospital or physicians are paid
9 232016
232016 10
Mary 84 yrs Obstructive airways (COPD)
heart disease mild depression
Stable at home on meds
very independent
Unpaid motivational coach
and security-guard - ldquoMattyrdquo
Healthcare experience
Emergency hospital admission
Delays and meds changed
HAI
Sent home ndash no comms or support
Readmission
Beforeafter
11
232016
Mary - six weeks later
After hospitalisation
Avoidable cost 4600$
12
232016
Name a proven interventions for any problems
Changes of medswrong meds
Hospital acquired infection
After care transition
Medication reconcilliation
Hand hygiene - bundle of interventions
After-care - Coleman CTI ndash Transition coach and
education 13
232016
Your experience
We have implemented solutions to prevent this
- Yes No sometimes
We have effective Med Rec - Medications changes
unlikely
HAI from Hand Hygiene non-compliance unlikely
After care information to PCP ensured within 2
days of discharge
Effective after-care support for older multiple
morbidity patients
14
232016
Why not implemented - your vote
No problem (one-off event)
Not aware of these proven effective solutions
Know how to implement - donrsquot have time or
resources to implement
To implement effectively - donrsquot have knowledge
and skills
Know solutions - uncertain if we would get same
results - would it work here
Donrsquot have research to show how to implement in
different settings
15
Johns Questhellip
Why not implemented
What would help fast amp widespread
implementation
How can research best help
16
232016
Conclusions so farhellip
hellipin 2 slides
232016 17
Success depends onhellip
Seed Gardenerplanting amp nurture Soil climate
Idea 10
AdaptionImplementation 30
Personalities 20
The 10203040 change success rule
Soil receptive ndash staff
readiness
amp wider Climate 40
No intervention survives first contact with context
18
232016
Implemented as planned
Intervention plan
Adapt to survive
Adapt the improvement change tested
elsewhere
Eg turn at risk patients every 4hrs not 2hrs
(Pressure ulcer intervention)
Adapt the context
Increase staffing by 05 FTE
Both - Adapt the change amp context
Need ldquoGoldilocks Improvement-Fit for
Take Uprdquo GIFTU
19
Context ndash innovation ldquoFitrdquo
20
232016
Fidelity and adaptive implementation
t
21
232016
Adapt to ldquoFitrdquo to setting and subjects
Adjust over time ndash dynamicgtgtgtgtgt
To do this-Resources data and skills
Research questionshellip
Is the adaptation more or less effective
How can we find effective adaptations in
different settings
How can we do lower-cost faster research
on questions which will help take up
22
232016
Research ldquoCaserdquo example
Improving physicians hand hygiene compliance
ndash most effective intervention
Pilot study 8 hospitals using QI methods Identified 24 causes
Different in each hospital
Interventions tailored to local cause
2nd study ndash web based tool Collect data about your causes (from 24) and used these
interventions to fix your local causes
8 Pilots compliance 48 81
Vs 174 organisations 58 84 (769
projects in
23
Variation reported in a UK study
24
232016
One lesson for research from case
Providers supplied data
Method enables easy local adaptation
But adaptations made are unknown
Effectiveness on average not the only knowledge needed by implementers
Need data to explain variations Not implemented
Context hinders
Adapt successfully or unsuccessfully
25
232016
More informative research
Experimental Trials Can and should we explain high and low outcomes
performers
Can we do subgroup analysis
Focus on non-experimental observational
studies
26
232016
Do you have any of these ldquoResearch
challengesrdquo Documenting
Use taxonomies of implementation methods
Use adaptation typology
Attributing Use Implementation outcome model
Use Logic Modelprogramme theory
Generalising ndash guidance Find which adaptations in which context get which results
Use (take up by practitioners)
= need Pre-study planning to get data for
27
Exercise
You are the head of a clinic
To enable physicians to follow antibiotic
prescribing guidelineshellip
Which behavioral change techniques would
be most effective
Would you use the same in FQC PHC and
Regan paediatrics outpatients 28
232016
Selection from 93 behavior change techniques (Michie 2013)
1 Instruction on how to perform a behavior
2 Punishment
3 Anticipation of future rewards or removal of punishment
4 Practice with Graded tasks
5 Mental rehearsal of successful performance
6 Examine Anticipated regret
7 Feedback on behavior
8 Other(s) monitoring with awareness
9 Social comparison
10 Behavioral contract
11 Social support
12 Identification of self as role model
29
232016
See also Powell 2015 73 implementation
strategies
Mixes direct interventions and indirect context
changes
Education
Audit and provide feedback
Create or change credentialing andor licensure
standards
Develop disincentives
Use capitated payments
Do we need to distinguish 30
232016
Document
Adaptions
1Who made the
modification
2What was modified
(Stirman 2009)
31
232016
Documenting critical context factors -
framworks CFIR (Damschroder 2009)
PHARIS (Rycroft-Malone 2002)
ORCA readiness - based on PHARIS
MUSIQ (Kaplan et al 2010) - QI
French et al 2009 (review of context measures
for evidence-based practice (EBP))
32
232016
Recommended practical tools ORCA (or CFIR httpwwwcfirguideorg)
33
change readiness (HRET)
4 areas the innovation target audience the organisation
the environment
34
Attributing ndash use causal chain frameworks Theory informed Case evaluation
35
232016
Programme theory models or logic models
36
232016
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
What I mean byhellip
Conditions or context for the intervention
= influences which indirectly help or hinder the
intervention
high workload affects hand hygiene
disruption to supplies
IT system
how the hospital or physicians are paid
9 232016
232016 10
Mary 84 yrs Obstructive airways (COPD)
heart disease mild depression
Stable at home on meds
very independent
Unpaid motivational coach
and security-guard - ldquoMattyrdquo
Healthcare experience
Emergency hospital admission
Delays and meds changed
HAI
Sent home ndash no comms or support
Readmission
Beforeafter
11
232016
Mary - six weeks later
After hospitalisation
Avoidable cost 4600$
12
232016
Name a proven interventions for any problems
Changes of medswrong meds
Hospital acquired infection
After care transition
Medication reconcilliation
Hand hygiene - bundle of interventions
After-care - Coleman CTI ndash Transition coach and
education 13
232016
Your experience
We have implemented solutions to prevent this
- Yes No sometimes
We have effective Med Rec - Medications changes
unlikely
HAI from Hand Hygiene non-compliance unlikely
After care information to PCP ensured within 2
days of discharge
Effective after-care support for older multiple
morbidity patients
14
232016
Why not implemented - your vote
No problem (one-off event)
Not aware of these proven effective solutions
Know how to implement - donrsquot have time or
resources to implement
To implement effectively - donrsquot have knowledge
and skills
Know solutions - uncertain if we would get same
results - would it work here
Donrsquot have research to show how to implement in
different settings
15
Johns Questhellip
Why not implemented
What would help fast amp widespread
implementation
How can research best help
16
232016
Conclusions so farhellip
hellipin 2 slides
232016 17
Success depends onhellip
Seed Gardenerplanting amp nurture Soil climate
Idea 10
AdaptionImplementation 30
Personalities 20
The 10203040 change success rule
Soil receptive ndash staff
readiness
amp wider Climate 40
No intervention survives first contact with context
18
232016
Implemented as planned
Intervention plan
Adapt to survive
Adapt the improvement change tested
elsewhere
Eg turn at risk patients every 4hrs not 2hrs
(Pressure ulcer intervention)
Adapt the context
Increase staffing by 05 FTE
Both - Adapt the change amp context
Need ldquoGoldilocks Improvement-Fit for
Take Uprdquo GIFTU
19
Context ndash innovation ldquoFitrdquo
20
232016
Fidelity and adaptive implementation
t
21
232016
Adapt to ldquoFitrdquo to setting and subjects
Adjust over time ndash dynamicgtgtgtgtgt
To do this-Resources data and skills
Research questionshellip
Is the adaptation more or less effective
How can we find effective adaptations in
different settings
How can we do lower-cost faster research
on questions which will help take up
22
232016
Research ldquoCaserdquo example
Improving physicians hand hygiene compliance
ndash most effective intervention
Pilot study 8 hospitals using QI methods Identified 24 causes
Different in each hospital
Interventions tailored to local cause
2nd study ndash web based tool Collect data about your causes (from 24) and used these
interventions to fix your local causes
8 Pilots compliance 48 81
Vs 174 organisations 58 84 (769
projects in
23
Variation reported in a UK study
24
232016
One lesson for research from case
Providers supplied data
Method enables easy local adaptation
But adaptations made are unknown
Effectiveness on average not the only knowledge needed by implementers
Need data to explain variations Not implemented
Context hinders
Adapt successfully or unsuccessfully
25
232016
More informative research
Experimental Trials Can and should we explain high and low outcomes
performers
Can we do subgroup analysis
Focus on non-experimental observational
studies
26
232016
Do you have any of these ldquoResearch
challengesrdquo Documenting
Use taxonomies of implementation methods
Use adaptation typology
Attributing Use Implementation outcome model
Use Logic Modelprogramme theory
Generalising ndash guidance Find which adaptations in which context get which results
Use (take up by practitioners)
= need Pre-study planning to get data for
27
Exercise
You are the head of a clinic
To enable physicians to follow antibiotic
prescribing guidelineshellip
Which behavioral change techniques would
be most effective
Would you use the same in FQC PHC and
Regan paediatrics outpatients 28
232016
Selection from 93 behavior change techniques (Michie 2013)
1 Instruction on how to perform a behavior
2 Punishment
3 Anticipation of future rewards or removal of punishment
4 Practice with Graded tasks
5 Mental rehearsal of successful performance
6 Examine Anticipated regret
7 Feedback on behavior
8 Other(s) monitoring with awareness
9 Social comparison
10 Behavioral contract
11 Social support
12 Identification of self as role model
29
232016
See also Powell 2015 73 implementation
strategies
Mixes direct interventions and indirect context
changes
Education
Audit and provide feedback
Create or change credentialing andor licensure
standards
Develop disincentives
Use capitated payments
Do we need to distinguish 30
232016
Document
Adaptions
1Who made the
modification
2What was modified
(Stirman 2009)
31
232016
Documenting critical context factors -
framworks CFIR (Damschroder 2009)
PHARIS (Rycroft-Malone 2002)
ORCA readiness - based on PHARIS
MUSIQ (Kaplan et al 2010) - QI
French et al 2009 (review of context measures
for evidence-based practice (EBP))
32
232016
Recommended practical tools ORCA (or CFIR httpwwwcfirguideorg)
33
change readiness (HRET)
4 areas the innovation target audience the organisation
the environment
34
Attributing ndash use causal chain frameworks Theory informed Case evaluation
35
232016
Programme theory models or logic models
36
232016
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
232016 10
Mary 84 yrs Obstructive airways (COPD)
heart disease mild depression
Stable at home on meds
very independent
Unpaid motivational coach
and security-guard - ldquoMattyrdquo
Healthcare experience
Emergency hospital admission
Delays and meds changed
HAI
Sent home ndash no comms or support
Readmission
Beforeafter
11
232016
Mary - six weeks later
After hospitalisation
Avoidable cost 4600$
12
232016
Name a proven interventions for any problems
Changes of medswrong meds
Hospital acquired infection
After care transition
Medication reconcilliation
Hand hygiene - bundle of interventions
After-care - Coleman CTI ndash Transition coach and
education 13
232016
Your experience
We have implemented solutions to prevent this
- Yes No sometimes
We have effective Med Rec - Medications changes
unlikely
HAI from Hand Hygiene non-compliance unlikely
After care information to PCP ensured within 2
days of discharge
Effective after-care support for older multiple
morbidity patients
14
232016
Why not implemented - your vote
No problem (one-off event)
Not aware of these proven effective solutions
Know how to implement - donrsquot have time or
resources to implement
To implement effectively - donrsquot have knowledge
and skills
Know solutions - uncertain if we would get same
results - would it work here
Donrsquot have research to show how to implement in
different settings
15
Johns Questhellip
Why not implemented
What would help fast amp widespread
implementation
How can research best help
16
232016
Conclusions so farhellip
hellipin 2 slides
232016 17
Success depends onhellip
Seed Gardenerplanting amp nurture Soil climate
Idea 10
AdaptionImplementation 30
Personalities 20
The 10203040 change success rule
Soil receptive ndash staff
readiness
amp wider Climate 40
No intervention survives first contact with context
18
232016
Implemented as planned
Intervention plan
Adapt to survive
Adapt the improvement change tested
elsewhere
Eg turn at risk patients every 4hrs not 2hrs
(Pressure ulcer intervention)
Adapt the context
Increase staffing by 05 FTE
Both - Adapt the change amp context
Need ldquoGoldilocks Improvement-Fit for
Take Uprdquo GIFTU
19
Context ndash innovation ldquoFitrdquo
20
232016
Fidelity and adaptive implementation
t
21
232016
Adapt to ldquoFitrdquo to setting and subjects
Adjust over time ndash dynamicgtgtgtgtgt
To do this-Resources data and skills
Research questionshellip
Is the adaptation more or less effective
How can we find effective adaptations in
different settings
How can we do lower-cost faster research
on questions which will help take up
22
232016
Research ldquoCaserdquo example
Improving physicians hand hygiene compliance
ndash most effective intervention
Pilot study 8 hospitals using QI methods Identified 24 causes
Different in each hospital
Interventions tailored to local cause
2nd study ndash web based tool Collect data about your causes (from 24) and used these
interventions to fix your local causes
8 Pilots compliance 48 81
Vs 174 organisations 58 84 (769
projects in
23
Variation reported in a UK study
24
232016
One lesson for research from case
Providers supplied data
Method enables easy local adaptation
But adaptations made are unknown
Effectiveness on average not the only knowledge needed by implementers
Need data to explain variations Not implemented
Context hinders
Adapt successfully or unsuccessfully
25
232016
More informative research
Experimental Trials Can and should we explain high and low outcomes
performers
Can we do subgroup analysis
Focus on non-experimental observational
studies
26
232016
Do you have any of these ldquoResearch
challengesrdquo Documenting
Use taxonomies of implementation methods
Use adaptation typology
Attributing Use Implementation outcome model
Use Logic Modelprogramme theory
Generalising ndash guidance Find which adaptations in which context get which results
Use (take up by practitioners)
= need Pre-study planning to get data for
27
Exercise
You are the head of a clinic
To enable physicians to follow antibiotic
prescribing guidelineshellip
Which behavioral change techniques would
be most effective
Would you use the same in FQC PHC and
Regan paediatrics outpatients 28
232016
Selection from 93 behavior change techniques (Michie 2013)
1 Instruction on how to perform a behavior
2 Punishment
3 Anticipation of future rewards or removal of punishment
4 Practice with Graded tasks
5 Mental rehearsal of successful performance
6 Examine Anticipated regret
7 Feedback on behavior
8 Other(s) monitoring with awareness
9 Social comparison
10 Behavioral contract
11 Social support
12 Identification of self as role model
29
232016
See also Powell 2015 73 implementation
strategies
Mixes direct interventions and indirect context
changes
Education
Audit and provide feedback
Create or change credentialing andor licensure
standards
Develop disincentives
Use capitated payments
Do we need to distinguish 30
232016
Document
Adaptions
1Who made the
modification
2What was modified
(Stirman 2009)
31
232016
Documenting critical context factors -
framworks CFIR (Damschroder 2009)
PHARIS (Rycroft-Malone 2002)
ORCA readiness - based on PHARIS
MUSIQ (Kaplan et al 2010) - QI
French et al 2009 (review of context measures
for evidence-based practice (EBP))
32
232016
Recommended practical tools ORCA (or CFIR httpwwwcfirguideorg)
33
change readiness (HRET)
4 areas the innovation target audience the organisation
the environment
34
Attributing ndash use causal chain frameworks Theory informed Case evaluation
35
232016
Programme theory models or logic models
36
232016
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Healthcare experience
Emergency hospital admission
Delays and meds changed
HAI
Sent home ndash no comms or support
Readmission
Beforeafter
11
232016
Mary - six weeks later
After hospitalisation
Avoidable cost 4600$
12
232016
Name a proven interventions for any problems
Changes of medswrong meds
Hospital acquired infection
After care transition
Medication reconcilliation
Hand hygiene - bundle of interventions
After-care - Coleman CTI ndash Transition coach and
education 13
232016
Your experience
We have implemented solutions to prevent this
- Yes No sometimes
We have effective Med Rec - Medications changes
unlikely
HAI from Hand Hygiene non-compliance unlikely
After care information to PCP ensured within 2
days of discharge
Effective after-care support for older multiple
morbidity patients
14
232016
Why not implemented - your vote
No problem (one-off event)
Not aware of these proven effective solutions
Know how to implement - donrsquot have time or
resources to implement
To implement effectively - donrsquot have knowledge
and skills
Know solutions - uncertain if we would get same
results - would it work here
Donrsquot have research to show how to implement in
different settings
15
Johns Questhellip
Why not implemented
What would help fast amp widespread
implementation
How can research best help
16
232016
Conclusions so farhellip
hellipin 2 slides
232016 17
Success depends onhellip
Seed Gardenerplanting amp nurture Soil climate
Idea 10
AdaptionImplementation 30
Personalities 20
The 10203040 change success rule
Soil receptive ndash staff
readiness
amp wider Climate 40
No intervention survives first contact with context
18
232016
Implemented as planned
Intervention plan
Adapt to survive
Adapt the improvement change tested
elsewhere
Eg turn at risk patients every 4hrs not 2hrs
(Pressure ulcer intervention)
Adapt the context
Increase staffing by 05 FTE
Both - Adapt the change amp context
Need ldquoGoldilocks Improvement-Fit for
Take Uprdquo GIFTU
19
Context ndash innovation ldquoFitrdquo
20
232016
Fidelity and adaptive implementation
t
21
232016
Adapt to ldquoFitrdquo to setting and subjects
Adjust over time ndash dynamicgtgtgtgtgt
To do this-Resources data and skills
Research questionshellip
Is the adaptation more or less effective
How can we find effective adaptations in
different settings
How can we do lower-cost faster research
on questions which will help take up
22
232016
Research ldquoCaserdquo example
Improving physicians hand hygiene compliance
ndash most effective intervention
Pilot study 8 hospitals using QI methods Identified 24 causes
Different in each hospital
Interventions tailored to local cause
2nd study ndash web based tool Collect data about your causes (from 24) and used these
interventions to fix your local causes
8 Pilots compliance 48 81
Vs 174 organisations 58 84 (769
projects in
23
Variation reported in a UK study
24
232016
One lesson for research from case
Providers supplied data
Method enables easy local adaptation
But adaptations made are unknown
Effectiveness on average not the only knowledge needed by implementers
Need data to explain variations Not implemented
Context hinders
Adapt successfully or unsuccessfully
25
232016
More informative research
Experimental Trials Can and should we explain high and low outcomes
performers
Can we do subgroup analysis
Focus on non-experimental observational
studies
26
232016
Do you have any of these ldquoResearch
challengesrdquo Documenting
Use taxonomies of implementation methods
Use adaptation typology
Attributing Use Implementation outcome model
Use Logic Modelprogramme theory
Generalising ndash guidance Find which adaptations in which context get which results
Use (take up by practitioners)
= need Pre-study planning to get data for
27
Exercise
You are the head of a clinic
To enable physicians to follow antibiotic
prescribing guidelineshellip
Which behavioral change techniques would
be most effective
Would you use the same in FQC PHC and
Regan paediatrics outpatients 28
232016
Selection from 93 behavior change techniques (Michie 2013)
1 Instruction on how to perform a behavior
2 Punishment
3 Anticipation of future rewards or removal of punishment
4 Practice with Graded tasks
5 Mental rehearsal of successful performance
6 Examine Anticipated regret
7 Feedback on behavior
8 Other(s) monitoring with awareness
9 Social comparison
10 Behavioral contract
11 Social support
12 Identification of self as role model
29
232016
See also Powell 2015 73 implementation
strategies
Mixes direct interventions and indirect context
changes
Education
Audit and provide feedback
Create or change credentialing andor licensure
standards
Develop disincentives
Use capitated payments
Do we need to distinguish 30
232016
Document
Adaptions
1Who made the
modification
2What was modified
(Stirman 2009)
31
232016
Documenting critical context factors -
framworks CFIR (Damschroder 2009)
PHARIS (Rycroft-Malone 2002)
ORCA readiness - based on PHARIS
MUSIQ (Kaplan et al 2010) - QI
French et al 2009 (review of context measures
for evidence-based practice (EBP))
32
232016
Recommended practical tools ORCA (or CFIR httpwwwcfirguideorg)
33
change readiness (HRET)
4 areas the innovation target audience the organisation
the environment
34
Attributing ndash use causal chain frameworks Theory informed Case evaluation
35
232016
Programme theory models or logic models
36
232016
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Mary - six weeks later
After hospitalisation
Avoidable cost 4600$
12
232016
Name a proven interventions for any problems
Changes of medswrong meds
Hospital acquired infection
After care transition
Medication reconcilliation
Hand hygiene - bundle of interventions
After-care - Coleman CTI ndash Transition coach and
education 13
232016
Your experience
We have implemented solutions to prevent this
- Yes No sometimes
We have effective Med Rec - Medications changes
unlikely
HAI from Hand Hygiene non-compliance unlikely
After care information to PCP ensured within 2
days of discharge
Effective after-care support for older multiple
morbidity patients
14
232016
Why not implemented - your vote
No problem (one-off event)
Not aware of these proven effective solutions
Know how to implement - donrsquot have time or
resources to implement
To implement effectively - donrsquot have knowledge
and skills
Know solutions - uncertain if we would get same
results - would it work here
Donrsquot have research to show how to implement in
different settings
15
Johns Questhellip
Why not implemented
What would help fast amp widespread
implementation
How can research best help
16
232016
Conclusions so farhellip
hellipin 2 slides
232016 17
Success depends onhellip
Seed Gardenerplanting amp nurture Soil climate
Idea 10
AdaptionImplementation 30
Personalities 20
The 10203040 change success rule
Soil receptive ndash staff
readiness
amp wider Climate 40
No intervention survives first contact with context
18
232016
Implemented as planned
Intervention plan
Adapt to survive
Adapt the improvement change tested
elsewhere
Eg turn at risk patients every 4hrs not 2hrs
(Pressure ulcer intervention)
Adapt the context
Increase staffing by 05 FTE
Both - Adapt the change amp context
Need ldquoGoldilocks Improvement-Fit for
Take Uprdquo GIFTU
19
Context ndash innovation ldquoFitrdquo
20
232016
Fidelity and adaptive implementation
t
21
232016
Adapt to ldquoFitrdquo to setting and subjects
Adjust over time ndash dynamicgtgtgtgtgt
To do this-Resources data and skills
Research questionshellip
Is the adaptation more or less effective
How can we find effective adaptations in
different settings
How can we do lower-cost faster research
on questions which will help take up
22
232016
Research ldquoCaserdquo example
Improving physicians hand hygiene compliance
ndash most effective intervention
Pilot study 8 hospitals using QI methods Identified 24 causes
Different in each hospital
Interventions tailored to local cause
2nd study ndash web based tool Collect data about your causes (from 24) and used these
interventions to fix your local causes
8 Pilots compliance 48 81
Vs 174 organisations 58 84 (769
projects in
23
Variation reported in a UK study
24
232016
One lesson for research from case
Providers supplied data
Method enables easy local adaptation
But adaptations made are unknown
Effectiveness on average not the only knowledge needed by implementers
Need data to explain variations Not implemented
Context hinders
Adapt successfully or unsuccessfully
25
232016
More informative research
Experimental Trials Can and should we explain high and low outcomes
performers
Can we do subgroup analysis
Focus on non-experimental observational
studies
26
232016
Do you have any of these ldquoResearch
challengesrdquo Documenting
Use taxonomies of implementation methods
Use adaptation typology
Attributing Use Implementation outcome model
Use Logic Modelprogramme theory
Generalising ndash guidance Find which adaptations in which context get which results
Use (take up by practitioners)
= need Pre-study planning to get data for
27
Exercise
You are the head of a clinic
To enable physicians to follow antibiotic
prescribing guidelineshellip
Which behavioral change techniques would
be most effective
Would you use the same in FQC PHC and
Regan paediatrics outpatients 28
232016
Selection from 93 behavior change techniques (Michie 2013)
1 Instruction on how to perform a behavior
2 Punishment
3 Anticipation of future rewards or removal of punishment
4 Practice with Graded tasks
5 Mental rehearsal of successful performance
6 Examine Anticipated regret
7 Feedback on behavior
8 Other(s) monitoring with awareness
9 Social comparison
10 Behavioral contract
11 Social support
12 Identification of self as role model
29
232016
See also Powell 2015 73 implementation
strategies
Mixes direct interventions and indirect context
changes
Education
Audit and provide feedback
Create or change credentialing andor licensure
standards
Develop disincentives
Use capitated payments
Do we need to distinguish 30
232016
Document
Adaptions
1Who made the
modification
2What was modified
(Stirman 2009)
31
232016
Documenting critical context factors -
framworks CFIR (Damschroder 2009)
PHARIS (Rycroft-Malone 2002)
ORCA readiness - based on PHARIS
MUSIQ (Kaplan et al 2010) - QI
French et al 2009 (review of context measures
for evidence-based practice (EBP))
32
232016
Recommended practical tools ORCA (or CFIR httpwwwcfirguideorg)
33
change readiness (HRET)
4 areas the innovation target audience the organisation
the environment
34
Attributing ndash use causal chain frameworks Theory informed Case evaluation
35
232016
Programme theory models or logic models
36
232016
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Name a proven interventions for any problems
Changes of medswrong meds
Hospital acquired infection
After care transition
Medication reconcilliation
Hand hygiene - bundle of interventions
After-care - Coleman CTI ndash Transition coach and
education 13
232016
Your experience
We have implemented solutions to prevent this
- Yes No sometimes
We have effective Med Rec - Medications changes
unlikely
HAI from Hand Hygiene non-compliance unlikely
After care information to PCP ensured within 2
days of discharge
Effective after-care support for older multiple
morbidity patients
14
232016
Why not implemented - your vote
No problem (one-off event)
Not aware of these proven effective solutions
Know how to implement - donrsquot have time or
resources to implement
To implement effectively - donrsquot have knowledge
and skills
Know solutions - uncertain if we would get same
results - would it work here
Donrsquot have research to show how to implement in
different settings
15
Johns Questhellip
Why not implemented
What would help fast amp widespread
implementation
How can research best help
16
232016
Conclusions so farhellip
hellipin 2 slides
232016 17
Success depends onhellip
Seed Gardenerplanting amp nurture Soil climate
Idea 10
AdaptionImplementation 30
Personalities 20
The 10203040 change success rule
Soil receptive ndash staff
readiness
amp wider Climate 40
No intervention survives first contact with context
18
232016
Implemented as planned
Intervention plan
Adapt to survive
Adapt the improvement change tested
elsewhere
Eg turn at risk patients every 4hrs not 2hrs
(Pressure ulcer intervention)
Adapt the context
Increase staffing by 05 FTE
Both - Adapt the change amp context
Need ldquoGoldilocks Improvement-Fit for
Take Uprdquo GIFTU
19
Context ndash innovation ldquoFitrdquo
20
232016
Fidelity and adaptive implementation
t
21
232016
Adapt to ldquoFitrdquo to setting and subjects
Adjust over time ndash dynamicgtgtgtgtgt
To do this-Resources data and skills
Research questionshellip
Is the adaptation more or less effective
How can we find effective adaptations in
different settings
How can we do lower-cost faster research
on questions which will help take up
22
232016
Research ldquoCaserdquo example
Improving physicians hand hygiene compliance
ndash most effective intervention
Pilot study 8 hospitals using QI methods Identified 24 causes
Different in each hospital
Interventions tailored to local cause
2nd study ndash web based tool Collect data about your causes (from 24) and used these
interventions to fix your local causes
8 Pilots compliance 48 81
Vs 174 organisations 58 84 (769
projects in
23
Variation reported in a UK study
24
232016
One lesson for research from case
Providers supplied data
Method enables easy local adaptation
But adaptations made are unknown
Effectiveness on average not the only knowledge needed by implementers
Need data to explain variations Not implemented
Context hinders
Adapt successfully or unsuccessfully
25
232016
More informative research
Experimental Trials Can and should we explain high and low outcomes
performers
Can we do subgroup analysis
Focus on non-experimental observational
studies
26
232016
Do you have any of these ldquoResearch
challengesrdquo Documenting
Use taxonomies of implementation methods
Use adaptation typology
Attributing Use Implementation outcome model
Use Logic Modelprogramme theory
Generalising ndash guidance Find which adaptations in which context get which results
Use (take up by practitioners)
= need Pre-study planning to get data for
27
Exercise
You are the head of a clinic
To enable physicians to follow antibiotic
prescribing guidelineshellip
Which behavioral change techniques would
be most effective
Would you use the same in FQC PHC and
Regan paediatrics outpatients 28
232016
Selection from 93 behavior change techniques (Michie 2013)
1 Instruction on how to perform a behavior
2 Punishment
3 Anticipation of future rewards or removal of punishment
4 Practice with Graded tasks
5 Mental rehearsal of successful performance
6 Examine Anticipated regret
7 Feedback on behavior
8 Other(s) monitoring with awareness
9 Social comparison
10 Behavioral contract
11 Social support
12 Identification of self as role model
29
232016
See also Powell 2015 73 implementation
strategies
Mixes direct interventions and indirect context
changes
Education
Audit and provide feedback
Create or change credentialing andor licensure
standards
Develop disincentives
Use capitated payments
Do we need to distinguish 30
232016
Document
Adaptions
1Who made the
modification
2What was modified
(Stirman 2009)
31
232016
Documenting critical context factors -
framworks CFIR (Damschroder 2009)
PHARIS (Rycroft-Malone 2002)
ORCA readiness - based on PHARIS
MUSIQ (Kaplan et al 2010) - QI
French et al 2009 (review of context measures
for evidence-based practice (EBP))
32
232016
Recommended practical tools ORCA (or CFIR httpwwwcfirguideorg)
33
change readiness (HRET)
4 areas the innovation target audience the organisation
the environment
34
Attributing ndash use causal chain frameworks Theory informed Case evaluation
35
232016
Programme theory models or logic models
36
232016
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Your experience
We have implemented solutions to prevent this
- Yes No sometimes
We have effective Med Rec - Medications changes
unlikely
HAI from Hand Hygiene non-compliance unlikely
After care information to PCP ensured within 2
days of discharge
Effective after-care support for older multiple
morbidity patients
14
232016
Why not implemented - your vote
No problem (one-off event)
Not aware of these proven effective solutions
Know how to implement - donrsquot have time or
resources to implement
To implement effectively - donrsquot have knowledge
and skills
Know solutions - uncertain if we would get same
results - would it work here
Donrsquot have research to show how to implement in
different settings
15
Johns Questhellip
Why not implemented
What would help fast amp widespread
implementation
How can research best help
16
232016
Conclusions so farhellip
hellipin 2 slides
232016 17
Success depends onhellip
Seed Gardenerplanting amp nurture Soil climate
Idea 10
AdaptionImplementation 30
Personalities 20
The 10203040 change success rule
Soil receptive ndash staff
readiness
amp wider Climate 40
No intervention survives first contact with context
18
232016
Implemented as planned
Intervention plan
Adapt to survive
Adapt the improvement change tested
elsewhere
Eg turn at risk patients every 4hrs not 2hrs
(Pressure ulcer intervention)
Adapt the context
Increase staffing by 05 FTE
Both - Adapt the change amp context
Need ldquoGoldilocks Improvement-Fit for
Take Uprdquo GIFTU
19
Context ndash innovation ldquoFitrdquo
20
232016
Fidelity and adaptive implementation
t
21
232016
Adapt to ldquoFitrdquo to setting and subjects
Adjust over time ndash dynamicgtgtgtgtgt
To do this-Resources data and skills
Research questionshellip
Is the adaptation more or less effective
How can we find effective adaptations in
different settings
How can we do lower-cost faster research
on questions which will help take up
22
232016
Research ldquoCaserdquo example
Improving physicians hand hygiene compliance
ndash most effective intervention
Pilot study 8 hospitals using QI methods Identified 24 causes
Different in each hospital
Interventions tailored to local cause
2nd study ndash web based tool Collect data about your causes (from 24) and used these
interventions to fix your local causes
8 Pilots compliance 48 81
Vs 174 organisations 58 84 (769
projects in
23
Variation reported in a UK study
24
232016
One lesson for research from case
Providers supplied data
Method enables easy local adaptation
But adaptations made are unknown
Effectiveness on average not the only knowledge needed by implementers
Need data to explain variations Not implemented
Context hinders
Adapt successfully or unsuccessfully
25
232016
More informative research
Experimental Trials Can and should we explain high and low outcomes
performers
Can we do subgroup analysis
Focus on non-experimental observational
studies
26
232016
Do you have any of these ldquoResearch
challengesrdquo Documenting
Use taxonomies of implementation methods
Use adaptation typology
Attributing Use Implementation outcome model
Use Logic Modelprogramme theory
Generalising ndash guidance Find which adaptations in which context get which results
Use (take up by practitioners)
= need Pre-study planning to get data for
27
Exercise
You are the head of a clinic
To enable physicians to follow antibiotic
prescribing guidelineshellip
Which behavioral change techniques would
be most effective
Would you use the same in FQC PHC and
Regan paediatrics outpatients 28
232016
Selection from 93 behavior change techniques (Michie 2013)
1 Instruction on how to perform a behavior
2 Punishment
3 Anticipation of future rewards or removal of punishment
4 Practice with Graded tasks
5 Mental rehearsal of successful performance
6 Examine Anticipated regret
7 Feedback on behavior
8 Other(s) monitoring with awareness
9 Social comparison
10 Behavioral contract
11 Social support
12 Identification of self as role model
29
232016
See also Powell 2015 73 implementation
strategies
Mixes direct interventions and indirect context
changes
Education
Audit and provide feedback
Create or change credentialing andor licensure
standards
Develop disincentives
Use capitated payments
Do we need to distinguish 30
232016
Document
Adaptions
1Who made the
modification
2What was modified
(Stirman 2009)
31
232016
Documenting critical context factors -
framworks CFIR (Damschroder 2009)
PHARIS (Rycroft-Malone 2002)
ORCA readiness - based on PHARIS
MUSIQ (Kaplan et al 2010) - QI
French et al 2009 (review of context measures
for evidence-based practice (EBP))
32
232016
Recommended practical tools ORCA (or CFIR httpwwwcfirguideorg)
33
change readiness (HRET)
4 areas the innovation target audience the organisation
the environment
34
Attributing ndash use causal chain frameworks Theory informed Case evaluation
35
232016
Programme theory models or logic models
36
232016
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Why not implemented - your vote
No problem (one-off event)
Not aware of these proven effective solutions
Know how to implement - donrsquot have time or
resources to implement
To implement effectively - donrsquot have knowledge
and skills
Know solutions - uncertain if we would get same
results - would it work here
Donrsquot have research to show how to implement in
different settings
15
Johns Questhellip
Why not implemented
What would help fast amp widespread
implementation
How can research best help
16
232016
Conclusions so farhellip
hellipin 2 slides
232016 17
Success depends onhellip
Seed Gardenerplanting amp nurture Soil climate
Idea 10
AdaptionImplementation 30
Personalities 20
The 10203040 change success rule
Soil receptive ndash staff
readiness
amp wider Climate 40
No intervention survives first contact with context
18
232016
Implemented as planned
Intervention plan
Adapt to survive
Adapt the improvement change tested
elsewhere
Eg turn at risk patients every 4hrs not 2hrs
(Pressure ulcer intervention)
Adapt the context
Increase staffing by 05 FTE
Both - Adapt the change amp context
Need ldquoGoldilocks Improvement-Fit for
Take Uprdquo GIFTU
19
Context ndash innovation ldquoFitrdquo
20
232016
Fidelity and adaptive implementation
t
21
232016
Adapt to ldquoFitrdquo to setting and subjects
Adjust over time ndash dynamicgtgtgtgtgt
To do this-Resources data and skills
Research questionshellip
Is the adaptation more or less effective
How can we find effective adaptations in
different settings
How can we do lower-cost faster research
on questions which will help take up
22
232016
Research ldquoCaserdquo example
Improving physicians hand hygiene compliance
ndash most effective intervention
Pilot study 8 hospitals using QI methods Identified 24 causes
Different in each hospital
Interventions tailored to local cause
2nd study ndash web based tool Collect data about your causes (from 24) and used these
interventions to fix your local causes
8 Pilots compliance 48 81
Vs 174 organisations 58 84 (769
projects in
23
Variation reported in a UK study
24
232016
One lesson for research from case
Providers supplied data
Method enables easy local adaptation
But adaptations made are unknown
Effectiveness on average not the only knowledge needed by implementers
Need data to explain variations Not implemented
Context hinders
Adapt successfully or unsuccessfully
25
232016
More informative research
Experimental Trials Can and should we explain high and low outcomes
performers
Can we do subgroup analysis
Focus on non-experimental observational
studies
26
232016
Do you have any of these ldquoResearch
challengesrdquo Documenting
Use taxonomies of implementation methods
Use adaptation typology
Attributing Use Implementation outcome model
Use Logic Modelprogramme theory
Generalising ndash guidance Find which adaptations in which context get which results
Use (take up by practitioners)
= need Pre-study planning to get data for
27
Exercise
You are the head of a clinic
To enable physicians to follow antibiotic
prescribing guidelineshellip
Which behavioral change techniques would
be most effective
Would you use the same in FQC PHC and
Regan paediatrics outpatients 28
232016
Selection from 93 behavior change techniques (Michie 2013)
1 Instruction on how to perform a behavior
2 Punishment
3 Anticipation of future rewards or removal of punishment
4 Practice with Graded tasks
5 Mental rehearsal of successful performance
6 Examine Anticipated regret
7 Feedback on behavior
8 Other(s) monitoring with awareness
9 Social comparison
10 Behavioral contract
11 Social support
12 Identification of self as role model
29
232016
See also Powell 2015 73 implementation
strategies
Mixes direct interventions and indirect context
changes
Education
Audit and provide feedback
Create or change credentialing andor licensure
standards
Develop disincentives
Use capitated payments
Do we need to distinguish 30
232016
Document
Adaptions
1Who made the
modification
2What was modified
(Stirman 2009)
31
232016
Documenting critical context factors -
framworks CFIR (Damschroder 2009)
PHARIS (Rycroft-Malone 2002)
ORCA readiness - based on PHARIS
MUSIQ (Kaplan et al 2010) - QI
French et al 2009 (review of context measures
for evidence-based practice (EBP))
32
232016
Recommended practical tools ORCA (or CFIR httpwwwcfirguideorg)
33
change readiness (HRET)
4 areas the innovation target audience the organisation
the environment
34
Attributing ndash use causal chain frameworks Theory informed Case evaluation
35
232016
Programme theory models or logic models
36
232016
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Johns Questhellip
Why not implemented
What would help fast amp widespread
implementation
How can research best help
16
232016
Conclusions so farhellip
hellipin 2 slides
232016 17
Success depends onhellip
Seed Gardenerplanting amp nurture Soil climate
Idea 10
AdaptionImplementation 30
Personalities 20
The 10203040 change success rule
Soil receptive ndash staff
readiness
amp wider Climate 40
No intervention survives first contact with context
18
232016
Implemented as planned
Intervention plan
Adapt to survive
Adapt the improvement change tested
elsewhere
Eg turn at risk patients every 4hrs not 2hrs
(Pressure ulcer intervention)
Adapt the context
Increase staffing by 05 FTE
Both - Adapt the change amp context
Need ldquoGoldilocks Improvement-Fit for
Take Uprdquo GIFTU
19
Context ndash innovation ldquoFitrdquo
20
232016
Fidelity and adaptive implementation
t
21
232016
Adapt to ldquoFitrdquo to setting and subjects
Adjust over time ndash dynamicgtgtgtgtgt
To do this-Resources data and skills
Research questionshellip
Is the adaptation more or less effective
How can we find effective adaptations in
different settings
How can we do lower-cost faster research
on questions which will help take up
22
232016
Research ldquoCaserdquo example
Improving physicians hand hygiene compliance
ndash most effective intervention
Pilot study 8 hospitals using QI methods Identified 24 causes
Different in each hospital
Interventions tailored to local cause
2nd study ndash web based tool Collect data about your causes (from 24) and used these
interventions to fix your local causes
8 Pilots compliance 48 81
Vs 174 organisations 58 84 (769
projects in
23
Variation reported in a UK study
24
232016
One lesson for research from case
Providers supplied data
Method enables easy local adaptation
But adaptations made are unknown
Effectiveness on average not the only knowledge needed by implementers
Need data to explain variations Not implemented
Context hinders
Adapt successfully or unsuccessfully
25
232016
More informative research
Experimental Trials Can and should we explain high and low outcomes
performers
Can we do subgroup analysis
Focus on non-experimental observational
studies
26
232016
Do you have any of these ldquoResearch
challengesrdquo Documenting
Use taxonomies of implementation methods
Use adaptation typology
Attributing Use Implementation outcome model
Use Logic Modelprogramme theory
Generalising ndash guidance Find which adaptations in which context get which results
Use (take up by practitioners)
= need Pre-study planning to get data for
27
Exercise
You are the head of a clinic
To enable physicians to follow antibiotic
prescribing guidelineshellip
Which behavioral change techniques would
be most effective
Would you use the same in FQC PHC and
Regan paediatrics outpatients 28
232016
Selection from 93 behavior change techniques (Michie 2013)
1 Instruction on how to perform a behavior
2 Punishment
3 Anticipation of future rewards or removal of punishment
4 Practice with Graded tasks
5 Mental rehearsal of successful performance
6 Examine Anticipated regret
7 Feedback on behavior
8 Other(s) monitoring with awareness
9 Social comparison
10 Behavioral contract
11 Social support
12 Identification of self as role model
29
232016
See also Powell 2015 73 implementation
strategies
Mixes direct interventions and indirect context
changes
Education
Audit and provide feedback
Create or change credentialing andor licensure
standards
Develop disincentives
Use capitated payments
Do we need to distinguish 30
232016
Document
Adaptions
1Who made the
modification
2What was modified
(Stirman 2009)
31
232016
Documenting critical context factors -
framworks CFIR (Damschroder 2009)
PHARIS (Rycroft-Malone 2002)
ORCA readiness - based on PHARIS
MUSIQ (Kaplan et al 2010) - QI
French et al 2009 (review of context measures
for evidence-based practice (EBP))
32
232016
Recommended practical tools ORCA (or CFIR httpwwwcfirguideorg)
33
change readiness (HRET)
4 areas the innovation target audience the organisation
the environment
34
Attributing ndash use causal chain frameworks Theory informed Case evaluation
35
232016
Programme theory models or logic models
36
232016
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
232016 17
Success depends onhellip
Seed Gardenerplanting amp nurture Soil climate
Idea 10
AdaptionImplementation 30
Personalities 20
The 10203040 change success rule
Soil receptive ndash staff
readiness
amp wider Climate 40
No intervention survives first contact with context
18
232016
Implemented as planned
Intervention plan
Adapt to survive
Adapt the improvement change tested
elsewhere
Eg turn at risk patients every 4hrs not 2hrs
(Pressure ulcer intervention)
Adapt the context
Increase staffing by 05 FTE
Both - Adapt the change amp context
Need ldquoGoldilocks Improvement-Fit for
Take Uprdquo GIFTU
19
Context ndash innovation ldquoFitrdquo
20
232016
Fidelity and adaptive implementation
t
21
232016
Adapt to ldquoFitrdquo to setting and subjects
Adjust over time ndash dynamicgtgtgtgtgt
To do this-Resources data and skills
Research questionshellip
Is the adaptation more or less effective
How can we find effective adaptations in
different settings
How can we do lower-cost faster research
on questions which will help take up
22
232016
Research ldquoCaserdquo example
Improving physicians hand hygiene compliance
ndash most effective intervention
Pilot study 8 hospitals using QI methods Identified 24 causes
Different in each hospital
Interventions tailored to local cause
2nd study ndash web based tool Collect data about your causes (from 24) and used these
interventions to fix your local causes
8 Pilots compliance 48 81
Vs 174 organisations 58 84 (769
projects in
23
Variation reported in a UK study
24
232016
One lesson for research from case
Providers supplied data
Method enables easy local adaptation
But adaptations made are unknown
Effectiveness on average not the only knowledge needed by implementers
Need data to explain variations Not implemented
Context hinders
Adapt successfully or unsuccessfully
25
232016
More informative research
Experimental Trials Can and should we explain high and low outcomes
performers
Can we do subgroup analysis
Focus on non-experimental observational
studies
26
232016
Do you have any of these ldquoResearch
challengesrdquo Documenting
Use taxonomies of implementation methods
Use adaptation typology
Attributing Use Implementation outcome model
Use Logic Modelprogramme theory
Generalising ndash guidance Find which adaptations in which context get which results
Use (take up by practitioners)
= need Pre-study planning to get data for
27
Exercise
You are the head of a clinic
To enable physicians to follow antibiotic
prescribing guidelineshellip
Which behavioral change techniques would
be most effective
Would you use the same in FQC PHC and
Regan paediatrics outpatients 28
232016
Selection from 93 behavior change techniques (Michie 2013)
1 Instruction on how to perform a behavior
2 Punishment
3 Anticipation of future rewards or removal of punishment
4 Practice with Graded tasks
5 Mental rehearsal of successful performance
6 Examine Anticipated regret
7 Feedback on behavior
8 Other(s) monitoring with awareness
9 Social comparison
10 Behavioral contract
11 Social support
12 Identification of self as role model
29
232016
See also Powell 2015 73 implementation
strategies
Mixes direct interventions and indirect context
changes
Education
Audit and provide feedback
Create or change credentialing andor licensure
standards
Develop disincentives
Use capitated payments
Do we need to distinguish 30
232016
Document
Adaptions
1Who made the
modification
2What was modified
(Stirman 2009)
31
232016
Documenting critical context factors -
framworks CFIR (Damschroder 2009)
PHARIS (Rycroft-Malone 2002)
ORCA readiness - based on PHARIS
MUSIQ (Kaplan et al 2010) - QI
French et al 2009 (review of context measures
for evidence-based practice (EBP))
32
232016
Recommended practical tools ORCA (or CFIR httpwwwcfirguideorg)
33
change readiness (HRET)
4 areas the innovation target audience the organisation
the environment
34
Attributing ndash use causal chain frameworks Theory informed Case evaluation
35
232016
Programme theory models or logic models
36
232016
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
No intervention survives first contact with context
18
232016
Implemented as planned
Intervention plan
Adapt to survive
Adapt the improvement change tested
elsewhere
Eg turn at risk patients every 4hrs not 2hrs
(Pressure ulcer intervention)
Adapt the context
Increase staffing by 05 FTE
Both - Adapt the change amp context
Need ldquoGoldilocks Improvement-Fit for
Take Uprdquo GIFTU
19
Context ndash innovation ldquoFitrdquo
20
232016
Fidelity and adaptive implementation
t
21
232016
Adapt to ldquoFitrdquo to setting and subjects
Adjust over time ndash dynamicgtgtgtgtgt
To do this-Resources data and skills
Research questionshellip
Is the adaptation more or less effective
How can we find effective adaptations in
different settings
How can we do lower-cost faster research
on questions which will help take up
22
232016
Research ldquoCaserdquo example
Improving physicians hand hygiene compliance
ndash most effective intervention
Pilot study 8 hospitals using QI methods Identified 24 causes
Different in each hospital
Interventions tailored to local cause
2nd study ndash web based tool Collect data about your causes (from 24) and used these
interventions to fix your local causes
8 Pilots compliance 48 81
Vs 174 organisations 58 84 (769
projects in
23
Variation reported in a UK study
24
232016
One lesson for research from case
Providers supplied data
Method enables easy local adaptation
But adaptations made are unknown
Effectiveness on average not the only knowledge needed by implementers
Need data to explain variations Not implemented
Context hinders
Adapt successfully or unsuccessfully
25
232016
More informative research
Experimental Trials Can and should we explain high and low outcomes
performers
Can we do subgroup analysis
Focus on non-experimental observational
studies
26
232016
Do you have any of these ldquoResearch
challengesrdquo Documenting
Use taxonomies of implementation methods
Use adaptation typology
Attributing Use Implementation outcome model
Use Logic Modelprogramme theory
Generalising ndash guidance Find which adaptations in which context get which results
Use (take up by practitioners)
= need Pre-study planning to get data for
27
Exercise
You are the head of a clinic
To enable physicians to follow antibiotic
prescribing guidelineshellip
Which behavioral change techniques would
be most effective
Would you use the same in FQC PHC and
Regan paediatrics outpatients 28
232016
Selection from 93 behavior change techniques (Michie 2013)
1 Instruction on how to perform a behavior
2 Punishment
3 Anticipation of future rewards or removal of punishment
4 Practice with Graded tasks
5 Mental rehearsal of successful performance
6 Examine Anticipated regret
7 Feedback on behavior
8 Other(s) monitoring with awareness
9 Social comparison
10 Behavioral contract
11 Social support
12 Identification of self as role model
29
232016
See also Powell 2015 73 implementation
strategies
Mixes direct interventions and indirect context
changes
Education
Audit and provide feedback
Create or change credentialing andor licensure
standards
Develop disincentives
Use capitated payments
Do we need to distinguish 30
232016
Document
Adaptions
1Who made the
modification
2What was modified
(Stirman 2009)
31
232016
Documenting critical context factors -
framworks CFIR (Damschroder 2009)
PHARIS (Rycroft-Malone 2002)
ORCA readiness - based on PHARIS
MUSIQ (Kaplan et al 2010) - QI
French et al 2009 (review of context measures
for evidence-based practice (EBP))
32
232016
Recommended practical tools ORCA (or CFIR httpwwwcfirguideorg)
33
change readiness (HRET)
4 areas the innovation target audience the organisation
the environment
34
Attributing ndash use causal chain frameworks Theory informed Case evaluation
35
232016
Programme theory models or logic models
36
232016
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Adapt to survive
Adapt the improvement change tested
elsewhere
Eg turn at risk patients every 4hrs not 2hrs
(Pressure ulcer intervention)
Adapt the context
Increase staffing by 05 FTE
Both - Adapt the change amp context
Need ldquoGoldilocks Improvement-Fit for
Take Uprdquo GIFTU
19
Context ndash innovation ldquoFitrdquo
20
232016
Fidelity and adaptive implementation
t
21
232016
Adapt to ldquoFitrdquo to setting and subjects
Adjust over time ndash dynamicgtgtgtgtgt
To do this-Resources data and skills
Research questionshellip
Is the adaptation more or less effective
How can we find effective adaptations in
different settings
How can we do lower-cost faster research
on questions which will help take up
22
232016
Research ldquoCaserdquo example
Improving physicians hand hygiene compliance
ndash most effective intervention
Pilot study 8 hospitals using QI methods Identified 24 causes
Different in each hospital
Interventions tailored to local cause
2nd study ndash web based tool Collect data about your causes (from 24) and used these
interventions to fix your local causes
8 Pilots compliance 48 81
Vs 174 organisations 58 84 (769
projects in
23
Variation reported in a UK study
24
232016
One lesson for research from case
Providers supplied data
Method enables easy local adaptation
But adaptations made are unknown
Effectiveness on average not the only knowledge needed by implementers
Need data to explain variations Not implemented
Context hinders
Adapt successfully or unsuccessfully
25
232016
More informative research
Experimental Trials Can and should we explain high and low outcomes
performers
Can we do subgroup analysis
Focus on non-experimental observational
studies
26
232016
Do you have any of these ldquoResearch
challengesrdquo Documenting
Use taxonomies of implementation methods
Use adaptation typology
Attributing Use Implementation outcome model
Use Logic Modelprogramme theory
Generalising ndash guidance Find which adaptations in which context get which results
Use (take up by practitioners)
= need Pre-study planning to get data for
27
Exercise
You are the head of a clinic
To enable physicians to follow antibiotic
prescribing guidelineshellip
Which behavioral change techniques would
be most effective
Would you use the same in FQC PHC and
Regan paediatrics outpatients 28
232016
Selection from 93 behavior change techniques (Michie 2013)
1 Instruction on how to perform a behavior
2 Punishment
3 Anticipation of future rewards or removal of punishment
4 Practice with Graded tasks
5 Mental rehearsal of successful performance
6 Examine Anticipated regret
7 Feedback on behavior
8 Other(s) monitoring with awareness
9 Social comparison
10 Behavioral contract
11 Social support
12 Identification of self as role model
29
232016
See also Powell 2015 73 implementation
strategies
Mixes direct interventions and indirect context
changes
Education
Audit and provide feedback
Create or change credentialing andor licensure
standards
Develop disincentives
Use capitated payments
Do we need to distinguish 30
232016
Document
Adaptions
1Who made the
modification
2What was modified
(Stirman 2009)
31
232016
Documenting critical context factors -
framworks CFIR (Damschroder 2009)
PHARIS (Rycroft-Malone 2002)
ORCA readiness - based on PHARIS
MUSIQ (Kaplan et al 2010) - QI
French et al 2009 (review of context measures
for evidence-based practice (EBP))
32
232016
Recommended practical tools ORCA (or CFIR httpwwwcfirguideorg)
33
change readiness (HRET)
4 areas the innovation target audience the organisation
the environment
34
Attributing ndash use causal chain frameworks Theory informed Case evaluation
35
232016
Programme theory models or logic models
36
232016
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Context ndash innovation ldquoFitrdquo
20
232016
Fidelity and adaptive implementation
t
21
232016
Adapt to ldquoFitrdquo to setting and subjects
Adjust over time ndash dynamicgtgtgtgtgt
To do this-Resources data and skills
Research questionshellip
Is the adaptation more or less effective
How can we find effective adaptations in
different settings
How can we do lower-cost faster research
on questions which will help take up
22
232016
Research ldquoCaserdquo example
Improving physicians hand hygiene compliance
ndash most effective intervention
Pilot study 8 hospitals using QI methods Identified 24 causes
Different in each hospital
Interventions tailored to local cause
2nd study ndash web based tool Collect data about your causes (from 24) and used these
interventions to fix your local causes
8 Pilots compliance 48 81
Vs 174 organisations 58 84 (769
projects in
23
Variation reported in a UK study
24
232016
One lesson for research from case
Providers supplied data
Method enables easy local adaptation
But adaptations made are unknown
Effectiveness on average not the only knowledge needed by implementers
Need data to explain variations Not implemented
Context hinders
Adapt successfully or unsuccessfully
25
232016
More informative research
Experimental Trials Can and should we explain high and low outcomes
performers
Can we do subgroup analysis
Focus on non-experimental observational
studies
26
232016
Do you have any of these ldquoResearch
challengesrdquo Documenting
Use taxonomies of implementation methods
Use adaptation typology
Attributing Use Implementation outcome model
Use Logic Modelprogramme theory
Generalising ndash guidance Find which adaptations in which context get which results
Use (take up by practitioners)
= need Pre-study planning to get data for
27
Exercise
You are the head of a clinic
To enable physicians to follow antibiotic
prescribing guidelineshellip
Which behavioral change techniques would
be most effective
Would you use the same in FQC PHC and
Regan paediatrics outpatients 28
232016
Selection from 93 behavior change techniques (Michie 2013)
1 Instruction on how to perform a behavior
2 Punishment
3 Anticipation of future rewards or removal of punishment
4 Practice with Graded tasks
5 Mental rehearsal of successful performance
6 Examine Anticipated regret
7 Feedback on behavior
8 Other(s) monitoring with awareness
9 Social comparison
10 Behavioral contract
11 Social support
12 Identification of self as role model
29
232016
See also Powell 2015 73 implementation
strategies
Mixes direct interventions and indirect context
changes
Education
Audit and provide feedback
Create or change credentialing andor licensure
standards
Develop disincentives
Use capitated payments
Do we need to distinguish 30
232016
Document
Adaptions
1Who made the
modification
2What was modified
(Stirman 2009)
31
232016
Documenting critical context factors -
framworks CFIR (Damschroder 2009)
PHARIS (Rycroft-Malone 2002)
ORCA readiness - based on PHARIS
MUSIQ (Kaplan et al 2010) - QI
French et al 2009 (review of context measures
for evidence-based practice (EBP))
32
232016
Recommended practical tools ORCA (or CFIR httpwwwcfirguideorg)
33
change readiness (HRET)
4 areas the innovation target audience the organisation
the environment
34
Attributing ndash use causal chain frameworks Theory informed Case evaluation
35
232016
Programme theory models or logic models
36
232016
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Fidelity and adaptive implementation
t
21
232016
Adapt to ldquoFitrdquo to setting and subjects
Adjust over time ndash dynamicgtgtgtgtgt
To do this-Resources data and skills
Research questionshellip
Is the adaptation more or less effective
How can we find effective adaptations in
different settings
How can we do lower-cost faster research
on questions which will help take up
22
232016
Research ldquoCaserdquo example
Improving physicians hand hygiene compliance
ndash most effective intervention
Pilot study 8 hospitals using QI methods Identified 24 causes
Different in each hospital
Interventions tailored to local cause
2nd study ndash web based tool Collect data about your causes (from 24) and used these
interventions to fix your local causes
8 Pilots compliance 48 81
Vs 174 organisations 58 84 (769
projects in
23
Variation reported in a UK study
24
232016
One lesson for research from case
Providers supplied data
Method enables easy local adaptation
But adaptations made are unknown
Effectiveness on average not the only knowledge needed by implementers
Need data to explain variations Not implemented
Context hinders
Adapt successfully or unsuccessfully
25
232016
More informative research
Experimental Trials Can and should we explain high and low outcomes
performers
Can we do subgroup analysis
Focus on non-experimental observational
studies
26
232016
Do you have any of these ldquoResearch
challengesrdquo Documenting
Use taxonomies of implementation methods
Use adaptation typology
Attributing Use Implementation outcome model
Use Logic Modelprogramme theory
Generalising ndash guidance Find which adaptations in which context get which results
Use (take up by practitioners)
= need Pre-study planning to get data for
27
Exercise
You are the head of a clinic
To enable physicians to follow antibiotic
prescribing guidelineshellip
Which behavioral change techniques would
be most effective
Would you use the same in FQC PHC and
Regan paediatrics outpatients 28
232016
Selection from 93 behavior change techniques (Michie 2013)
1 Instruction on how to perform a behavior
2 Punishment
3 Anticipation of future rewards or removal of punishment
4 Practice with Graded tasks
5 Mental rehearsal of successful performance
6 Examine Anticipated regret
7 Feedback on behavior
8 Other(s) monitoring with awareness
9 Social comparison
10 Behavioral contract
11 Social support
12 Identification of self as role model
29
232016
See also Powell 2015 73 implementation
strategies
Mixes direct interventions and indirect context
changes
Education
Audit and provide feedback
Create or change credentialing andor licensure
standards
Develop disincentives
Use capitated payments
Do we need to distinguish 30
232016
Document
Adaptions
1Who made the
modification
2What was modified
(Stirman 2009)
31
232016
Documenting critical context factors -
framworks CFIR (Damschroder 2009)
PHARIS (Rycroft-Malone 2002)
ORCA readiness - based on PHARIS
MUSIQ (Kaplan et al 2010) - QI
French et al 2009 (review of context measures
for evidence-based practice (EBP))
32
232016
Recommended practical tools ORCA (or CFIR httpwwwcfirguideorg)
33
change readiness (HRET)
4 areas the innovation target audience the organisation
the environment
34
Attributing ndash use causal chain frameworks Theory informed Case evaluation
35
232016
Programme theory models or logic models
36
232016
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Research questionshellip
Is the adaptation more or less effective
How can we find effective adaptations in
different settings
How can we do lower-cost faster research
on questions which will help take up
22
232016
Research ldquoCaserdquo example
Improving physicians hand hygiene compliance
ndash most effective intervention
Pilot study 8 hospitals using QI methods Identified 24 causes
Different in each hospital
Interventions tailored to local cause
2nd study ndash web based tool Collect data about your causes (from 24) and used these
interventions to fix your local causes
8 Pilots compliance 48 81
Vs 174 organisations 58 84 (769
projects in
23
Variation reported in a UK study
24
232016
One lesson for research from case
Providers supplied data
Method enables easy local adaptation
But adaptations made are unknown
Effectiveness on average not the only knowledge needed by implementers
Need data to explain variations Not implemented
Context hinders
Adapt successfully or unsuccessfully
25
232016
More informative research
Experimental Trials Can and should we explain high and low outcomes
performers
Can we do subgroup analysis
Focus on non-experimental observational
studies
26
232016
Do you have any of these ldquoResearch
challengesrdquo Documenting
Use taxonomies of implementation methods
Use adaptation typology
Attributing Use Implementation outcome model
Use Logic Modelprogramme theory
Generalising ndash guidance Find which adaptations in which context get which results
Use (take up by practitioners)
= need Pre-study planning to get data for
27
Exercise
You are the head of a clinic
To enable physicians to follow antibiotic
prescribing guidelineshellip
Which behavioral change techniques would
be most effective
Would you use the same in FQC PHC and
Regan paediatrics outpatients 28
232016
Selection from 93 behavior change techniques (Michie 2013)
1 Instruction on how to perform a behavior
2 Punishment
3 Anticipation of future rewards or removal of punishment
4 Practice with Graded tasks
5 Mental rehearsal of successful performance
6 Examine Anticipated regret
7 Feedback on behavior
8 Other(s) monitoring with awareness
9 Social comparison
10 Behavioral contract
11 Social support
12 Identification of self as role model
29
232016
See also Powell 2015 73 implementation
strategies
Mixes direct interventions and indirect context
changes
Education
Audit and provide feedback
Create or change credentialing andor licensure
standards
Develop disincentives
Use capitated payments
Do we need to distinguish 30
232016
Document
Adaptions
1Who made the
modification
2What was modified
(Stirman 2009)
31
232016
Documenting critical context factors -
framworks CFIR (Damschroder 2009)
PHARIS (Rycroft-Malone 2002)
ORCA readiness - based on PHARIS
MUSIQ (Kaplan et al 2010) - QI
French et al 2009 (review of context measures
for evidence-based practice (EBP))
32
232016
Recommended practical tools ORCA (or CFIR httpwwwcfirguideorg)
33
change readiness (HRET)
4 areas the innovation target audience the organisation
the environment
34
Attributing ndash use causal chain frameworks Theory informed Case evaluation
35
232016
Programme theory models or logic models
36
232016
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Research ldquoCaserdquo example
Improving physicians hand hygiene compliance
ndash most effective intervention
Pilot study 8 hospitals using QI methods Identified 24 causes
Different in each hospital
Interventions tailored to local cause
2nd study ndash web based tool Collect data about your causes (from 24) and used these
interventions to fix your local causes
8 Pilots compliance 48 81
Vs 174 organisations 58 84 (769
projects in
23
Variation reported in a UK study
24
232016
One lesson for research from case
Providers supplied data
Method enables easy local adaptation
But adaptations made are unknown
Effectiveness on average not the only knowledge needed by implementers
Need data to explain variations Not implemented
Context hinders
Adapt successfully or unsuccessfully
25
232016
More informative research
Experimental Trials Can and should we explain high and low outcomes
performers
Can we do subgroup analysis
Focus on non-experimental observational
studies
26
232016
Do you have any of these ldquoResearch
challengesrdquo Documenting
Use taxonomies of implementation methods
Use adaptation typology
Attributing Use Implementation outcome model
Use Logic Modelprogramme theory
Generalising ndash guidance Find which adaptations in which context get which results
Use (take up by practitioners)
= need Pre-study planning to get data for
27
Exercise
You are the head of a clinic
To enable physicians to follow antibiotic
prescribing guidelineshellip
Which behavioral change techniques would
be most effective
Would you use the same in FQC PHC and
Regan paediatrics outpatients 28
232016
Selection from 93 behavior change techniques (Michie 2013)
1 Instruction on how to perform a behavior
2 Punishment
3 Anticipation of future rewards or removal of punishment
4 Practice with Graded tasks
5 Mental rehearsal of successful performance
6 Examine Anticipated regret
7 Feedback on behavior
8 Other(s) monitoring with awareness
9 Social comparison
10 Behavioral contract
11 Social support
12 Identification of self as role model
29
232016
See also Powell 2015 73 implementation
strategies
Mixes direct interventions and indirect context
changes
Education
Audit and provide feedback
Create or change credentialing andor licensure
standards
Develop disincentives
Use capitated payments
Do we need to distinguish 30
232016
Document
Adaptions
1Who made the
modification
2What was modified
(Stirman 2009)
31
232016
Documenting critical context factors -
framworks CFIR (Damschroder 2009)
PHARIS (Rycroft-Malone 2002)
ORCA readiness - based on PHARIS
MUSIQ (Kaplan et al 2010) - QI
French et al 2009 (review of context measures
for evidence-based practice (EBP))
32
232016
Recommended practical tools ORCA (or CFIR httpwwwcfirguideorg)
33
change readiness (HRET)
4 areas the innovation target audience the organisation
the environment
34
Attributing ndash use causal chain frameworks Theory informed Case evaluation
35
232016
Programme theory models or logic models
36
232016
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Variation reported in a UK study
24
232016
One lesson for research from case
Providers supplied data
Method enables easy local adaptation
But adaptations made are unknown
Effectiveness on average not the only knowledge needed by implementers
Need data to explain variations Not implemented
Context hinders
Adapt successfully or unsuccessfully
25
232016
More informative research
Experimental Trials Can and should we explain high and low outcomes
performers
Can we do subgroup analysis
Focus on non-experimental observational
studies
26
232016
Do you have any of these ldquoResearch
challengesrdquo Documenting
Use taxonomies of implementation methods
Use adaptation typology
Attributing Use Implementation outcome model
Use Logic Modelprogramme theory
Generalising ndash guidance Find which adaptations in which context get which results
Use (take up by practitioners)
= need Pre-study planning to get data for
27
Exercise
You are the head of a clinic
To enable physicians to follow antibiotic
prescribing guidelineshellip
Which behavioral change techniques would
be most effective
Would you use the same in FQC PHC and
Regan paediatrics outpatients 28
232016
Selection from 93 behavior change techniques (Michie 2013)
1 Instruction on how to perform a behavior
2 Punishment
3 Anticipation of future rewards or removal of punishment
4 Practice with Graded tasks
5 Mental rehearsal of successful performance
6 Examine Anticipated regret
7 Feedback on behavior
8 Other(s) monitoring with awareness
9 Social comparison
10 Behavioral contract
11 Social support
12 Identification of self as role model
29
232016
See also Powell 2015 73 implementation
strategies
Mixes direct interventions and indirect context
changes
Education
Audit and provide feedback
Create or change credentialing andor licensure
standards
Develop disincentives
Use capitated payments
Do we need to distinguish 30
232016
Document
Adaptions
1Who made the
modification
2What was modified
(Stirman 2009)
31
232016
Documenting critical context factors -
framworks CFIR (Damschroder 2009)
PHARIS (Rycroft-Malone 2002)
ORCA readiness - based on PHARIS
MUSIQ (Kaplan et al 2010) - QI
French et al 2009 (review of context measures
for evidence-based practice (EBP))
32
232016
Recommended practical tools ORCA (or CFIR httpwwwcfirguideorg)
33
change readiness (HRET)
4 areas the innovation target audience the organisation
the environment
34
Attributing ndash use causal chain frameworks Theory informed Case evaluation
35
232016
Programme theory models or logic models
36
232016
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
One lesson for research from case
Providers supplied data
Method enables easy local adaptation
But adaptations made are unknown
Effectiveness on average not the only knowledge needed by implementers
Need data to explain variations Not implemented
Context hinders
Adapt successfully or unsuccessfully
25
232016
More informative research
Experimental Trials Can and should we explain high and low outcomes
performers
Can we do subgroup analysis
Focus on non-experimental observational
studies
26
232016
Do you have any of these ldquoResearch
challengesrdquo Documenting
Use taxonomies of implementation methods
Use adaptation typology
Attributing Use Implementation outcome model
Use Logic Modelprogramme theory
Generalising ndash guidance Find which adaptations in which context get which results
Use (take up by practitioners)
= need Pre-study planning to get data for
27
Exercise
You are the head of a clinic
To enable physicians to follow antibiotic
prescribing guidelineshellip
Which behavioral change techniques would
be most effective
Would you use the same in FQC PHC and
Regan paediatrics outpatients 28
232016
Selection from 93 behavior change techniques (Michie 2013)
1 Instruction on how to perform a behavior
2 Punishment
3 Anticipation of future rewards or removal of punishment
4 Practice with Graded tasks
5 Mental rehearsal of successful performance
6 Examine Anticipated regret
7 Feedback on behavior
8 Other(s) monitoring with awareness
9 Social comparison
10 Behavioral contract
11 Social support
12 Identification of self as role model
29
232016
See also Powell 2015 73 implementation
strategies
Mixes direct interventions and indirect context
changes
Education
Audit and provide feedback
Create or change credentialing andor licensure
standards
Develop disincentives
Use capitated payments
Do we need to distinguish 30
232016
Document
Adaptions
1Who made the
modification
2What was modified
(Stirman 2009)
31
232016
Documenting critical context factors -
framworks CFIR (Damschroder 2009)
PHARIS (Rycroft-Malone 2002)
ORCA readiness - based on PHARIS
MUSIQ (Kaplan et al 2010) - QI
French et al 2009 (review of context measures
for evidence-based practice (EBP))
32
232016
Recommended practical tools ORCA (or CFIR httpwwwcfirguideorg)
33
change readiness (HRET)
4 areas the innovation target audience the organisation
the environment
34
Attributing ndash use causal chain frameworks Theory informed Case evaluation
35
232016
Programme theory models or logic models
36
232016
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
More informative research
Experimental Trials Can and should we explain high and low outcomes
performers
Can we do subgroup analysis
Focus on non-experimental observational
studies
26
232016
Do you have any of these ldquoResearch
challengesrdquo Documenting
Use taxonomies of implementation methods
Use adaptation typology
Attributing Use Implementation outcome model
Use Logic Modelprogramme theory
Generalising ndash guidance Find which adaptations in which context get which results
Use (take up by practitioners)
= need Pre-study planning to get data for
27
Exercise
You are the head of a clinic
To enable physicians to follow antibiotic
prescribing guidelineshellip
Which behavioral change techniques would
be most effective
Would you use the same in FQC PHC and
Regan paediatrics outpatients 28
232016
Selection from 93 behavior change techniques (Michie 2013)
1 Instruction on how to perform a behavior
2 Punishment
3 Anticipation of future rewards or removal of punishment
4 Practice with Graded tasks
5 Mental rehearsal of successful performance
6 Examine Anticipated regret
7 Feedback on behavior
8 Other(s) monitoring with awareness
9 Social comparison
10 Behavioral contract
11 Social support
12 Identification of self as role model
29
232016
See also Powell 2015 73 implementation
strategies
Mixes direct interventions and indirect context
changes
Education
Audit and provide feedback
Create or change credentialing andor licensure
standards
Develop disincentives
Use capitated payments
Do we need to distinguish 30
232016
Document
Adaptions
1Who made the
modification
2What was modified
(Stirman 2009)
31
232016
Documenting critical context factors -
framworks CFIR (Damschroder 2009)
PHARIS (Rycroft-Malone 2002)
ORCA readiness - based on PHARIS
MUSIQ (Kaplan et al 2010) - QI
French et al 2009 (review of context measures
for evidence-based practice (EBP))
32
232016
Recommended practical tools ORCA (or CFIR httpwwwcfirguideorg)
33
change readiness (HRET)
4 areas the innovation target audience the organisation
the environment
34
Attributing ndash use causal chain frameworks Theory informed Case evaluation
35
232016
Programme theory models or logic models
36
232016
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Do you have any of these ldquoResearch
challengesrdquo Documenting
Use taxonomies of implementation methods
Use adaptation typology
Attributing Use Implementation outcome model
Use Logic Modelprogramme theory
Generalising ndash guidance Find which adaptations in which context get which results
Use (take up by practitioners)
= need Pre-study planning to get data for
27
Exercise
You are the head of a clinic
To enable physicians to follow antibiotic
prescribing guidelineshellip
Which behavioral change techniques would
be most effective
Would you use the same in FQC PHC and
Regan paediatrics outpatients 28
232016
Selection from 93 behavior change techniques (Michie 2013)
1 Instruction on how to perform a behavior
2 Punishment
3 Anticipation of future rewards or removal of punishment
4 Practice with Graded tasks
5 Mental rehearsal of successful performance
6 Examine Anticipated regret
7 Feedback on behavior
8 Other(s) monitoring with awareness
9 Social comparison
10 Behavioral contract
11 Social support
12 Identification of self as role model
29
232016
See also Powell 2015 73 implementation
strategies
Mixes direct interventions and indirect context
changes
Education
Audit and provide feedback
Create or change credentialing andor licensure
standards
Develop disincentives
Use capitated payments
Do we need to distinguish 30
232016
Document
Adaptions
1Who made the
modification
2What was modified
(Stirman 2009)
31
232016
Documenting critical context factors -
framworks CFIR (Damschroder 2009)
PHARIS (Rycroft-Malone 2002)
ORCA readiness - based on PHARIS
MUSIQ (Kaplan et al 2010) - QI
French et al 2009 (review of context measures
for evidence-based practice (EBP))
32
232016
Recommended practical tools ORCA (or CFIR httpwwwcfirguideorg)
33
change readiness (HRET)
4 areas the innovation target audience the organisation
the environment
34
Attributing ndash use causal chain frameworks Theory informed Case evaluation
35
232016
Programme theory models or logic models
36
232016
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Exercise
You are the head of a clinic
To enable physicians to follow antibiotic
prescribing guidelineshellip
Which behavioral change techniques would
be most effective
Would you use the same in FQC PHC and
Regan paediatrics outpatients 28
232016
Selection from 93 behavior change techniques (Michie 2013)
1 Instruction on how to perform a behavior
2 Punishment
3 Anticipation of future rewards or removal of punishment
4 Practice with Graded tasks
5 Mental rehearsal of successful performance
6 Examine Anticipated regret
7 Feedback on behavior
8 Other(s) monitoring with awareness
9 Social comparison
10 Behavioral contract
11 Social support
12 Identification of self as role model
29
232016
See also Powell 2015 73 implementation
strategies
Mixes direct interventions and indirect context
changes
Education
Audit and provide feedback
Create or change credentialing andor licensure
standards
Develop disincentives
Use capitated payments
Do we need to distinguish 30
232016
Document
Adaptions
1Who made the
modification
2What was modified
(Stirman 2009)
31
232016
Documenting critical context factors -
framworks CFIR (Damschroder 2009)
PHARIS (Rycroft-Malone 2002)
ORCA readiness - based on PHARIS
MUSIQ (Kaplan et al 2010) - QI
French et al 2009 (review of context measures
for evidence-based practice (EBP))
32
232016
Recommended practical tools ORCA (or CFIR httpwwwcfirguideorg)
33
change readiness (HRET)
4 areas the innovation target audience the organisation
the environment
34
Attributing ndash use causal chain frameworks Theory informed Case evaluation
35
232016
Programme theory models or logic models
36
232016
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Selection from 93 behavior change techniques (Michie 2013)
1 Instruction on how to perform a behavior
2 Punishment
3 Anticipation of future rewards or removal of punishment
4 Practice with Graded tasks
5 Mental rehearsal of successful performance
6 Examine Anticipated regret
7 Feedback on behavior
8 Other(s) monitoring with awareness
9 Social comparison
10 Behavioral contract
11 Social support
12 Identification of self as role model
29
232016
See also Powell 2015 73 implementation
strategies
Mixes direct interventions and indirect context
changes
Education
Audit and provide feedback
Create or change credentialing andor licensure
standards
Develop disincentives
Use capitated payments
Do we need to distinguish 30
232016
Document
Adaptions
1Who made the
modification
2What was modified
(Stirman 2009)
31
232016
Documenting critical context factors -
framworks CFIR (Damschroder 2009)
PHARIS (Rycroft-Malone 2002)
ORCA readiness - based on PHARIS
MUSIQ (Kaplan et al 2010) - QI
French et al 2009 (review of context measures
for evidence-based practice (EBP))
32
232016
Recommended practical tools ORCA (or CFIR httpwwwcfirguideorg)
33
change readiness (HRET)
4 areas the innovation target audience the organisation
the environment
34
Attributing ndash use causal chain frameworks Theory informed Case evaluation
35
232016
Programme theory models or logic models
36
232016
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
See also Powell 2015 73 implementation
strategies
Mixes direct interventions and indirect context
changes
Education
Audit and provide feedback
Create or change credentialing andor licensure
standards
Develop disincentives
Use capitated payments
Do we need to distinguish 30
232016
Document
Adaptions
1Who made the
modification
2What was modified
(Stirman 2009)
31
232016
Documenting critical context factors -
framworks CFIR (Damschroder 2009)
PHARIS (Rycroft-Malone 2002)
ORCA readiness - based on PHARIS
MUSIQ (Kaplan et al 2010) - QI
French et al 2009 (review of context measures
for evidence-based practice (EBP))
32
232016
Recommended practical tools ORCA (or CFIR httpwwwcfirguideorg)
33
change readiness (HRET)
4 areas the innovation target audience the organisation
the environment
34
Attributing ndash use causal chain frameworks Theory informed Case evaluation
35
232016
Programme theory models or logic models
36
232016
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Document
Adaptions
1Who made the
modification
2What was modified
(Stirman 2009)
31
232016
Documenting critical context factors -
framworks CFIR (Damschroder 2009)
PHARIS (Rycroft-Malone 2002)
ORCA readiness - based on PHARIS
MUSIQ (Kaplan et al 2010) - QI
French et al 2009 (review of context measures
for evidence-based practice (EBP))
32
232016
Recommended practical tools ORCA (or CFIR httpwwwcfirguideorg)
33
change readiness (HRET)
4 areas the innovation target audience the organisation
the environment
34
Attributing ndash use causal chain frameworks Theory informed Case evaluation
35
232016
Programme theory models or logic models
36
232016
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Documenting critical context factors -
framworks CFIR (Damschroder 2009)
PHARIS (Rycroft-Malone 2002)
ORCA readiness - based on PHARIS
MUSIQ (Kaplan et al 2010) - QI
French et al 2009 (review of context measures
for evidence-based practice (EBP))
32
232016
Recommended practical tools ORCA (or CFIR httpwwwcfirguideorg)
33
change readiness (HRET)
4 areas the innovation target audience the organisation
the environment
34
Attributing ndash use causal chain frameworks Theory informed Case evaluation
35
232016
Programme theory models or logic models
36
232016
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Recommended practical tools ORCA (or CFIR httpwwwcfirguideorg)
33
change readiness (HRET)
4 areas the innovation target audience the organisation
the environment
34
Attributing ndash use causal chain frameworks Theory informed Case evaluation
35
232016
Programme theory models or logic models
36
232016
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
change readiness (HRET)
4 areas the innovation target audience the organisation
the environment
34
Attributing ndash use causal chain frameworks Theory informed Case evaluation
35
232016
Programme theory models or logic models
36
232016
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Attributing ndash use causal chain frameworks Theory informed Case evaluation
35
232016
Programme theory models or logic models
36
232016
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Programme theory models or logic models
36
232016
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Attributing ndash use causal chain frameworks
37
232016
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Attributing ndash check implementation before
later outcomes (Proctor 2012)
38
232016
Johns quest ndash enlightenment
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Summary
Challenges both implementing and researching
implementation
One approach = Adaptive implementation research
Plan data to gather to
Document adaptation and context
Assess implementation outcomes
Attribute outcomes through causal chain map
39
232016
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Implications for you
Trial - Experimental evaluation ndash who doing
this Do we need to identify and explain ldquooutliersrdquo
Do we have data to do this
Non-experimentalobservational - ndash who (inc natural experiments) - ob
Plan and use existing data
Ask a cross-section of ldquoinformed observersrdquo
Find out implementers needs for information
effectively to take up improvements
Responsible for implementation during or
after your research
40
232016
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Implications for you
Responsibilities for implementation during or
after your research
ldquoAnother essential component of implementation
research involves the enhancement of readiness
through the creation of effective climate and culture
in an organization or communityrdquo (Implementation research glossary Brownson et al 2012)
41
232016
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Questions to you
Do you study implementation
What do you need to do better in
your implementation research
Anything a surprise
Examples or experience you have
about this 42
232016
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Questions posed
Why not implemented
What would help fast amp widespread
implementation
How can research best help
how best can implementation researchers help
practitioners to implement proven improvements
locally
Is it really their role to do so 43
232016
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
44
232016
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
What do you think Anything a surprise
Examples or experience you have
about this
45
232016
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
6)
46
232016
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
47
232016
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
48
232016
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
What do you think Anything a surprise
Examples or experience you have
about this
49
232016
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Conclusions
through
Discussion 50
232016
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Which was most surprising interesting or useful ndash vote
1)
51
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Surprises
Most useful
Might not be true for us
52
232016
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Action Pre-mortem
Crystal ball shows 1 year ahead
no results from the project
1)What are the most likely causes
2)How could we have known before
3)Possible actions we can take now
53
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Resources
54
232016
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Frameworks for studying context ndash list (pictures
later) See references
Clinicians implementing clinical research
PARHIS Guidance (Stetler et al 2011) and related
Context Assessment Index (CAI) (McCormack et al 2008) ndash the best
validated
Alberta Context Tool (ACT) (Estabrooks et al 2008)
CFIR Damschroder et al 2009 and French et al 2009
Quality Improvement Projects MUSIQ (Kaplan
et al 2012) or French et al 2009)
See also ORCA readiness for change assessment
(Helfrich et al 2009)
EMR implementation (Oslashvretveit et al 2007)
55
232016
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
BUT how many services can do thishellip
ldquoevaluating the program and continually adapting
the program on the basis of evaluation results and
changes to the context of your organization
(eg changes in staff changes to the community
changes in the population served)
can help ensure that the program remains relevant
and addresses any potential challenges that occur
over timerdquo
ldquoEvaluationrdquo = lunchtime review ltltltgtgtgtRCT 56
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Implementation approach ndash ldquo3Srdquo
Strategy Steps over time
Feb 1)Form project team
March 2)Gather initial data
April 3) etc
57
Supports
Systems for data
Facilitators 232016
ldquo3S frameworkrdquo to describe ldquoImplementation approachrdquo
Structure responsibilities accountability reporting
Example ndash QI breakthrough collaborative
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Recommended practical tools Brach et al 2008 Will it work here
58
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
French et al 2009 synthesis of 30 instruments measuring context
59
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Context for EBP (PARiHS) Rycroft
C
60
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
232016 61
Barriers analysis ndash the beginning of ldquocontext discussionrdquo
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Example 1 ndash adoption by clinicians of recommendation
for more effective treatment of depression Medications amp CBT for moderate depression
Implementation strategy Simple guidelines training feedback
on outcomes access to experts
Study found
patient outcomes 10 improved
Guess physicians following guidelines
40 not following guidelines
ldquoImplementation fidelity not metrdquo
practice changed in some but not fully and in many not at all
Phase 2 found barriers to change
consultation time extra and opinion leader ldquolukewarmrdquo
62
232016
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
2)Barriers to take up
the lack of information technology systems
physician culture
beliefs and habits
development amp function of guideline (Kenefick et al 2008)
63
232016
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Example studies of implementation strategies
Specialist-nurse led clinics for implementing lipid control cost-
effective (Mason 2005)
Multifaceted strategy for CBT cost-effective (Scheeres 2008 amp gen
Mortimer 2013)
Financial incentives for implementing ACE inhibitor amp other Qndic
cost-effective (Walker 2009)
Audit and feedback for implementing intensified control of blood
glucose is cost-effective (Hoomans 2009)
No co-payments for implementing preventive medication is cost-
effective (Choudry 2011)
Structured patient education with group for implementing self-
management is not cost-effective (Gillespie 2014)
64
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Types of fidelity 1Copy the proven intervention
Treatment practice service delivery model
Whatever it takes to reproduce this in every day life and operations
2Copy the implementation approach To enable patients to take the intevention use exacty the same reminder
system they found was effective for enabling patient uptake
3Copy both
4Copy the logic of the intervention
65
The letter kills but the spirit gives life
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
4)Copy the logic of the intervention ndash the spirit
The effective ingredients to enable practitioners to
follow hand hygiene were
Motivation (eg patient talks about MRSA)
Ability (Gell dispensers everywhere agreement
excuse for latetake longer)
Triggers (reminders)
Rewards (performance feedback etc)
You make the mix which fits your service
Is that adaption or fidelity to logic or both
66
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
5)Fidelity to guidance for adaption
for targeting or tailoring
Following the guidelines for adapting treatments
to older patients with multiple morbidity
67
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Resources
Tools
Web sites
68
232016
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
References
69
232016
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
References
70
232016
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
DETAILS
71
232016
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Practice-customerrsquos questions
Is the implementation approach effective for
enabling providers to take up the new way In setting where we can rigorously evaluate it
In a range of typical settings
Does the ldquonew wayrdquo then result in better patient and
cost outcomes
What conditions do we need to implement it
Which adaptations are possible
How much does it cost and how long before results
72
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
2) Action evaluation
Researchers share tasks with implementers
in all stages of research
Researchers present their data to
implementers
Implementers adjust intervention (or not)
+ve gain insights and data
-ve explain effects of researchers (additional
intervention)
-ve researchers not available in other typical
settings (generalisation)
73
232016
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Coleman care transitions model = people
leaving hospital - support for self care
74
232016
1)Education
2)Coach support at
home
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
RCT evaluated ndash proven effective
Research funded
version
Intervention
specified in
protocol
Implementation
not described
75
232016
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Groups on Coleman dissemination
evaluation
76
232016
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Evaluation type 1 Fidelity to planned practice Coleman 2013 core elements to get
outcomes
77
232016
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Groups on Coleman dissemination
evaluation Did this study evaluate implementation
What data did they collect How
What was ldquothe interventionrdquo and what was
ldquothe implementationrdquo activities structure and
support
How do we know if their conclusions follow
from the data or a personal opinion
78
232016
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Did subjects ldquotake uprdquo the change as
intended Evaluation type 2 for adaptive
implementation
How well did they adapt
intervention during
implementation better to achieve
implementation outcomes
Role of evaluation = a) Help
adaption b) feedback on success Models and examples
79
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Other studies
Descriptive study by facilitators of
adaptions
hellipwith unclear evidence of outcomes but
very useful to practice improvement
- More funding for and research of this type
80
232016
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Partnership research with SLL
81
Improvement
projects
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
John Oslashvretveit ndash Medical Management Center Karolinska
Institutet
Applied healthcare research in a medical
university
Grown since 2002 to 87 staff and 35 PHD
researchers
Example projects
Funding models for Value based purchasing
Implementation of improvements for chronic care
Integrated care
Most mixed methods many non-experimental
Details httpkiseenlimemedical-management-centre
John httpkiseenpeoplejohovr
httpkiseenlimeprogramme-for-improvement-
implementation-and-evaluation-research-piie
82
232016
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
Culture and translation
83
232016
The strengths of Ca and US
sect diversity
sect innovation
sect entrepreneurship
sect openness
sect can come together rapidly and attack a
problem
sect massive resources and talent
How are you going to create into the future the
new California public health 21316
ldquoNo pants metro dayrdquo
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
84
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
85
232016
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip
232016 86
Conclusions
1 Surpriseshellip
2 Usefulhellip
3 Not mentioned hellipLook this uphellip