making information useful for decision makers · making information useful for decision makers...

39
Making information useful for decision makers Jeremy Grimshaw MD, PhD Director of the Canadian Cochrane Centre and Network Clinical Epidemiology Program, Ottawa Health Research Institute Department of Medicine, University of Ottawa

Upload: others

Post on 08-May-2020

19 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Making information useful for decision makersJeremy Grimshaw MD, PhDDirector of the Canadian Cochrane Centre and NetworkClinical Epidemiology Program, Ottawa Health Research InstituteDepartment of Medicine, University of Ottawa

Page 2: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Outline

• Background• Effectiveness of strategies targeting different

decision makers• Current knowledge translation concepts• Health technology applications

Page 3: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Background

• Health technology assessments aim to inform decision making at all levels of health care systems.

• Considerable development over last two decades in health technology methods.

• Health technologies are struggling to optimise the dissemination and implementation of health technology assessments to maximise their impact.

Page 4: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Background

Potential target audiences for HTA• Policy makers

– Macro (national/provincial)– Meso (organisational)

• Managerial• Provider• Consumer

Page 5: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Background

• Most of the approaches to dissemination and implementation are more often based on beliefs than on scientific evidence

• ‘Evidence based medicine should be complemented by evidence based implementation’

Grol (1997). British Medical Journal.

Page 6: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Effectiveness of strategies targeting different decision makers

Cochrane Effective Practice and Organisation of Care (EPOC) Group aims to undertake systematic reviews of interventions to improve healthcare delivery and healthcare systems including:

• Professional interventions (e.g. continuing medical education, audit and feedback)

• Financial interventions (e.g. professional incentives)• Organisational interventions (e.g. the expanded role of

pharmacists)• Regulatory interventions

Bero, Eccles, Grilli, Grimshaw, Gruen, Mayhew, Oxman, Zwarenstein (2005). Cochrane Library.

Page 7: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

EPOC progress to date - register and reviews• Register of 3500+ primary studies• 33 reviews, 26 protocols• Collaborating with over 180 researchers from 12

countries

Bero, Eccles, Grilli, Grimshaw, Gruen, Mayhew, Oxman, Zwarenstein (2005). Cochrane Library.

Effectiveness of strategies targeting different decision makers

Page 8: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Effectiveness of strategies targeting policy makers and managers

• Health policy-makers perceptions of their use of evidence: a systematic review

• 24 studies involving 2014 interviews with health policy makers

• Most studies focused on hypothetical scenarios or retrospective perception of the use of evidence in relation to specific cases.

Innvaer, Vist, Trummald, Oxman (2002). Journal of Health Services Research and Policy

Page 9: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Effectiveness of strategies targeting different decision makers

• Facilitators– Personal contact (13/24)– Timely relevance (13/24)– Inclusion of summaries with policy recommendations

(11/24)• Barriers

– Absence of personal contact (11/24)– Lack of timeliness or relevance of research (9/24)– Mutual mistrust (8/24)– Power and budget struggles (7/24)

Innvaer, Vist, Trummald, Oxman (2002). Journal of Health Services Research and Policy

Page 10: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Effectiveness of strategies targeting health care professionals

Effective Health Care Bulletin (1999). Getting evidence into practice. Effective Health Care Bulletin, 5(1). London, Royal Society of Medicine Press.

Also available from:http://www.york.ac.uk/inst/crd/ehc51.pdf

Page 11: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Effectiveness of strategies targeting health care professionals

Multifaceted interventions

Educational outreachLocal opinion leadersDidactic educational session

RemindersAudit and feedbackDissemination of printed educational materials

Generally effectiveMixed effectsGenerally ineffective

Bero et al (1998). BMJGrimshaw et al (2002). Medical Care

Page 12: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Effectiveness of strategies targeting health care professionals

Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay C, Vale L et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess2004.

(Available from:http://www.hta.nhsweb.nhs.uk/)

Page 13: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Effectiveness of strategies targeting health care professionals

Results• Included 285 reports of 235 studies, yielding

309 separate comparisons• Overall methodological quality poor (eg unit

of analysis errors common)• Poor description of interventions• Only 27% of studies used theories and/or

psychological constructs • 29.4% comparisons reported any economic

data

Page 14: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Effectiveness of strategies targeting health care professionals

–1.0%, +34.0%

+1.3%, +16.0%

+3.6%, +17.0%

Range

+14.1%14Reminders

+7.0%5Audit and feedback

+8.1%5Educational materials

Median effect size

Number of CRCTs

Intervention

Single interventions

Page 15: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Effectiveness of strategies targeting health care professionals

16 28 46 63 56 N =

Absolute effect size

Number of interventions in treatment group

>4 4 3 2 1

80%

60%

40%

20%

0%

-20%

-40%

-60%

-80%

Multifaceted interventions

Page 16: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Effectiveness of strategies targeting health care professionals

Multifaceted interventions including educational outreach•13 CRCT

Range –4% to +17.4% (median +6.0%)

Page 17: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Effectiveness of strategies targeting health care professionals

• Improvements in direction of effect in 86% of comparisons

• Reminders most consistently observed to be effective

• Educational outreach only led to modest effects • Dissemination of educational materials may lead

to modest but potentially important effects (similar effects to more intensive interventions)

• Multifaceted interventions not necessarily more effective than single interventions

Page 18: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Effectiveness of strategies targeting patients

• Cochrane Consumers and Communication Review Group aims to undertake systematic reviews of interventions to improve communication with consumers

• Register of 4538 primary studies• 12 systematic reviews and 20 protocols

– Decision aids for people facing health treatment or screening decisions

– Interventions for helping patients to follow prescriptions for medications

– Recordings or summaries of consultations for people with cancer

Page 19: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Effectiveness of strategies targeting patients

• Decision aids for people facing health treatment or screening decisions

• Included 35 RCTs• Decision aids improved:

– knowledge – more realistic expectations– lower decisional conflict related to feeling informed– increased proportion of people active in decision making– reduced proportion of people who remained undecided post

intervention• Decision aids had a variable effect on which healthcare options

were selected.O'Connor AM, et al (2003). Cochrane

Page 20: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Effectiveness of strategies targeting different decision makers

Summary• Relatively sparse evidence base especially for

policy makers and managers• Health technology agencies need to use

considerable judgment about how to disseminate HTAs to different target audiences

Page 21: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Current knowledge translation concepts

• Knowledge translation is about ensuring that:– stakeholders are aware of and use research evidence

to inform their decision making – research is informed by current available evidence

and the experiences and information needs of stakeholders

Page 22: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Knowledge translation models

• Push• Pull• Linkage and exchange• Knowledge brokers

Page 23: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Push models (1)

• Improving dissemination by researchers• Five key questions

– What should be transferred?– To whom should research knowledge be transferred?– By whom should research knowledge be transferred?– How should research knowledge be transferred?– With what effect should research knowledge be

transferred?

Lavis JN, Robertson D, Woodside JN, Mcleod CB, Abelson J (2003) Milbank Quarterly

Page 24: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Push models (2)

• Multiple approaches targeting different stakeholder groups– Actionable messages– Policy briefings (1, 3, 25)– Policy seminar and workshops– Toolkits– Peer reviewed papers– Other – patient decision aids, guidelines

Page 25: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Pull models

• Increase demand from decision makers• Enhancing receptor capacity

– Critical appraisal skills– EXTRA training program– Policy maker secondments

Page 26: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Linkage and exchange

• Creating new relationships between HTA agencies and policy makers to enhance mutual understanding and develop partnerships for framing and conducting HTA.

Page 27: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Knowledge brokers

• Using knowledge brokers to build and nurture relationships between those involved in joint knowledge production (researchers and policy makers)

Page 28: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Making information useful for decision makers

• Health technology agencies need clarity on their role in dissemination and implementation activities and what they should be held accountable for.

Page 29: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Making information useful for decision makers

• Improving receptivity of stakeholder audiences– Developing relationships with key stakeholder

audiences– Knowledge brokers, regular workshops,

regular communications• Improving linkages with partner organizations

with complementary responsibilities

Page 30: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Making information useful for decision makers

• Development of a formative dissemination plan throughout all stages of HTA process including– Planning HTA– Conduct of HTA– Interpretation of HTA findings– Dissemination of HTA findings– Further KT activities

Page 31: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Making information useful for decision makers

• Five key questions– What should be transferred?– To whom should research knowledge be transferred?– By whom should research knowledge be transferred?– How should research knowledge be transferred?– With what effect should research knowledge be

transferred?

Lavis JN, Robertson D, Woodside JN, Mcleod CB, Abelson J (2003) Milbank Quarterly

Page 32: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Making information useful for decision makers

Planning and conduct of HTA• Identifying potential stakeholders (policy makers,

managers, health care professionals, patients)• Engagement in peer review of HTA protocol to ensure

that HTA questions and methods will address decision makers information needs

• Engagement of stakeholders in advisory group throughout HTA process to:– advise on conduct and interpretation of HTA – develop stakeholders as champions of HTA– sensitize stakeholders about forthcoming HTA

Page 33: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Making information useful for decision makers

Disseminating findings of HTA• Dissemination plans may vary depending on target

group and HTA findings• Base model

– Development of actionable messages– Use of multiple formats and channels

• Policy maker briefing papers (1,3,25)• Publications• Website

• Additional specific activities often with partner organizations

Page 34: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Example – horizon scanning

Regulatory Body

National Policy Makers

Local Policy Makers

Professional Organizations

Health Care Professionals

Patients Research Funders

Researchers Medical Insurers

Stop useStop use, promote researchPromote use for limited indicationsPromote use for new indicationPromote widespread use

Abbreviations: P = primary target audience for early warning message; S = secondary target audience for early warning message

Page 35: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Example - early warning message concerning a new drug

Regulatory Body

National Policy Makers

Local Policy Makers

Professional Organizations

Health Care Professionals

Patients Research Funders

Researchers Medical Insurers

Stop use P P S S S S PStop use, promote research P P S S S S P P PPromote use for limited indications P P P S P P PPromote use for new indication P P P S P S PPromote widespread use P P P S P P P

Abbreviations: P = primary target audience for early warning message; S = secondary target audience for early warning message

Page 36: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Example - early warning message concerning a new surgical procedure

Regulatory Body

National Policy Makers

Local Policy Makers

Professional Organizations

Health Care Professionals

Patients Research Funders

Researchers Medical Insurers

Stop use S P P P P PStop use, promote research S P P S S P P PPromote use for limited indications S P P P P PPromote use for new indication S P P P S PPromote widespread use S P P P P P

Abbreviations: P = primary target audience for early warning message; S = secondary target audience for early warning message

Page 37: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Summary

• Health technology agencies (and evidence based organizations) are struggling to optimise the dissemination and implementation of health technology assessments to maximise their impact.

• Limited but potentially informative evidence base about how to target different stakeholder groups

Page 38: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Summary

• Future activities should focus on:– developing relationships with key stakeholder

audiences– embedding knowledge translation throughout

planning and conduct of HTAs– tailoring KT activities to target audience and key

messages• Further research is needed to develop better

evidence base to support future dissemination and implementation activities

Page 39: Making information useful for decision makers · Making information useful for decision makers Planning and conduct of HTA • Identifying potential stakeholders (policy makers, managers,

Contact details

[email protected]• EPOC – [email protected]• Grimshaw JM, Thomas RE, MacLennan G, Fraser C, Ramsay C,

Vale L et al. Effectiveness and efficiency of guideline dissemination and implementation strategies. Health Technol Assess 2004.(Available from: http://www.hta.nhsweb.nhs.uk/)