Formulation of a Strategy: A Formulation of a Strategy: A Framework for ActionFramework for Action

IOM Workshop on Standards for Systematic Reviews and IOM Workshop on Standards for Systematic Reviews and Clinical Practice GuidelinesClinical Practice Guidelines

May 10-11, 2011May 10-11, 2011University of California-Washington Conference CenterUniversity of California-Washington Conference Center

Stephanie ChangStephanie ChangAHRQAHRQ

AgendaAgenda Disclaimers and DisclosuresDisclaimers and Disclosures ReflectionsReflections A Framework for actionA Framework for action

Disclaimers and DisclosuresDisclaimers and Disclosures

Represent self not EPCsRepresent self not EPCs Financial COIFinancial COI

– AHRQ employeeAHRQ employee Intellectual COIIntellectual COI

– GeneralistGeneralist– Academic research vs EHC programAcademic research vs EHC program– EPC Program EPC Program – Funder of systematic reviewsFunder of systematic reviews– GovernmentGovernment

Reflections on standardsReflections on standards

CPG have less standards, but more CPG have less standards, but more controversycontroversy– SR: 84 instructions, 21 standards, 4 areasSR: 84 instructions, 21 standards, 4 areas– CPG: 20 instructions, 8 standardsCPG: 20 instructions, 8 standards

SR considered a more mature scienceSR considered a more mature science– In early adulthood?In early adulthood?– From wild west to assembly lineFrom wild west to assembly line

Definition of standardDefinition of standard– Provisional, needing beta testingProvisional, needing beta testing– Sets an ideal or goalSets an ideal or goal– Intended to provide uniformity, justificationIntended to provide uniformity, justification– ““doesn’t mean you have to do all of them”doesn’t mean you have to do all of them”– No minimum threshhold, no weightingNo minimum threshhold, no weighting

Reflections on SR standardsReflections on SR standards

General agreementGeneral agreement Areas of disagreement due to limited Areas of disagreement due to limited

resources - Good science vs practicalitiesresources - Good science vs practicalities– Is there a minimum?Is there a minimum?– When evidence and resources are limited how to When evidence and resources are limited how to

weigh values weigh values ValidityValidity TransparencyTransparency Patient centerednessPatient centeredness Timliness, efficiency, utilityTimliness, efficiency, utility

Research bias? Research bias? Balance between internal and external validity – is no Balance between internal and external validity – is no

review better than review with potential bias or error?review better than review with potential bias or error?

Implications on need for better coordination, Implications on need for better coordination, tools to help automate processestools to help automate processes– May result in fewer SR, GLMay result in fewer SR, GL

A framework for actionA framework for action

IOM work on outlining standards is doneIOM work on outlining standards is done Pilot testingPilot testing

– Can it work?Can it work? Has this been done by any group? Is the tool Has this been done by any group? Is the tool too unwieldy? How much resources will it take? too unwieldy? How much resources will it take?

– Will it work?Will it work? Is there face validity to the criteria? Are groups Is there face validity to the criteria? Are groups willing to implement the standards?willing to implement the standards?

– Is it worth it?Is it worth it? Are SRs (or CPGs) better, more valid? Do the Are SRs (or CPGs) better, more valid? Do the benefits outweigh the costs?benefits outweigh the costs?

Uptake depends on:Uptake depends on:– Face validity and willingness by implementers Face validity and willingness by implementers – Environment that supports standardsEnvironment that supports standards– Regulations and requirementsRegulations and requirements

FundersFunders PublishersPublishers ?Regulators??Regulators?

Evaluating the standardsEvaluating the standards

Generally accepted vs variationGenerally accepted vs variation– Limited evidence, disagreement on validityLimited evidence, disagreement on validity– Prioritization due to limited resourcesPrioritization due to limited resources

Adopt and implement those that are agreed Adopt and implement those that are agreed uponupon

Consider ability to gather empiric evidenceConsider ability to gather empiric evidence– Standards to improve relevance/usefulnessStandards to improve relevance/usefulness– Standards to improve validityStandards to improve validity– Standards to improve Standards to improve

transparency/trustworthinesstransparency/trustworthiness Consider if values outweigh resource issues in Consider if values outweigh resource issues in

areas where evidence cannot help.areas where evidence cannot help.

AHRQ workAHRQ work

Training materialsTraining materials– Training modulesTraining modules– Community ForumCommunity Forum

ToolsTools– SRDR, Abstraktr, Meta-analystSRDR, Abstraktr, Meta-analyst

Funded partnersFunded partners– GRADE workshopGRADE workshop– CUE, Community ForumCUE, Community Forum

Systematic reviews, translation guides and other Systematic reviews, translation guides and other dissemination materialsdissemination materials– Post all topic nominationsPost all topic nominations

Guidelines Clearinghouse, USPSTFGuidelines Clearinghouse, USPSTF Methods researchMethods research

http://effectivehealthcare.ahrq.govhttp://effectivehealthcare.ahrq.gov