standards for systematic reviews of clinical effectiveness research standards for systematic reviews...
TRANSCRIPT
Standards for Systematic Reviews of Clinical
Effectiveness Research
Standards for Systematic Reviews of Clinical
Effectiveness Research
Institute of MedicineJanuary 14, 2010
Institute of MedicineJanuary 14, 2010
Sandra Zelman Lewis, PhD
Asst VP, Health & Science Policy
American College of Chest Physicians
Sandra Zelman Lewis, PhD
Asst VP, Health & Science Policy
American College of Chest Physicians
Who Produces ACCP SRs?Who Produces ACCP SRs?
1. Does your organization produce its own SRs? To what extent does your organization participate in the review?
o ACCP uses 3 models:• Proposal to AHRQ for SR on narrow subtopic. Membership on TEP. Development of PICO questions. Review of final report
• ACCP issues RFP to evidence centers for direct contracting. Development of PICO questions. Oversight of process
• ACCP in-house methodologist. Infrequently. Modeled after AHRQ process
1. Does your organization produce its own SRs? To what extent does your organization participate in the review?
o ACCP uses 3 models:• Proposal to AHRQ for SR on narrow subtopic. Membership on TEP. Development of PICO questions. Review of final report
• ACCP issues RFP to evidence centers for direct contracting. Development of PICO questions. Oversight of process
• ACCP in-house methodologist. Infrequently. Modeled after AHRQ process
ChallengesChallenges
2. What are the greatest challenges in using SRs to develop CPGs?
o Dearth of SRs in areas of clinical interest/needo Appropriate patient populations (including those with
comorbidities) not always representedo Quality of process: study limitations, imprecision,
inconsistency of results, indirectness of evidence, and publication bias
2. What are the greatest challenges in using SRs to develop CPGs?
o Dearth of SRs in areas of clinical interest/needo Appropriate patient populations (including those with
comorbidities) not always representedo Quality of process: study limitations, imprecision,
inconsistency of results, indirectness of evidence, and publication bias
QualityQuality
4. Do you use any specific instruments or methods to ensure the quality of your SRs?
o Assess based on GRADE Evidence Profile criteria:. Study limitations. Imprecision. Inconsistency of results. Indirectness of evidence. Publication bias
4. Do you use any specific instruments or methods to ensure the quality of your SRs?
o Assess based on GRADE Evidence Profile criteria:. Study limitations. Imprecision. Inconsistency of results. Indirectness of evidence. Publication bias
FundingFunding
3. How are your SRs funded? Do you accept industry funding? How do you identify and address potential conflict of interest? o Funded through ACCP general funds, some NIH grants,
and/or some industry support with solid firewalls:. Supporters have no say in panel chair, composition,
evidence center, reviewers, or project staff. No influence on scope or PICO questions . Not on any calls or at any meetings (no knowledge of
them at all). No review of any manuscript; first view is publication. Panelists, evidence center staff, etc not informed of
sources of support. All panelists, EPC staff, reviewers, etc must pass
rigorous COI review (financial and intellectual)
3. How are your SRs funded? Do you accept industry funding? How do you identify and address potential conflict of interest? o Funded through ACCP general funds, some NIH grants,
and/or some industry support with solid firewalls:. Supporters have no say in panel chair, composition,
evidence center, reviewers, or project staff. No influence on scope or PICO questions . Not on any calls or at any meetings (no knowledge of
them at all). No review of any manuscript; first view is publication. Panelists, evidence center staff, etc not informed of
sources of support. All panelists, EPC staff, reviewers, etc must pass
rigorous COI review (financial and intellectual)
ACCP Conflict of Interest ProcessACCP Conflict of Interest Process
1. Initial Disclosures and Vetting:
During the guideline panel member nomination phase
1. Initial Disclosures and Vetting:
During the guideline panel member nomination phase
2. Subsequent Disclosures and Vetting:
Prior to all face-to-face meetings and on regular schedule
2. Subsequent Disclosures and Vetting:
Prior to all face-to-face meetings and on regular schedule
3. Reviewers’ Disclosures and Vetting:
Disclosures of all individuals appointed as reviewers
3. Reviewers’ Disclosures and Vetting:
Disclosures of all individuals appointed as reviewers
4. Final Disclosures:
Updated at time of submission for publication
4. Final Disclosures:
Updated at time of submission for publication
ACCP Conflict of Interest Evaluation ProcessACCP Conflict of Interest Evaluation Process
YESYES
Individual has a conflict of interest to disclose:
Review by Policy and Procedures Subcommittee, Chair, Vice-Chair, and/ or other designee and presented to full HSP Committee
Individual has a conflict of interest to disclose:
Review by Policy and Procedures Subcommittee, Chair, Vice-Chair, and/ or other designee and presented to full HSP Committee
Recommendation 3:
Unacceptable
Member is prohibited from participation
Recommendation 3:
Unacceptable
Member is prohibited from participation
Recommendation 1: Participation
Permitted COI not a source of bias
Recommendation 1: Participation
Permitted COI not a source of bias
Recommendation 2: Participation
permittedWith precepts to
preclude bias
Recommendation 2: Participation
permittedWith precepts to
preclude bias
Participation permittedIndividual has nothing to disclose, evaluation completed
Participation permittedIndividual has nothing to disclose, evaluation completed
Individual Prohibited from ParticipationIndividual Prohibited from Participation
Conflict of Interest Disclosure Form
Completed
Conflict of Interest Disclosure Form
CompletedNoNo
Recommendation 4: Referral to ACCP
Executive COI Committee
Recommendation 4: Referral to ACCP
Executive COI Committee
Setting StandardsSetting Standards
5. This IOM committee is charged with recommending standards for SRs of clinical effectiveness research. Are there steps in your SR process that could be standardized? o Use of evidence profiles to focus on the clinical outcomeso Assessment of standard quality measures (as listed
above)
6. What would be the implications for your organization if IOM were to recommend a standard grading scheme for characterizing the strength of evidence?o We would welcome the opportunity to review and
consider applying for future guidelines not yet in process.o We would also welcome the opportunity to participate in
future development of this grading scheme.
5. This IOM committee is charged with recommending standards for SRs of clinical effectiveness research. Are there steps in your SR process that could be standardized? o Use of evidence profiles to focus on the clinical outcomeso Assessment of standard quality measures (as listed
above)
6. What would be the implications for your organization if IOM were to recommend a standard grading scheme for characterizing the strength of evidence?o We would welcome the opportunity to review and
consider applying for future guidelines not yet in process.o We would also welcome the opportunity to participate in
future development of this grading scheme.
The FutureThe FutureThe FutureThe Future
The IOM report and new standards should be presented to appropriate audiences:
HTA and SR producers, evidence center methodologists, guideline developers and implementers will be attending the Guidelines International Network conference (see next slide) and guideline methodology course
Guidelines International Network (G-I-N)
2010 Conference - ChicagoAugust 26-28, 2010 – Conference Dates
August 25, 2010 – Pre-meeting Course in Guideline Methodology
Chicago, Illinois, USA
www.GIN2010.org
Host: American College of Chest Physicians