implementing the grade method in guideline development: real- world experiences contemplation stage:...
TRANSCRIPT
Implementing the GRADE Method in Guideline Development: Real-
World Experiences
Contemplation Stage: To GRADE or Not to GRADE?
Sheila A. Agyeman, MHA
Director of Evidence Based Medicine
American Gastroenterological Association Institute
September 15, 2009
AGA Institute Guideline Development Process Prior to 2007
Searching for and appraising the evidence
•Authors performed literature searches
•Table below was provided to aide authors in organizing findings but not a requirement
•Evidence summarized in Technical Review document
•Technical review (intended for academics) summarized into Medical Position Statement (intended for practicing clinicians)
Reference Year Study Design/type
No. of Subjects
Length of Study
Intervention Outcome Measurement
Statistical Analysis
Conclusions Comments
AGA Institute Guideline Development Process Prior to 2007
Inconsistencies identified in Pre-2007 Process
•Technical review and medical position statement not adequately evidence based
•Disconnect between technical review and medical position statement
Not guiding readers in interpreting the data
Not giving clear clinical recommendations
•Concept of quality of evidence and strength of recommendations not well emphasized
AGA Institute Guideline Development Current Process
Modifications made to the process• Construction of specific clinical questions to be addressed
in technical review
• Implementation of U.S. Preventive Services Task Force (USPSTF) Ratings – Management of GERD guideline
• Composition of Medical Position Panel o Patient/patient advocateo Payero Gastroenterologist in community practiceo Gastroenterologist with knowledge in health services
research o Primary care physician, surgeon or pathologist (included
as needed based on topic)
AGA Institute Guideline Development Current Process
Challenges of current process
•USPSTF grading system primarily designed for preventive services
•USPSTF grading system changed after first guideline was already in the final stages (mid 2007)
•USPSTF grading system caused confusion amongst guideline authors (Dysplasia in IBD guideline) how to grade evidence and recommendations (e.g., diagnostic tests)
AGA Institute Guideline Development Current Process
What made the AGA Institute decide to consider switching from USPSTF to using GRADE?
•Invitations to collaborate with other GI and non-GI societies
•Establish uniformity across all GI societies
Further invitations for collaboration led to the discovery that more societies are using or also considering GRADE
Considered using GRADE to reduce the confusion in different grading schemes used in various guidelines
Joining effort to establish an international standard
AGA Institute Guideline Development Current Process
Benefits of using GRADE we considered:
•A very rigorous and formalized process that will ensure evidence-based guidelines necessary for value-based health care initiatives (e.g., P4P)
•Already closely resembles AGA Institute’s current process
•Provides increased transparency of judgments of quality of evidence and strength of recommendations
•Investment into GRADE evidence profiles in the long-term could be future asset at the time of guidelines update
•Availability of GRADE methodologists who could assist in the process
AGA Institute Guideline Development Current Process
Initial reservations about using GRADE
•Higher costs?
Need to hire full time methodologist? Answer: GRADE not necessarily more complex than other grading systems.
Increased training of staff and authors? Answer: Any grading system would require some form of training
•Requires more time and lengthens the development process?
• Answer: No. The most time consuming and costly phase of the guideline development process is the systematic review of the evidence. This is independent from the grading system used.
AGA Institute Guideline Development Current Process
Next Steps
•Develop an implementation plan
Train staff in GRADE process
Train committee members in GRADE process
Include GRADE experts on committee
•Include in budget cycle for 2010-2011