how can we improve the quality of cochrane reviews (in 8 minutes)? david tovey
TRANSCRIPT
How can we improve the quality of Cochrane Reviews
(in 8 minutes)?
David Tovey
What is quality?
?
Valid
Relevant
Accessible
Timely
Moving from process to end point
Moving from process to end point
What is a “good enough” review?
“You should see the rubbish that is submitted to us..”
“And the authors are volunteers....so what can we do?”
Hmmm....what would she do?
What does these chaps think about improving quality?
What’s the plan?
Agree standards
Set standards
Measure against standards
Act
And training....
Results: 3 priorities
• Presenting effects
• Significance and non significance
• Harms
Absolute and relative effects
“If Cochrane reviews continue to express results solely in [relative] terms, they will continue to mislead clinicians, reporters, and the general public in just the way the pharmaceutical and vaccine companies would like.”
“****s were significantly more effective for [outcome] than placebo (OR 3.20, 95% CI:2.85, 4.27).”
“ A recommended approach is to re-express an odds ratio or a risk ratio as a variety of NNTs across a range of assumed control risks...”
“If a relative effect measure ..is chosen for meta-analysis, then a control group risk needs to be specified as part of the calculation of an ARR or NNT”
No evidence of an effect?
“Combined data from the ... studies revealed a non-significant difference between groups”
“There was a trend towards an increase in risk of [outcome]”
“Review authors are advised not to describe results as not statistically significant or non significant”
0 Favours interventionFavours control
0 Favours interventionFavours control
0 Favours interventionFavours control
Harms“A Cochrane review that considers only the favourable outcomes of the interventions....will lack balance and make the intervention look more favourable than it should..
Different types of studies may be needed to evaluate different outcomes”
Harms“A Cochrane review that considers only the favourable outcomes of the interventions....will lack balance and make the intervention look more favourable than it should..
Different types of studies may be needed to evaluate different outcomes”
“Observational studies are almost always necessary to assess harms adequately”1
1Chou, Aronson, Adkins et al 2010
Conclusions
• Review quality is of paramount importance (Strategic review recommendation #1)
• Quality is multi-faceted
• We need to work together to identify, agree and implement explicit review quality standards – Across review groups (warranted and unwarranted
variation)