transparent and accurate reporting increases reliability, utility and impact of your research:...
TRANSCRIPT
Transparent and accurate reporting increases reliability, utility and impact
of your research:
Reporting guidelines and the EQUATOR Network
Iveta Simera
The EQUATOR NetworkCentre for Statistics in Medicine, Oxford, UK
13 November 2010
2
Biomedical research & its publication
• Biomedical research should advance scientific knowledge and – directly or indirectly – lead to improvements in prevention or management of illness
• Publications are usually the only tangible evidence that a study was done, how it was done, and what the findings were– PubMed: more then 20 mil citations
(500,000 added each year on average = around 1 per minute)
3
Purpose of research publication
• Large number of articles but are they reliable? Can they be used?
• Articles are written for multiple readerships:– Clinicians
•To learn how to treat their patients better– Researchers:
•To inform their own research•To help plan a similar study •To include the study in a systematic review
– Patients/consumers:•To aid personal decision-making
– Policy makers/purchasers: •To aid policy decision-making
• … should present sufficiently detailed information to allow assessment of study reliability and relevance and comparison across studies (important for SR and CPG)
4
Much evidence of poor reporting
• Hundreds of reviews of methodology of published research articles– Highlighting severe
deficiencies in reporting (biased or unusable research reports)
– These deficiencies limit or prevent use of many of the published findings further in research or clinical practice
5
Examples: Poor reporting of RCT methodolgoy
• 519 RCTs published in Dec 2000: failing to report key aspects of methodology
– 79% did not report the method of random sequence generation *
– 82% did not report the method of allocation concealment *
– 73% did not report a sample size calculation *– 55% did not report a defined primary outcome (s) – 60% did not report whether blinded
(Chan & Altman Lancet 2005)
• 616 RCTs published in Dec 2006: – Reporting of several important aspects of trial methods
improved between 2000 and 2006 (*); however, the quality of reporting remains well below an acceptable level.
(Hopewell BMJ 2010)
6
Example: Poor description of intervention
• Glasziou et al. (BMJ 2008) – assessed descriptions of treatments in 80 published
articles(55 randomised trials & 25 systematic reviews)
– crucial elements of the interventions were missing in 41 of those studies (of 25 SR only 3 provided intervention description sufficient for implementation)
• Herbert & Bo (BMJ 2005)– Discuss effect of quality of intervention on SR– “Assessment of the quality of the intervention relies on
sufficient detail in trial reports, but many reports provide only superficial description of complex interventions. .. Interventions should be described in sufficient detail to enable reader to assess if the intervention was administered well as the quality of intervention can affect the results of clinical trials.”
7
Systematic reviews and clinical guidelines
• Poor reporting is a serious problem for SR and CPG
• In the UK – Cochrane reviews are highly regarded and yet ..– “The biggest problem was the quality of reporting, which
did not allow us to judge the important methodological items ...”
– “Data reporting was poor. 15 trials met the inclusion criteria for this review but only 4 could be included as data were impossible to use in the other 11.”
– “If the CONSORT recommendations were followed in the reporting of future studies, the effects of Morita therapy would be clearer. Much important data within the included studies were so poorly reported that clinicians, funders and recipients of care might have reason to feel let down by the research community.”
(Cochrane Library, accessed on 18 Sept 07)
8
Examples: Poor reporting of systematic reviews
• Reviews are not immune to the problems of poor reporting– Epidemiological and reporting characteristics and
bias-related aspects of 300 systematic reviews (of which 125 were Cochrane reviews). The overall quality of reporting of key aspects of methodology was very inconsistent with particularly discouraging findings for non-Cochrane reviews.
(Moher; PLoS Med, 2007, 4(3), e78)
9
Common problems in research reporting
• Unethical reporting practices with serious adverse consequences– Non-reporting or delayed reporting of whole studies– Omissions or misinterpretation of results in abstracts– Omission of crucial information in the description of
research methods and interventions– Inconsistencies between study protocol and publication– Selective reporting of only certain outcomes– Inadequate statistical reporting– Presenting data (graphs) in confusing or misleading ways
(particularly important for presenting benefits and harms)– General misinterpretation of study results (spin)
10
How to improve research reporting?
11
How to improve research reporting?
12
How to improve research reporting?
13
Reporting guidelines (RG)
• RG provide structured advice on what to include in a research report
• RG specify a minimum set of items required for a clear and transparent account of what was done and what was found in a research study, reflecting in particular issues that might introduce bias into the research
• Benefits of using RG:– Improved accuracy and transparency of publications – Easier appraisal of reports for research quality and relevance– Better further use of presented findings – Improved efficiency of literature searching
14
RG example: CONSORT Statement
15
http://www.consort-statement.org/
16
17
Other reporting guidelines
• PRISMA (SR/meta-analyses of RCTs) • STARD (diagnostic studies) • STROBE (observational studies)
... and many others (over 100 RG identified)
• As yet most guidelines have had limited impact– Passive dissemination through publication only - not
widely known– Complience not required by journals
18
EQUATOR Network
• EQUATOR Network is an international initiative set up to improve reliability and value of medical research literature
• EQUATOR promotestransparent accuratecompleteand timely
reporting of health research studies
19
EQUATOR main goals and activites
Website launched in Oct 2007
www.equator-network.org
EQUATOR seven major goals
• Promote responsible research reporting in practice (wider use of RG)
• Develop a comprehensive online resource centre
• Develop and establish an education and training programme
• Assist in the development, dissemination and implementation of
robust reporting guidelines
• Expand EQUATOR activities globally
• Assess use of reporting guidelines
• Audit reporting quality across the health research literature
20
EQUATOR online resources
All collated resources are available in our Library
Some of the resources are grouped according to relevance to our main user groups
21
Resources for editors
22
Guidance on reporting in journals’ I to A:examples
• General medical journals:
• Prakticky lekar– http://www.prolekare.cz/prakticky-lekar-pokyny– Very brief and general instructions on research
reporting
• BMJ (British Medical Journal)– http://resources.bmj.com/bmj/authors/types-of-
article/research – Very detailed, clearly worded requirements and links
to internationally accepted guidelines
23
Prakt Lekar – I to A
24
BMJ - I to A
25
Guidance on reporting in journals’ I to A:examples
• Specialty (Association) journals:
• Cesko-Slovenska Dermatologie– http://www.prolekare.cz/cesko-slovenska-
dermatologie-pokyny – Some instructions on reporting scientific content, no
reference to any internationally accepted RG
• British Journal of Dermatology– http://www.wiley.com/bw/submit.asp?ref=0007-
0963&site=1
26
Cesko-Slovenska Dermatologie
27
BJD
28
How to shift the ‘reporting culture’
• Collaboration of all parties involved in research publishing needed on a global scale– Scientists, research organisations, funders and regulators– Journals (editors, peer reviewers, publishers)– Other organisations (higher education, REC, ..)
• Working towards:– .. accurate, complete, transparent and timely reporting of
research studies is considered a ‘norm’ (not something extra)
• How to achieve this?– Clearly defined policies, requirements and expectations– Provision of tools and other resources– Education and training– Motivation and incentives– Application of safeguards and checks
• See ‘Steps to consider’ in Simera et al. BMC Medicine 2010,8:24
29
EQUATOR on a global scale
• Last year, our website has been visited from most countries in the world
• Traffic doubled compared with the previous year
30
EQUATOR Spanish website – launched July 2010
We are looking for collaborators to establish local centres of activities supporting better reporting of research studies
Signed collaboration agreement with PAHO to raise standards of research reporting in South America and Caribbean
31
Czech medical research contributes to SR
• More difficult to write a good paper (although many really bad papers were written by English native speakers!)– Important is a good understanding of principles of
good research methodology (including terminology) and principles of responsible reporting
– Translating educational materials (e.g. E&E papers) into ‘local’ languages can improve understanding
-Selected for inclusion in Cochrane SR of RCTs
-No information about pts allocation into groups
-6 emails with the authors
-Final answer:
-“It was a double-blind study”
www.equator-network.org
Dr Iveta Simera, Head of Programme DevelopmentEQUATOR Network, Centre for Statistics in Medicine, Oxford, UK
33
EQUATOR Steering Group
Doug Altman, Centre for Statistics in Medicine, University of Oxford, UK
John Hoey, University of Toronto, Canada
Ana Marusic, University of Split, Croatia
David Moher, Ottawa Health Research Institute, Canada
Kenneth F. Schulz, Family Health International, Chapel Hill, USA