statistical presentation in international scientific publications 2. a symptomatic review malcolm...
Post on 19-Dec-2015
214 views
TRANSCRIPT
Statistical presentation in international scientific publications 2. A symptomatic review
Malcolm CampbellLecturer in Statistics, School of Nursing, Midwifery &
Social Work, The University of Manchester
Statistical Editor, Health & Social Care in the Community
26 March 2008 Statistical presentation - 2. A symptomatic review 2
2. A symptomatic review Contents
• 2.1 Statistical reporting
• 2.2 General (but not definitive) advice
• 2.3 The structure of a paper
• 2.4 Statistical errors in submitted papers
• 2.5 What should we do?
26 March 2008 Statistical presentation - 2. A symptomatic review 3
2.1 Statistical reportingPresenting statistics in scientific papers
• A scientific research paper should contain sufficient information to allow readers to
1. assess the authors’ aims, material, methods, findings and interpretation
2. decide whether their conclusions are justified
3. apply the authors’ conclusions to their own practice
4. repeat the study in their own setting, if required
• Quantitative research papers show (or hide?) information statistically– numbers, estimates, statistics, test results, p-values…
– but the reader still needs to be able to do 1 to 4
26 March 2008 Statistical presentation - 2. A symptomatic review 4
Statistical reporting in research papersLies, damned lies and submissions to journals
• Papers submitted to healthcare journals have a generally poor standard of statistical reporting– many papers do not have the right structure
– occasionally the study design is at fault
– most authors choose the correct methods to analyse their data but fall down on the description of the methods and the presentation of the results
– others present sequences of p-values without explanation, inviting the reader to “guess the test”
– many brush non-significant p-values out of the way
– tables of findings are often inconsistent, too complicated or depend on the text for their explanation
26 March 2008 Statistical presentation - 2. A symptomatic review 5
2.2 General (but not definitive) adviceConventions, recommendations and suggestions
• These presentations give general advice on how to present statistical findings in papers– there are common conventions, style and sense
– some recommendations on good practice
• There are few absolute rules– different journals (and editors) have different styles
• always read the journal’s “Instructions to Authors”
– it may be necessary to bend the rules occasionally
– accept that research in healthcare may not be perfect!
• Important bottom line: readability
26 March 2008 Statistical presentation - 2. A symptomatic review 6
The Good, the Bad and the UglyExamples of good, bad and ugly practice
• We will look at examples of all three in published papers
– “The Good” are ideals
– “The Bad” may be slightly wrong
– but “The Ugly” should have been corrected before printing
26 March 2008 Statistical presentation - 2. A symptomatic review 7
2.3 The structure of a paperQuantitative papers are usually laid out as follows
• Title: What was the study about, in a line?
• Authors: Who “performed” the study?
• Abstract: What was the study about, in a paragraph?
• Introduction: Why was the study carried out?
• Methods: How was the study carried out?
• Results: What did the study find?
• Discussion: What do the findings mean?
• Acknowledgements: Who should be thanked?
• References: What essential references were used?
26 March 2008 Statistical presentation - 2. A symptomatic review 8
IMRaD for healthcare papersThe standard structure for research papers (ICJME, 2005) • Many (but not all) healthcare journals adopt this
structure for main text of research papers– Introduction, Methods, Results and Discussion
– these sections may be broken down into subsections
• It reflects the process of scientific investigation– why the study was done, how it was done, what was
found, what it means
• though not always the order in which things were done!
– use IMRaD even when planning for a journal that uses other sections to get overall structure right
• should be easy to make minor changes to the structure before submission
26 March 2008 Statistical presentation - 2. A symptomatic review 9
Structures in selected journalsSections used by healthcare journals (* = structured Abstract)
BMJ HSCC Midwifery J Adv. Nursing
J Clinical Nursing
Nurse Educ Today
Abstract* Abstract Abstract* Abstract* Abstract* Abstract
Introduction Introduction Introduction Introduction/
Background
Introduction/
Background
Introduction/
Lit review
Lit review Background
Methods Methods Methods The study Methods Methods
Results Results Findings Results Results Results
Discussion Discussion Discussion Discussion Discussion Discussion
Conclusions Conclusions Conclusions
Acknowledg Acknowledg Acknowledg Acknowledg
Refs (Vanc) Refs (Harv) Refs (Harv) Refs (Harv) Refs (Harv) Refs (Harv)
26 March 2008 Statistical presentation - 2. A symptomatic review 10
Common faults with authors’ structureHow not to lay out a quantitative paper (eg Hall et al, 1998)
• The Introduction should not include any findings or conclusions
• The Methods should not include any findings– information not known before the data were collected
should ideally be in the Results
• The Results should not repeat detailed findings already summarised in tables
• The Discussion should not introduce findings not already in the Results
• The References should not be in the style of another journal (ie not the BMJ!)
26 March 2008 Statistical presentation - 2. A symptomatic review 11
Methods or Results?What goes where (eg ICJME, 2007)
• International Committee of Medical Journal Editors (2007) at http://www.icmje.org/ :– “The Methods section should include only information that
was available at the time the plan or protocol for the study was written; all information obtained during the conduct of the study belongs in the Results section. ”
• I would expect the Results to start with – number of participants, participation rate, details of
characteristics of the participants…
• Sometimes better for readability to include some “results” in the Methods
• Not all papers or journals follow this!
26 March 2008 Statistical presentation - 2. A symptomatic review 12
The BadLack of structure confusing
• Papanikolaou et al (2003)– Pressure ulcer risk assessment: application of logistic
analysis, J Advanced Nursing 44(2), 128-136
• title misleading – they meant logistic regression
• does not follow IMRaD structure
– characteristics of participants in Methods section
– analysis of variables used for logistic regression with percentage breakdown per group also in Methods section
– Results starts with the main analysis, logistic regression
• [also no sample size calculation (only 25 patients with pressure ulcers v 473 without)
• inconsistent percentages and p-values of “0.00”]
26 March 2008 Statistical presentation - 2. A symptomatic review 13
2.4 Statistical errors in submitted papers100 consecutive non-RCT peer reviews+ (Altman, 1998)
Yes Unclear No
Objective clear? 83 6 11
Appropriate study design? 72 25 3
Source of subjects? * 83 6 10
Sample size calculation? * 0 0 63
Satisfactory response rate? * 49 23 2
Methods described adequately? 47 0 53
Statistical analyses appropriate? 41 37 22
Statistical presentation satisfactory? 14 0 86
Confidence intervals given? 51 0 41
(CIs given inappropriately) (+8)
Conclusions justified? 40 49 11
Paper statistically acceptable? 4 0 96
If not, could it become acceptable? 89 6 1
* not always applicable + for BMJ 1991-1993
26 March 2008 Statistical presentation - 2. A symptomatic review 14
2.5 What should we do? Never fear - help is at hand! – Three sources of advice
1. ICMJE uniform requirements for manuscripts– International Committee of Medical Journal Editors
(2007) at http://www.icmje.org/
• eg use double spacing, pages should be numbered, start each section/table/figure on a new page
2. Journal’s instructions to authors– please read them!
3. Published reporting guidelines for different study designs– eg CONSORT, STARD, TREND, STROBE…
26 March 2008 Statistical presentation - 2. A symptomatic review 15
Guidelines for statistical reportingStandardised reporting guidelines
• There are published guidelines for reporting certain types of study designs– cover the type of information to be reported and in what
order/sections of the paper
– but not the finer details of numerical reporting
• refer to journals’ guidelines to authors
• and more later!
• Go to the EQUATOR Network for more details– a new initiative, Enhancing the QUAlity and
Transparency Of health Research Network
– http://www.equator-network.org/
26 March 2008 Statistical presentation - 2. A symptomatic review 16
The CONSORT statement (2001)Checklist/flow diagram for reporting randomised trials
• CONsolidated Standards Of Reporting Trials – for reporting (parallel) randomised controlled trials
– the statement – eg Moher et al (2001)
– explanation and elaboration – eg Altman et al (2001)
– http://www.consort-statement.org/
• download both documents and templates of checklist and flow diagram
26 March 2008 Statistical presentation - 2. A symptomatic review 17
The STARD statement (2003) Guidelines for studies of diagnostic accuracy
• STAndards for Reporting of Diagnostic Accuracy – for studies comparing outcomes from new diagnostic
test against reference standard
– the statement – Bossuyt et al (2003a)
• CONSORT adapted for studies of diagnostic accuracy
– explanation and elaboration – Bossuyt et al (2003b)
– http://www.stard-statement.org/
• download both documents and templates of checklist and flow diagram
26 March 2008 Statistical presentation - 2. A symptomatic review 18
The TREND statement (2004)Checklist for reporting non-randomised trials
• Transparent Reporting of Evaluations with Non-randomized Designs – for reporting comparative trials without randomisation
– the statement – Des Jarlais et al (2004)
• CONSORT + theoretical basis of intervention, tests of baseline equivalence and corresponding adjustments of main analyses
– http://www.trend-statement.org/
• download the statement
26 March 2008 Statistical presentation - 2. A symptomatic review 19
The STROBE statement (2007)Checklist/flow diagram for reporting observational studies
• STrengthening the Reporting of OBservational studies in Epidemiology– reporting of cohort studies, case-control studies and
cross-sectional studies (surveys)
– the statement – eg von Elm et al (2007)• CONSORT + sources of potential bias + outcomes
related to type of design + potential confounders and corresponding adjustments of main analyses
– explanation and elaboration – eg Vandenbroucke et al (2007)
– http://www.strobe-statement.org/• download both documents and templates of checklist
and flow diagram
26 March 2008 Statistical presentation - 2. A symptomatic review 20
Other guidelinesFor other kinds of studies
• Extensions to CONSORT for other kinds of randomised trials – cluster RCTs – Campbell et al (2004) [no relation!]
– non-inferiority/equivalence trials – Piaggio et al (2006)
– herbal medicinal interventions – Gagnier et al (2006)
– (coming) non-pharmaceutical interventions
– http://www.consort-statement.org/?o=1044
• download all documents