publication bias & negative results
TRANSCRIPT
![Page 1: PUBLICATION BIAS & NEGATIVE RESULTS](https://reader033.vdocuments.net/reader033/viewer/2022051520/58eccdbe1a28abf77b8b4633/html5/thumbnails/1.jpg)
NEGATIVE RESULTS &PUBLICATION BIAS
![Page 2: PUBLICATION BIAS & NEGATIVE RESULTS](https://reader033.vdocuments.net/reader033/viewer/2022051520/58eccdbe1a28abf77b8b4633/html5/thumbnails/2.jpg)
PRISONER’S DILEMMA
SO WHO PUBLISHESNEGATIVE RESULTS
????
![Page 3: PUBLICATION BIAS & NEGATIVE RESULTS](https://reader033.vdocuments.net/reader033/viewer/2022051520/58eccdbe1a28abf77b8b4633/html5/thumbnails/3.jpg)
TITLE• Negativity towards negative results: a
discussion of the disconnect between
scientific worth and scientific culture.
![Page 4: PUBLICATION BIAS & NEGATIVE RESULTS](https://reader033.vdocuments.net/reader033/viewer/2022051520/58eccdbe1a28abf77b8b4633/html5/thumbnails/4.jpg)
AUTHORS
1 Faculty of Science, Medicine and Health, Illawarra Health and Medical research institute, University of Wollongong, NSW 522, Australia.
2Schizophrenia Research Institute, NSW 2010, Australia.
Natalie Matosin1,2,*, Elisabeth Frank1,2, Martin Engel1,2, Jeremy S.
Lum1,2, and Kelly A1,2.
![Page 5: PUBLICATION BIAS & NEGATIVE RESULTS](https://reader033.vdocuments.net/reader033/viewer/2022051520/58eccdbe1a28abf77b8b4633/html5/thumbnails/5.jpg)
JOURNAL :
VOLUME : 7
ISSUE : 2014
PAGES : 171 – 173
IMPACT FACTOR : 4.316
![Page 6: PUBLICATION BIAS & NEGATIVE RESULTS](https://reader033.vdocuments.net/reader033/viewer/2022051520/58eccdbe1a28abf77b8b4633/html5/thumbnails/6.jpg)
“What gets us into trouble is not what we don’t know, it’s what we know for sure that
just ain’t so.” – Mark Twain.
![Page 7: PUBLICATION BIAS & NEGATIVE RESULTS](https://reader033.vdocuments.net/reader033/viewer/2022051520/58eccdbe1a28abf77b8b4633/html5/thumbnails/7.jpg)
SCIENCE – Flawless system of knowledge.
REALITY – This is not always the case.
BACKGROUND
![Page 8: PUBLICATION BIAS & NEGATIVE RESULTS](https://reader033.vdocuments.net/reader033/viewer/2022051520/58eccdbe1a28abf77b8b4633/html5/thumbnails/8.jpg)
NEGATIVE / NEWBut what happens when you obtain results that support the null hypothesis, or do not fit with the current scientific thinking???
POSITIVEDissemination of results are straightforward when the findings are positive.
![Page 9: PUBLICATION BIAS & NEGATIVE RESULTS](https://reader033.vdocuments.net/reader033/viewer/2022051520/58eccdbe1a28abf77b8b4633/html5/thumbnails/9.jpg)
Issues surrounding Publication bias and difficulty in
communicating the negative results.
THIS ARTICLE IS ABOUT
![Page 10: PUBLICATION BIAS & NEGATIVE RESULTS](https://reader033.vdocuments.net/reader033/viewer/2022051520/58eccdbe1a28abf77b8b4633/html5/thumbnails/10.jpg)
PRESSUE ON SCIENTISTS
HIGH-IMPACT STUDIES.
ILLOGICAL or IMPRACTICAL
SEARCH OF SIGNIFICANCE
FOCUS ON POSITIVE
FILE DRAWER EFFECT
(SCARGLE 1999)
![Page 11: PUBLICATION BIAS & NEGATIVE RESULTS](https://reader033.vdocuments.net/reader033/viewer/2022051520/58eccdbe1a28abf77b8b4633/html5/thumbnails/11.jpg)
Fanelli - 2010,2012• “Papers are less likely to be
published and to be cited if they report ‘negative’ results”
(Fanelli, 2010)
• Consequently, the amount of non-significant data reported is progressively declining
(Fanelli, 2012)
![Page 12: PUBLICATION BIAS & NEGATIVE RESULTS](https://reader033.vdocuments.net/reader033/viewer/2022051520/58eccdbe1a28abf77b8b4633/html5/thumbnails/12.jpg)
REASON
Selectiveness of ‘high impact’ journals
Have a bold statement in the
submission form:Negative results are not
accepted.
![Page 13: PUBLICATION BIAS & NEGATIVE RESULTS](https://reader033.vdocuments.net/reader033/viewer/2022051520/58eccdbe1a28abf77b8b4633/html5/thumbnails/13.jpg)
Positive Results = High Impact Results
Why negative result is not considered equally important??
![Page 14: PUBLICATION BIAS & NEGATIVE RESULTS](https://reader033.vdocuments.net/reader033/viewer/2022051520/58eccdbe1a28abf77b8b4633/html5/thumbnails/14.jpg)
Science should tell all sides of the story.
Scientific principles are always under reconsideration
New evidence will refute old evidence and current scientific thinking
BUT THIS IS DIFFICULT TO ACHIEVE
![Page 15: PUBLICATION BIAS & NEGATIVE RESULTS](https://reader033.vdocuments.net/reader033/viewer/2022051520/58eccdbe1a28abf77b8b4633/html5/thumbnails/15.jpg)
ANDREW WAKEFIELD MMR CONTROVERSYUK based researcher Dr. Andrew Wakefield.
Child vaccination (specifically the MMR vaccine) causes
Incidence Of Autism (Wakefield et al., 1998).
Led to Panic & decade long decrease in child immunisation
![Page 16: PUBLICATION BIAS & NEGATIVE RESULTS](https://reader033.vdocuments.net/reader033/viewer/2022051520/58eccdbe1a28abf77b8b4633/html5/thumbnails/16.jpg)
13 studies with convincing negative results published 1998 - 2010
Support against Wakefield’s claims failed to gain the same level of attention as the original study
article retraction in 2010
![Page 17: PUBLICATION BIAS & NEGATIVE RESULTS](https://reader033.vdocuments.net/reader033/viewer/2022051520/58eccdbe1a28abf77b8b4633/html5/thumbnails/17.jpg)
TRANSITIONING BETWEEN PARADIGMSOnce the body of evidence for the competing paradigm overtakes the evidence in support of the dominant paradigm, then scientists will easily switch allegiance (Kuhn, 1970)Humans have an inbuilt need to support the status quo, and therefore have an innate difficulty in overriding pre-existing beliefs (Jost and Hunyady, 2003)
![Page 18: PUBLICATION BIAS & NEGATIVE RESULTS](https://reader033.vdocuments.net/reader033/viewer/2022051520/58eccdbe1a28abf77b8b4633/html5/thumbnails/18.jpg)
CORRECTING THE LITERATURE: AN UPHILL BATTLE
Extreme difficulties in correcting the literature have been experienced by many, but discussed openly by few.
![Page 19: PUBLICATION BIAS & NEGATIVE RESULTS](https://reader033.vdocuments.net/reader033/viewer/2022051520/58eccdbe1a28abf77b8b4633/html5/thumbnails/19.jpg)
Dr Peter Wilmshurst
“Amrinone increased heart contractility in a clinical trial” (Benotti et al., 1978)
“Although amrinone increased the strength of contraction of normal heart muscle, it did not affect contractility in patients with heart failure” (Wilmshurst, 2003)
Wilmshurst spent two decades to disseminating his negative findings
and remained unsuccessful.
![Page 20: PUBLICATION BIAS & NEGATIVE RESULTS](https://reader033.vdocuments.net/reader033/viewer/2022051520/58eccdbe1a28abf77b8b4633/html5/thumbnails/20.jpg)
DR.DAVID VAUXPeer-reviewed “a mechanism to overcome the rejection of transplanted tissue ’’ in Nature
(Bellgrau et al., 1995)News and Views’ piece, published in the same
issue of Nature
(Vaux, 1995)He started conducting experiments in his own laboratory.
But failed to produce results
![Page 21: PUBLICATION BIAS & NEGATIVE RESULTS](https://reader033.vdocuments.net/reader033/viewer/2022051520/58eccdbe1a28abf77b8b4633/html5/thumbnails/21.jpg)
He tried submitting his new findings and tried to refute his earlier article but ultimately
he failed.
“Little did we know that instead of providing an answer to transplant rejection, these experiments would teach us a great deal about editorial practices and the difficulty of correcting errors once they appear in the literature” (Vaux, 2013)
After 2 years he retracted his article.
![Page 22: PUBLICATION BIAS & NEGATIVE RESULTS](https://reader033.vdocuments.net/reader033/viewer/2022051520/58eccdbe1a28abf77b8b4633/html5/thumbnails/22.jpg)
Because the results were unpublished
Research groups had continued to follow the same lines of thought and the same paths of investigation, only to all fail in the same way, ultimately wasting time and resources.
Both unsuccessfully attempted to alter depression-like behaviour in the CD1 mouse strain with a variety of classical antipsychotics.
Both failed
![Page 23: PUBLICATION BIAS & NEGATIVE RESULTS](https://reader033.vdocuments.net/reader033/viewer/2022051520/58eccdbe1a28abf77b8b4633/html5/thumbnails/23.jpg)
SIGNIFICANCE BIAS
Statistically significant results have been shown to be three times more likely to be published than papers with null results. K.Dickerson et al,1987
MISUSED AND MISINTERPRETED
DOES NOT SUPPORT HYPOTHESIS
ITS ONLY A TOOL TO REJECT NULL HYPOTHESIS
![Page 24: PUBLICATION BIAS & NEGATIVE RESULTS](https://reader033.vdocuments.net/reader033/viewer/2022051520/58eccdbe1a28abf77b8b4633/html5/thumbnails/24.jpg)
BIAS IN META - ANALYSISFUNNEL PLOTS
Exploring Publication Bias
SYMMETRICAL – NORMAL DISTRIBUTION ASYMMETRICAL
Gaping hole where negative studies should be
![Page 25: PUBLICATION BIAS & NEGATIVE RESULTS](https://reader033.vdocuments.net/reader033/viewer/2022051520/58eccdbe1a28abf77b8b4633/html5/thumbnails/25.jpg)
WHY NEGATIVE RESULTS ARE NOT PURSUED???
• Time and effort to construct the paper.
• Surviving the peer review is not out weighed
by the benefits.• Not considered high impact
knowledge.• Will not result in a highly cited paper.
![Page 26: PUBLICATION BIAS & NEGATIVE RESULTS](https://reader033.vdocuments.net/reader033/viewer/2022051520/58eccdbe1a28abf77b8b4633/html5/thumbnails/26.jpg)
How are we going to reverse the anti-negative-finding culture?
“Perhaps a lab should have to correct for the total number of published results in a given year”
Researchers should be obligated to retract their previous works throughout the progression of their career as they “…[find] that [their] previous tests in old papers are no longer significant in light of their success and, ironically, [their] contribution to the field”
(Lieberman and Cunningham, 2009)
![Page 27: PUBLICATION BIAS & NEGATIVE RESULTS](https://reader033.vdocuments.net/reader033/viewer/2022051520/58eccdbe1a28abf77b8b4633/html5/thumbnails/27.jpg)
NEGATING THE NEGATIVITYRevolt against publication bias.
![Page 28: PUBLICATION BIAS & NEGATIVE RESULTS](https://reader033.vdocuments.net/reader033/viewer/2022051520/58eccdbe1a28abf77b8b4633/html5/thumbnails/28.jpg)
Publication bias is a common theme and it still remains an issue.
“…negative findings are still a low priority for publication, so we need to find ways to make publishing them more attractive” (O’Hara, 2011)
![Page 29: PUBLICATION BIAS & NEGATIVE RESULTS](https://reader033.vdocuments.net/reader033/viewer/2022051520/58eccdbe1a28abf77b8b4633/html5/thumbnails/29.jpg)
Direction of scientific research Not determined by the pressure to win
the ‘significance lottery’
But Systematic, hypothesis-driven
attempts to fill holes in our knowledge
![Page 30: PUBLICATION BIAS & NEGATIVE RESULTS](https://reader033.vdocuments.net/reader033/viewer/2022051520/58eccdbe1a28abf77b8b4633/html5/thumbnails/30.jpg)
It Is Our Duty As Scientists To
(1)Publish all data, no matter what the outcome, because a negative finding is still an important finding.
(2)Have a hypothesis to explain the finding.
![Page 31: PUBLICATION BIAS & NEGATIVE RESULTS](https://reader033.vdocuments.net/reader033/viewer/2022051520/58eccdbe1a28abf77b8b4633/html5/thumbnails/31.jpg)
CONCLUSIONIf the experiment- Planned properly- The data has not been manipulated- Or pulled out of context - Compiled evidence of a negative
result, thenIt should provide an explanation as to
why we are seeing what we are seeing???
![Page 32: PUBLICATION BIAS & NEGATIVE RESULTS](https://reader033.vdocuments.net/reader033/viewer/2022051520/58eccdbe1a28abf77b8b4633/html5/thumbnails/32.jpg)
Only by truly rethinking the current scientific culture,
Will negative results be esteemed for their entire value.
Only then can we work towards an improved scientific paradigm.
![Page 33: PUBLICATION BIAS & NEGATIVE RESULTS](https://reader033.vdocuments.net/reader033/viewer/2022051520/58eccdbe1a28abf77b8b4633/html5/thumbnails/33.jpg)
Having negative results isn’t bad, what’s bad is failing to report them.
![Page 34: PUBLICATION BIAS & NEGATIVE RESULTS](https://reader033.vdocuments.net/reader033/viewer/2022051520/58eccdbe1a28abf77b8b4633/html5/thumbnails/34.jpg)
REFERENCES1) Egger M, Davey Smith G, Schneider M,
Minder C. Bias in meta-analysis detected by a simple, graphical test. BMJ : British Medical Journal. 1997;315(7109):629-634.
2) Dickersin, Kay, et al. "Publication bias and clinical trials." Controlled clinical trials 8.4 (1987): 343-353.