statistics 1b interludes most studies crave ‘significance’ · posted on aoril 21 2013 126...
TRANSCRIPT
1
Statistics 1B Interludes
13. ‘Not statistically significant’
Most studies crave ‘significance’
XKCD, Jan 2015
Collects real phrases from academic papers
Today’s story!• Measured insecticide metabolites in urine of 571
pregnant women• 6 years later measured metabolites in 287 children• Correlated with behavioural problems• 5 metabolites at 3 levels, mothers/children, 3 outcome
scales• = 60 95% confidence intervals for associations
(adjusted with logistic regression)
• Only one excluded 1.
2
30 more sensitive tests? Children shown below• Paper and press release only reported
the few significant results [2 positive and 1 negative]
• A green jelly-bean example?
• And maybe children with behaviouralproblems get more head lice? [reverse causation]
But sometimes ‘non-significance’ is of interest …
February 2015
But did it really show no benefit?
3
Conclusions
• Point estimates for all consumption levels show protection
• Confidence intervals are wide as few deaths in the baseline (never-drinker) category
• Wide CIs include plausible protective effects• But authors essentially interpret ‘not
significantly different’ as ‘no effect’• A serious misuse of statistics