p values - part 4 the p value and ‘rules’ robin beaumont 10/03/2012 with much help from...
Post on 30-Dec-2015
219 Views
Preview:
TRANSCRIPT
P Values - part 4The P value and ‘rules’
Robin Beaumont10/03/2012
With much help from
Professor Geoff Cumming
Putting it all together
P Valuesampling
probability
statistic
Rules Alternatives
Summary so far• A P value is a conditional probability which considers a range of
outcomes = ‘area’ in a PDF graph .• The SEM formula allows us to: predict the accuracy of your
estimated mean across a infinite number of samples!• Taking another random sample will give us a different P value• How different? - Does not follow a normal distribution• Dance of the p values – Geoff Cumming • Depends upon if the null hypothesis is actually true in reality!
Remember we have assumed so far that the null hypothesis is true.
Review
Rules
t density: sx = 9.037 n =15
0
12096
-2.656t 2.656
Shaded area=0.0188
Original units:
0
Rule -> If our result has a P value more extreme than our level of acceptability = our critical valueReject the parameter value we based the p value on. = reject the null hypothesis
Given that the sample was obtained from a population with a mean (i.e. parameter value) of 120 a sample with a T(n=15) statistic of -2.656 or 2.656 or one more extreme with occur 1.8% of the time. This is less than one in twenty (i.e. P value < 0.05). Mark the one in twenty value = the critical value for T(n=15) = 2.144= alpha (α) level =0.05)
Therefore we dismiss the possibility that our sample came from a population with a mean of 120
Rules
Say one in twenty 1/20 =Or 1/100
Or 1/1000or . . . .
What value do we now give the parameter value ?
When p value is in the critical region Reject the parameter value we based the p value on is considered untenable = reject the null hypothesis
What is the mean of the population if we have now ruled out one value?
Set a level of acceptability = critical value (CV)
• Allows decision making • Moves thing forward• If the sample did not come from the ‘null distribution’ indicates there is some
effect• Population mean of 120 for substance X indicates they have a propensity to a
range of nasty diseases• Given them miracle drug Y which is believed to reduce the level has this have
any effect in our sample of 15?• If the probability of obtaining our t value (i.e. + associated p value) is below a
certain critical threshold we can say that our sample does not come from the null distribution and there is a effect.
Why do we want to reject the null distribution?
Fisher – only know and only consider the model we have i.e. The parameter we have used in our model – when we reject it we accept that any value but that one can replace it.
Neyman and Pearson + Gossling
Bayesians
Hnull = μ=120 versus alternative Halt = μ≠120 [μ = population mean]Hnull = T= 0 versus alternative Halt = T≠ 0 [T = t statistic]
Hnull = μ=120 versus alternative Halt = μ = XXX [μ = population mean]Hnull = T= 0 versus alternative Halt = T = xxx [T = t statistic]
Fisher – infinitely many alternative ‘red’ distributions
View 1 - The infinite variety of alternative distributions - Fisher
Alternative distributions:
Become flatter further away from null
When coincides with null= same shape
How do we define the alternative hypothesis?Distance measures: Effect size / Non centrality Parameter
Become more asymmetrical as further away from null
Null distribution t(df, ncp=0) Alternative distributions t(df, ncp)
View 2 - The single specified alternative – Neyman + Pearson•Take the distribution around the sample value •Then work our the difference SD’s:
Delta = Δ Non centrality parameter = capital = d x √15 = -.6857 x 3.872 = -2.6557
Population mean of 120 for substance X, SD= 35 units/100 ml. 15 random subjects take miracle drug have a mean = 96
Gpower shows it all!
Population mean of 120 for substance X, SD= 35 units/100 ml. 15 random subjects take miracle drug have a mean = 96
Red line = null distributionRed areas = critical regions = α alpha regionsBlue line = specified alternative distributionBlue shaded area = β beta regions (far right– very small)
α = the reject region
= 120= 96
Correct decisions
incorrect decisions
Two correct and 2 incorrect decisions
Correct decisions
Power = 1 - Beta
Power is good
Insufficient power – unlikely
to get a p value in the critical region
Too much power always p value in
critical region but possibly trivial effect
size
More Power the better up unto a certain point!
Cumming - 2008 Replication and P intervalsRed line = null distributionRed areas = critical regions = α alpha regionsBlue line = specified alternative distributionBlue shaded area = β beta regions (far right– very small)
ImplicationsPower and Cohens effect size measure d reflect one another. When power high p value distribution does provides a measure of evidence for the specific alternative hypothesis.Computer simulations - reality differentReplication is a vitally important research strategy – meta analysis.Power analysis during study design and Confidence intervals along with effect size measures when reporting results – alternative strategy?
Always specify specific alternative hypothesiswww.robin-beaumont.co.uk/virtualclassroom/contents.html
Students bloomers
• The p value did not indicate much statistic significance
• Given that the population comes from one population
• The p value is 0.003 thus rejecting the null hypothesis and there is a statistical significance
• Correlation = 0.25 (p<0.001) indicating that assuming that the data come from a bivariate normal distribution with a correlation of zero you would obtain a correlation of <0.000. There is 95% chance that the relationship among the variables is not due to chance
top related