bayes for beginners reverend thomas bayes (1702-61) velia cardin marta garrido
TRANSCRIPT
Bayes for Beginners
Reverend Thomas Bayes (1702-61)
Velia Cardin
Marta Garrido
http://www.fil.ion.ucl.ac.uk/spm/software/spm2/
“In addition to WLS estimators and classical inference, SPM2 also supports Bayesian estimation and inference. In this instance the statistical parametric maps become posterior probability maps Posterior Probability Maps (PPMs), where the posterior probability is a probability of an effect given the data. There is no multiple comparison problem in Bayesian inference and the posterior probabilities do not require adjustment.”
Overview
VeliaBayes vs Frequentist approachAn exampleBayes theorem
Marta Bayesian InferencePosterior Probability Distribution Bayes in SPMSummary
vs
Bayesian statistics
Frequentist Statistics
With a frequentist approach….
The frequentist conclusion is restricted to the data at hand, it doesn’t take into account previous, valuable information.
In general, we want to relate an event (E) to a hypothesis (H) …and the probability of E given H
If the p-value is sufficiently small, you reject the null hypothesis, but…
it doesn’t say anything about the probability of Hi.
We obtain a p-value that is the probability, given a true H0, for the outcome to be more or equally extreme as the observed outcome.
Conclusions depends on previous evidence. Bayesian approach is not data analysis per se, it brings different types of evidence to answer the questions of importance.
In general, we want to relate an event (E) to a hypothesis (H)and the probability of E given H
Given a prior state of knowledge or belief, it tells how to update beliefs based upon observations (current data).
The probability of a H being true is determined.
With a Bayesian approach…
You can compare the probabilities of different H for a same E
A probability distribution of the parameter or hypothesis is obtained
Macaulay Culkin Busted for Drugs!
Our observations….
DREW BARRYMORE REVEALS ALCOHOL AND DRUG PROBLEMS STARTED AGED EIGHT
Feldman , arrested and charged with heroin possession
Corey Haim in a spiral of prescription drug abuse!
Dana Plato died of a drug overdose at age 34
Todd Bridges on suspicion of shooting and stabbing alleged drug dealer in a crack house. ...
We took a random sample of 40 people, 10 of them were young stars, being 3 of them addicted to drugs. From the other 30, just one.
Our hypothesis is: “Young actors have more probability of becoming drug-addicts”
Drug-addicted
young actors
1 29
10
30
3 7
4 36 40
control
YA+
YA-
D+ D-
With a frequentist approach we will test:
Hi:’Conditions A and B have different effects’ Young actors have a different probability of becoming drug addicts than the rest of the people
H0:’There is no difference in the effect of conditions A and B’
This is not what we want to know!!!
…and we have strong believes that young actors have more probability of becoming drug addicts!!!
The statistical test of choice is 2 and Yates’ correction:
2 = 3.33 p=0.07
We can’t reject the null hypothesis, and the information the p is giving us is basically that if we “do this experiment” many times, 7% of the times we will obtain this result if there is no difference between both conditions.
We want to know if
1 (0.025)29 (0.725)
10 (0.25)
30 (0.75)
3 (0.075)7 (0.175)
4 (0.1) 36 (0.9) 40 (1)
p(D+YA+)
p (D+ YA+) = p (D+ and YA+) / p (YA+)
p (D+YA+) p (YA+D+)
Reformulating
This is Bayes’ Theorem !!!
YA+
YA-
D-D+
total
total
p (YA+ D+) = p (D+ and YA+) / p (D+)
p (D+ YA+) = 0.075 / 0.25 = 0.3
p (YA+ D+) = 0.075 / 0.1 = 0.75
0.3 0.75
p (D+ and YA+) = p (YA+ D+) * p (D+)
p (D+ YA-) = p (D+ and YA-) / p (YA-)
p (D+YA+) > p (D+YA-)
p (D+ YA-) = 0.025 / 0.75 = 0.033
p (D+YA+) > p (D+YA-)
0.3 > 0.033
With a Bayesian approach…
p (D+YA-)
p (YA+D+)p (YA+ D+) * p (D+) p (YA+)
p (D+ YA+)
Substituting p (D+ and YA+) on
p (D+ YA+) = p (D+ and YA+) / p (YA+)
It relates the conditional density of a parameter (posterior probability) with its unconditional density (prior, since depends on information present before the experiment).
The likelihood is the probability of the data given the parameter and represents the data now available.
Bayes’ Theorem for a given parameter
p (data) = p (data) p () / p (data)
1/P (data) is basically a normalizing constant
Posterior likelihood x prior
The prior is the probability of the parameter and represents what was thought before seeing the data.
The posterior represents what is thought given both prior information and the data just seen.
In fMRI….
• Classical – ‘What is the
likelihood of getting these data given no activation occurred?’
• Bayesian (SPM2)– ‘What is the chance of
getting these parameters, given these data?
)(
)|()(
)(
),()|(
yp
ypp
yp
ypyp
)|( yp
)|( yp
What you know about the model after the data arrive, , is
what you knew before, , and what the data told you, .
)(p
In order to make probability statements about given y we begin with a model
)|()(),( yppyp
Bayesian Inference
joint prob. distribution
where
)|()()(
)|()()(
yppyp
yppyp
discrete case
continuous caseor
)|()()|( yppyp
likelihoodpriorposterior
Likelihood: p(y|) = (Md, d-1)
Prior: p() = (Mp, p-1)
Posterior: p(y) ∝ p(y|)*p() = (Mpost, post-1)
Mp
p-1
Mpost
post-1
d-1
Md
post = d + p Mpost = d Md + p Mp
post
Posterior Probability Distributionprecision = 1/2
The effects of different precisions
p = d
p < d
p > d
p ≈ 0
Multivariate Distributions
spatial normalization segmentation
and Bayesian inference in…
Posterior Probability Maps (PPM)Dynamic Causal Modelling (DCM)
SPM uses priors for estimation in…
Bayes in SPM
Shrinkage PriorsSmall, variable effect Large, variable effect
Small, consistent effect Large, consistent effect
Thresholding
p(| y) = 0.95
Summary
In Bayesian estimation we…
1. …start with the formulation of a model that we hope is adequate to describe the situation of interest.
2. …observe the data and when the information available changes it is necessary to update the degrees of belief (probability).
3. …evaluate the fit of the model. If necessary we compute predictive distributions for future observations.
priors over the parameters
posterior distributions
new priors over the parameters
Prejudices or scientific judgment?
The selection of a prior is subjective
and arbitrary.
It is reasonable to draw conclusions in the light of
some reason.
Bayesian methods use probability models for quantifying uncertainty in inferences based on statistical data analysis.
• http://www.stat.ucla.edu/history/essay.pdf (Bayes’ original essay!!!)• http://www.cs.toronto.edu/~radford/res-bayes-ex.html• http://www.gatsby.ucl.ac.uk/~zoubin/bayesian.html
• A. Gelman, J.B. Carlin, H.S. Stern and D.B. Rubin, 2nd ed. Bayesian Data Analysis. Chapman & Hall/CRC.
• Mackay D. Information Theory, Inference and Learning Algorithms. Chapter 37: Bayesian inference and sampling theory. Cambridge University Press, 2003.
• Berry D, Stangl D. Bayesian Methods in Health-Realated Research. In: Bayesian Biostatistics. Berry D and Stangl K (eds). Marcel Dekker Inc, 1996.
• Friston KJ, Penny W, Phillips C, Kiebel S, Hinton G, Ashburner J. Classical and Bayesian inference in neuroimaging: theory. Neuroimage. 2002 Jun;16(2):465-83.
References
Bayes for Beginners
Reverend Thomas Bayes (1702-61)
…good-bayes!!!
“We don’t see what we don’t seek.”
E. M. Forster