introduction to probability theory and graphical models
Post on 15-Feb-2016
55 Views
Preview:
DESCRIPTION
TRANSCRIPT
Introduction to probability theory and graphical models
Translational Neuroimaging Seminar on Bayesian InferenceSpring 2013
Jakob HeinzleTranslational Neuromodeling Unit (TNU) Institute for Biomedical Engineering (IBT)University and ETH Zürich
Bayesian Inference - Introduction to probability theory
Literature and References
• Literature:• Bishop (Chapters 1.2, 1.3, 8.1, 8.2)• MacKay (Chapter 2)• Barber (Chapters 1, 2, 3, 4)
• Many images in this lecture are taken from the above references.
2
Bayesian Inference - Introduction to probability theory
Probability distribution
3
A probability P(x=true) is defined on a sample space (domain) and defines (for every possible) event in the sample space the certainty
of it to occur.
Sample space: dom(X)={0,1},
Probabilities sum to one.
Bishop, Fig. 1.11
Bayesian Inference - Introduction to probability theory
Probability theory: Basic rules
4
.
Sum rule* - P(X) is also called the marginal distribution
Product rule -
* According to Bishop
Bayesian Inference - Introduction to probability theory
Conditional and marginal probability
5
Bayesian Inference - Introduction to probability theory
Conditional and marginal probability
6
Bishop, Fig. 1.11
Bayesian Inference - Introduction to probability theory
Independent variables
7
Question for later: What does this mean for Bayes?
Bayesian Inference - Introduction to probability theory
Probability theory: Bayes’ theorem
8
is derived from the product rule
Bayesian Inference - Introduction to probability theory
Rephrasing and naming of Bayes’ rule
9
MacKay
D: data, q: parameters, H: hypothesis we put into the model.
Bayesian Inference - Introduction to probability theory
Example: Bishop Fig. 1.9
10
Box (B): blue (b) or red (r)Fruit (F): apple (a) or orange (o)
p(B=r) = 0.4, p(B=b) = 0.6.
What is the probability of having a red box if one has drawn an orange?
Bishop, Fig. 1.9
Bayesian Inference - Introduction to probability theory
Probability density
11
Bayesian Inference - Introduction to probability theory 12
PDF and CDF
Bishop, Fig. 1.12
Bayesian Inference - Introduction to probability theory
Cumulative distribution
13
𝑃 (𝑧)=∫−∞
𝑧
𝑝 (𝑥 )𝑑𝑥
Short example: How to use the cumulative distribution to transform a uniform distribution!
Bayesian Inference - Introduction to probability theory
Marginal densities
14
p
Integration instead of summing
Bayesian Inference - Introduction to probability theory
Two views on probability
15
● Probability can …– … describe the frequency of outcomes in random
experiments classical interpretation.
– … describe the degree of belief about a particular event Bayesian viewpoint or subjective interpretation of probability.
MacKay, Chapter 2
Bayesian Inference - Introduction to probability theory
Expectation of a function
16
Or
Bayesian Inference - Introduction to probability theory
Graphical models
17
1. They provide a simple way to visualize the structure of a probabilistic model and can be used to design and motivate new models.
2. Insights into the properties of the model, including conditional independence properties, can be obtained by inspection of the graph.
3. Complex computations, required to perform inference and learning in sophisticated models, can be expressed in terms of graphical manipulations, in which underlying mathematical expressions are carried along implicitly.
Bishop, Chap. 8
Bayesian Inference - Introduction to probability theory 18
Graphical models overview
Directed Graph
For summary of definitions see Barber, Chapter 2
Undirected Graph
Names: nodes (vertices), edges (links), paths, cycles, loops, neighbours
Bayesian Inference - Introduction to probability theory 19
Graphical models overview
Barber, Introduction
Bayesian Inference - Introduction to probability theory
Graphical models
20
Bishop, Fig. 8.1
Bayesian Inference - Introduction to probability theory
Graphical models: parents and children
21
Node a is a parent of node b, node b is a child of node a.
Bishop, Fig. 8.1
Bayesian Inference - Introduction to probability theory
Belief networks = Bayesian belief networks = Bayesian Networks
22
Bishop, Fig. 8.2
In general:
Every probability distributioncan be expressed as a Directed acyclic graph (DAG)
Important: No directed cycles!
Bayesian Inference - Introduction to probability theory
Conditional independence
23
A variable a is conditionally independent of b given c, if
In bayesian networks conditional independence can betested by following some simple rules
Bayesian Inference - Introduction to probability theory
Conditional independence – tail-to-tail path
24
Is a independent of b?
No! Yes!Bishop, Chapter 8.2
Bayesian Inference - Introduction to probability theory
Conditional independence – head-to-tail path
25
No! Yes!Bishop, Chapter 8.2
Is a independent of b?
Bayesian Inference - Introduction to probability theory
Conditional independence – head-to-head path
26
Yes! No!Bishop, Chapter 8.2
Is a independent of b?
Bayesian Inference - Introduction to probability theory
Conditional independence – notation
27
Bishop, Chapter 8.2
Bayesian Inference - Introduction to probability theory
Conditional independence – three basic structures
28
Bishop, Chapter 8.2.2
Bayesian Inference - Introduction to probability theory
More conventions in graphical notations
29
Bishop, Chapter 8
= =
Regression model Short form Parameters explicit
Bayesian Inference - Introduction to probability theory
More conventions in graphical notations
30
Bishop, Chapter 8
Trained on data tn
Complete model usedfor prediction
Bayesian Inference - Introduction to probability theory
Summary – things to remember
• Probabilities and how to compute with the Product rule, Bayes’ Rule, Sum rule
• Probability densities PDF, CDF
• Conditional and Marginal distributions
• Basic concepts of graphical models Directed vs. Undirected, nodes and edges, parents and children.
• Conditional independence in graphs and how to check it.
31
Bishop, Chapter 8.2.2
top related