cs221 probability theory
DESCRIPTION
TRANSCRIPT
Probability Theory Review
CS221: Introduction to Artificial IntelligenceNaran Bayanbat
10/14/2011
Slides used material from CME106 course reader and CS229 handouts
Topics
• Axioms of Probability• Product and chain rules • Bayes Theorem• Random variables• PDFs and CDFs• Expected value and variance
Introduction
• Sample space - set of all possible outcomes of a random experiment– Dice roll: {1, 2, 3, 4, 5, 6}– Coin toss: {Tails, Heads}
• Event space - subsets of elements in a sample space– Dice roll: {1, 2, 3} or {2, 4, 6}– Coin toss: {Tails}
Introduction
• Axioms of Probability– for all
–
Set operations
• Let and • and
• Properties:
• If then
Conditional Probability
• probability of A given B
A B
Conditional Probability
• probability of A given B
A BΩ
Conditional Probability
• A and B are independent if•
• A and B are conditionally independent given C if
Conditional Probability
• Joint probability:
• Product rule:•
• Chain rule of probability:
Conditional Probability
• probability of A given B• Example:– probability that school is closed – probability that it snows– probability of school closing if it snows– probability of snowing if school is closed
Conditional Probability
• probability of A given B• Example:– probability that school is closed – probability that it snows
P(A, B) 0.005
P(B) 0.02
P(A|B) 0.25
𝑃 ( 𝐴∨𝐵 )= 𝑃 ( 𝐴∩𝐵 )𝑃 (𝐵 )
Bayes Theorem
• We can relate P(A|B) and P(B|A) through Bayes’ rule:
Bayes Theorem
• We can relate P(A|B) and P(B|A) through Bayes’ rule:
Posterior Probability Likelihood
Normalizing Constant
Prior Probability
Bayes Theorem
• We can relate P(A|B) and P(B|A) through Bayes’ rule:
• can be eliminated with law of total probability:
=
Random Variables
• Random variable X is a function s.t.
• Examples:– Russian roulette: X = 1 if gun fires and X = 0 otherwise and – X = # of heads in 10 coin tosses
Do ya feel lucky, punk?
Cumulative Distribution Functions
• Defined as such that • Properties:
Probability Density Functions
• For discrete random variables, defined as:
• Relates to CDFs:
• =
Probability Density Functions
• For continuous random variables, defined as:
• Relates to CDFs:
Probability Density Functions
• Example:• Let X be the angular position of freely spinning pointer on
a circle
Probability Density Functions
• Example:• Let X be the angular position of freely spinning pointer on
a circle f(X)
1/2
2
X
Probability Density Functions
• Example:• Let X be the angular position of freely spinning pointer on
a circle f(X)
1/2
2
X
Probability Density Functions
• Example:• Let X be the angular position of freely spinning pointer on
a circle
• for
f(x)
1/2
2
x
F(x)
1
2
x
Probability Density Functions
• Example:• Let X be the angular position of freely spinning pointer on
a circle
• for • =
f(x)
1/2
2
x
F(x)
1
2
x
Expectation
• Given a discrete r.v. X and a function
• For a continuous r.v. X and a function
Expectation
• Expectation of a r.v. X is known as the first moment, or the mean value, of that variable ->
• Properties:• for any constant • for any constant • +
Variance
• For a random variable X, define:
• Another common form:
• Properties:• for any constant • for any constant
Gaussian Distributions
• Let , then
where represents the mean and the variance• Occurs naturally in many phenomena– Noise/error– Central limit theorem
Questions?