cpsc340 - computer science at ubc
TRANSCRIPT
![Page 1: CPSC340 - Computer Science at UBC](https://reader031.vdocuments.net/reader031/viewer/2022012915/61c5fb61433b9b29ea567c83/html5/thumbnails/1.jpg)
CPSC340
Nando de FreitasOctober, 2012University of British Columbia
Bayesian learning
![Page 2: CPSC340 - Computer Science at UBC](https://reader031.vdocuments.net/reader031/viewer/2022012915/61c5fb61433b9b29ea567c83/html5/thumbnails/2.jpg)
Maximum likelihood revision
![Page 3: CPSC340 - Computer Science at UBC](https://reader031.vdocuments.net/reader031/viewer/2022012915/61c5fb61433b9b29ea567c83/html5/thumbnails/3.jpg)
Maximum likelihood revision
![Page 4: CPSC340 - Computer Science at UBC](https://reader031.vdocuments.net/reader031/viewer/2022012915/61c5fb61433b9b29ea567c83/html5/thumbnails/4.jpg)
Outline of the lecture
This lecture introduces us to our second strategy for learning: Bayesian learning. The goal is for you to learn:
� Definition Beta prior.� How to use Bayes rule to go from prior beliefs and the likelihood of the data to posterior beliefs.likelihood of the data to posterior beliefs.
![Page 5: CPSC340 - Computer Science at UBC](https://reader031.vdocuments.net/reader031/viewer/2022012915/61c5fb61433b9b29ea567c83/html5/thumbnails/5.jpg)
Bayesian learning procedure
Step 1: Given n data, x1:n = {x1, x2,…, xn }, write down the expression for the likelihood:
p( x1:n |θ θ θ θ ) =
Step 2: Specify a prior:p(θ θ θ θ )
Step 3: Compute the posterior:
p(θ θ θ θ | x1:n ) p( x1:n |θ θ θ θ ) p(θ θ θ θ )
p(θ θ θ θ | x1:n ) p( x1:n |θ θ θ θ ) p(θ θ θ θ )
p( x1:n )=
ffff
![Page 6: CPSC340 - Computer Science at UBC](https://reader031.vdocuments.net/reader031/viewer/2022012915/61c5fb61433b9b29ea567c83/html5/thumbnails/6.jpg)
Bayesian learning procedure
Posterior: Compute the posterior:
p(θ θ θ θ | x1:n ) p( x1:n |θ θ θ θ ) p(θ θ θ θ )ffff
Marginal likelihood: p( x1:n ) =
![Page 7: CPSC340 - Computer Science at UBC](https://reader031.vdocuments.net/reader031/viewer/2022012915/61c5fb61433b9b29ea567c83/html5/thumbnails/7.jpg)
Bayesian learning for coin model
Step 1: Write down the likelihood of the data (i.i.d. Bernoulli in our case):
p( xi |θ θ θ θ ) =
p( x1:n |θ θ θ θ ) =
![Page 8: CPSC340 - Computer Science at UBC](https://reader031.vdocuments.net/reader031/viewer/2022012915/61c5fb61433b9b29ea567c83/html5/thumbnails/8.jpg)
Bayesian learning for coin model
Step 2: Specify a prior on θθθθ. For this, we need to introduce the Beta distribution.
![Page 9: CPSC340 - Computer Science at UBC](https://reader031.vdocuments.net/reader031/viewer/2022012915/61c5fb61433b9b29ea567c83/html5/thumbnails/9.jpg)
Bayesian learning for coin model
Step 2: Specify a prior on θθθθ. For this, we need to introduce the Beta distribution.
![Page 10: CPSC340 - Computer Science at UBC](https://reader031.vdocuments.net/reader031/viewer/2022012915/61c5fb61433b9b29ea567c83/html5/thumbnails/10.jpg)
Bayesian learning for coin model
Step 3: Compute the posterior:
p(θ θ θ θ | x1:n ) p( x1:n |θ θ θ θ ) p(θ θ θ θ )ffff
![Page 11: CPSC340 - Computer Science at UBC](https://reader031.vdocuments.net/reader031/viewer/2022012915/61c5fb61433b9b29ea567c83/html5/thumbnails/11.jpg)
Example
Suppose we observe the data, x1:6 = {1, 1, 1, 1, 1, 1}, where each xi
comes from the same Bernoulli distribution (i.e. it is independent and identically distributed (iid)). What is a good guess of θθθθ?
We can compute the posterior and use its mean as the estimate.
Using a prior Beta(2,2):
Using a prior Beta(1,0.01):
![Page 12: CPSC340 - Computer Science at UBC](https://reader031.vdocuments.net/reader031/viewer/2022012915/61c5fb61433b9b29ea567c83/html5/thumbnails/12.jpg)
Next lecture
In the next lecture, we apply our learning strategies to Bayesian networks.