basic probability (most slides borrowed, with permission, from andrew moore of cmu and google)...
TRANSCRIPT
![Page 1: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/1.jpg)
Basic Probability
(most slides borrowed, with permission, from Andrew Moore of CMU and Google)http://www.cs.cmu.edu/~awm/tutorials
![Page 2: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/2.jpg)
![Page 3: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/3.jpg)
![Page 4: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/4.jpg)
![Page 5: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/5.jpg)
![Page 6: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/6.jpg)
![Page 7: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/7.jpg)
![Page 8: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/8.jpg)
A
![Page 9: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/9.jpg)
A
B
![Page 10: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/10.jpg)
Notation Digression
• P(A) is shorthand for P(A=true)• P(~A) is shorthand for P(A=false)• Same notation applies to other binary RVs:
P(Gender=M), P(Gender=F)• Same notation applies to multivalued RVs:
P(Major=history), P(Age=19), P(Q=c)• Note: upper case letters/names for variables,
lower case letters/names for values• For multivalued RVs, P(Q) is shorthand for
P(Q=q) for some unknown q
![Page 11: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/11.jpg)
…
![Page 12: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/12.jpg)
…
k
k…
![Page 13: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/13.jpg)
k
k
![Page 14: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/14.jpg)
k
k
Integration over nuisance variables
![Page 15: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/15.jpg)
![Page 16: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/16.jpg)
![Page 17: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/17.jpg)
![Page 18: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/18.jpg)
![Page 19: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/19.jpg)
![Page 20: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/20.jpg)
R
QS
P(H|F) = R/(Q+R)
P(F|H) = R/(S+R)
![Page 21: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/21.jpg)
![Page 22: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/22.jpg)
![Page 23: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/23.jpg)
![Page 24: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/24.jpg)
![Page 25: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/25.jpg)
Let’s-Make-A-Deal Problem
• Game show with 3 doors• Behind one door is a prize• Player chooses a door, say door 1• Host opens door 2 to reveal no prize• Host offers player opportunity to switch from
door 1 to door 3.• Should player stay with door 1, switch to door
3, or does it not matter?
![Page 26: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/26.jpg)
Let’s-Make-A-Deal Problem• D = door containing prize
C = contestant’s choiceH = door revealed by host
• P(D = d | C, H)• P(D = d | C, H) ~ P(H | D=d, C) P(D = d|C)• P(D = d | C, H) ~ P(H | D=d, C) P(D = d)• E.g., C=1, H=2:
• P(H=2|C=1,D=1) = .5• P(H=2|C=1,D=2) = 0 • P(H=2|C=1,D=3) = 1
![Page 27: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/27.jpg)
Let’s-Make-A-Deal Problem: Lessons
• Likelihood function is based on a generative model of the environment (host’s behavior)
• Griffiths and Tenenbaum made their own generative assumptions– About how the experimenter would pick an age
ttotal
• Solving a problem involves modeling the generative environment– Equivalent to learning the joint probability
distribution
![Page 28: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/28.jpg)
![Page 29: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/29.jpg)
![Page 30: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/30.jpg)
![Page 31: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/31.jpg)
![Page 32: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/32.jpg)
![Page 33: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/33.jpg)
![Page 34: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/34.jpg)
![Page 35: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/35.jpg)
![Page 36: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/36.jpg)
![Page 37: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/37.jpg)
If you have joint distribution, you can perform anyinference in the domain.
![Page 38: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/38.jpg)
![Page 39: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/39.jpg)
![Page 40: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/40.jpg)
![Page 41: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/41.jpg)
![Page 42: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/42.jpg)
e.g., model of the environment
![Page 43: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/43.jpg)
![Page 44: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/44.jpg)
![Page 45: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/45.jpg)
![Page 46: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/46.jpg)
![Page 47: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/47.jpg)
![Page 48: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/48.jpg)
![Page 49: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/49.jpg)
![Page 50: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/50.jpg)
![Page 51: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/51.jpg)
![Page 52: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/52.jpg)
![Page 53: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/53.jpg)
![Page 54: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/54.jpg)
![Page 55: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/55.jpg)
![Page 56: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/56.jpg)
![Page 57: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/57.jpg)
![Page 58: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/58.jpg)
![Page 59: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/59.jpg)
![Page 60: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/60.jpg)
![Page 61: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/61.jpg)
![Page 62: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/62.jpg)
Naïve Bayes Model
![Page 63: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/63.jpg)
![Page 64: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/64.jpg)
![Page 65: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/65.jpg)
![Page 66: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/66.jpg)
Bayes Classifier
• Assume you want to predict some output Y which has arity n and values y1, y2, y3, … yn.
• Assume there are m input attributes called X1, X2, X3, … Xm.
• For each of n values yi, build a density estimator, Di, that estimatesP(X1, X2, X3, … Xm |Y=yi)
• Ypredict = argmaxy P(Y=y| X1, X2, X3, … Xm ) = argmaxy P(X1, X2, X3, … Xm | Y=y) P(Y=y)
![Page 67: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/67.jpg)
![Page 68: Basic Probability (most slides borrowed, with permission, from Andrew Moore of CMU and Google) awm/tutorials](https://reader030.vdocuments.net/reader030/viewer/2022032705/56649d945503460f94a7c188/html5/thumbnails/68.jpg)
Machine Learning Vs. Cognitive Science
• For machine learning, density estimation is required to do classification, prediction, etc.
• For cognitive science, density estimation under a particular model is a theory about what is going on in the brain when an individual learns.