probabilistic graphical models 輪読会 chapter5
TRANSCRIPT
-
Probabilistic Graphical Models
Chapter 5 Local Probabilistic Models
-
CPDtbd
-
5.1 Tabular CPDsCPD CPTs V al(Pa ) V al(X)
2X252 = 32
A B Px|A,B
1 1 0.2
1 0 0.1
0 1 0.3
0 0 0.4
x5
-
5.2 Deterministic CPDs
5.2.1 Representation
X CPDf : V al(Pa ) V al(X)CPD
P (xPa ) =
f[2] or / and[] X = Y + Z
X
X {10
x = f(Pa )Xotherwise
-
5.2.2 Independencies
Example 5.3
2CAB
ABC(D EA,B)
CABdseparation
dseparation
-
5.2.2 Independencies
(X Y Z) deterministic CPDPaX (Z ) X Z dseparation
i+
i+
-
5.2.2 Independencies
5.1GD,X,Y ,ZZXY deterministically separated P I (G) PP (XPa )X DP (X Y Z)
exercise 5.1
l
X
-
5.2.2 Independencies
5.2GD,X,Y ,ZDETSEP(G,D,X,Y ,Z)P I (G) PP (XPa )X DP (X Y Z)
DETSEP
l
X
-
5.2.2 Independencies
Example 5.4
CBAXOR
ACB(D EB,C)
-
5.2.2 Independencies
Revisit Example 5.3 (Example 5.5)
CABOR
A = a CORB
P (DB, a ) = P (Da )BDA = a CP (XY ,Z) = P (XZ)(X Y Z)
1
1 1
0
-
5.2.2 Independencies
Definition 5.1
X,Y ,ZCX Y Zcc V al(C)
P (XY ,Z, c) = P (XZ, c)whereneverP (Y ,Z, c) > 0
Z(X Y Z, c)cXY contextually independent
contextspecific independencies CSI
c
-
5.2.2 Independencies
Revisit Example 5.3 with CSI (Example 5.6)
CABOR
A = a BC = c (C Ba )(D Ba )
C = c A = a B = b (A Bc )(D Ec )
C = c B = b A = a (D Eb , c )
1 1
c1
c1
0 0 0
c0
c0
1 0 1
c0 1
-
5.3 Context-Specific CPDs
5.3.1 Representation
Example 5.7
Job ... Apply ... SATLetterLetterSAT,SATLetterP (J a , s , l ) = P (J a , s , l )1 1 1 1 1 0
-
5.3.1.1 Tree-CPDs
Example 5.7 with Tree-CPD (Example 5.8)
J""
ex. P (J a , s , l )J : P (j ) = 0.1, andP (j ) = 0.9
1 1 0
0 1
-
5.3.1.1 Tree-CPDs
Definition 5.2
1 rooted treettP (X)
tZ Pa Ztarc, Z = z forz V al(Z)t2
X
i i
-
5.3.1.1 Tree-CPDs
Example 5.9
SAT84treeCPD
0
1 1
-
5.3.1.1 Tree-CPDs
Revisit Example 3.7 (Example 5.10)
JobChoiceLetter1Letter2CL L 1 2
-
5.3.1.1 Tree-CPDs
Revisit Example 3.7 (Example 5.10)
CPD multiplexer CPD
Definition 5.3
V al(A) = {1, ..., k}P (Y a,Z , ...,Z ) = 1 {Y = Z } CPD P (Y A,Z , ...,Z ) multiplexer CPD
multiplexer CPDex.10Choice
1 k a
1 k
-
5.3.1.1 Tree-CPDs
Revisit Example 3.7 (Example 5.10)
CL ,L LJL
1 2
-
5.3.1.1 Tree-CPDs
CPDCPD
-
5.3.1.2 Rule CPDs
Definition 5.4 rule
c; pcCp [0, 1]CScope[]
-
5.3.1.1 Tree-CPDs
Revisit Example 5.7 (Example 5.11)
CPDP (XPa )CPD
X
-
5.3.1.2 Rule CPDs
Definition 5.5 rulebased CPD
rulebased CPD P (XPa )R
RScope[] {X} Pa
{X} Pa (x,u)c(x,u)c; p RP (XU) P (xu) = 1CPD
Example 5.11
X
X
X
x
-
5.3.1.2 Rule CPDs
Example 5.12
XPa = {A,B,C}XCPD
CPD1
X
-
5.3.1.2 Rule CPDs
Proposition 5.1
BBCPD P (XPa )R R R X,P () = p
X
X
XX+
X+
c;pR,c
-
5.3.1.2 Rule CPDs
Revisit Example 5.12 with tree-CPD (Example 5.13)
Example 5.12
-
5.3.1.3 Other Representatioins
decision diagram
-
5.3 Context-Specific CPDs
5.3.2 Independencies
"a SL"
contextspecificCPDCPDcontextspecific CPD
0
-
5.3.2 Independencies
Revisit Example 5.7 with independence (Example 5.14)
a SATLetter(J S,La )
a , s Letter(J La , s )
0
c0
1 1
c1 1
-
5.3.2 Independencies
ctreeCPDX(Pa Scope[c])
Pa = {A,S,L}Scope[c] = {A}, (J S,La )
Scope[c] = {A,S}, (J La , s )
X
J
c0
c1 1
-
5.3.2 Independencies
Revisit Example 5.10 with independence (Example 5.15)
JobLetterLetter
ex. (J L c )c 2 1
-
5.3.2 Independencies
Revisit Example 5.14 with rule (Example 5.16)
s s a
(J Ls )
XYXcY
1
1
c1
-
5.3.2 Independencies
Deifinition 5.6 reduced rule
= c ; pC = ccc cc = c Scope[c ] Scope[c]Scope[c ] Scope[c]creduced rule [c] = c ; p R reduced rule R[c] = {[c] : R, c}
-
5.3.2 Independencies
Revisit Example 5.12 with reduced rule (Example 5.17)
Example 5.12R[a ]
a a
1
1 1
-
5.3.2 Independencies
Proposition 5.2 reduced rule
RXrulebased CPDR RcY Pa Y Scope[c] = X R[c]Y Scope[] = (X Y Pa Y , c)
exercise 5.4
""CSIY reduced rule exercise 5.7
c
X
c X
-
5.3.2 Independencies
Revisit Example 5.7 with CSI-sep (Example 5.18)
IntelligenceIntteligence
-
5.3.2 Independencies
A = a S J ,L J dseparation
0
-
5.3.2 Independencies
Deifinition 5.7 spurious edge
P (XPa )CPDY Pa cP (XPa )(X Y Pa {Y }, c )Y Xc spurious c = cPa Pa c
CPDreduced rule spurious Y reduced ruleR[c]cY X spurious
X X
X c X
X X
-
5.3.2 Independencies
CSIseparation
CSIseparation CSIdseparation
dseparation
-
5.3.2 Independencies
Revisit Example 5.7 with CSI-sep (Example 5.18)
S J ,L J spurious aaJIJDdseparatedCSISEPa JIa dseparatedba , s
0 0
1 1
-
5.3.2 Independencies
CSIseparationcontextspecific
Theorem 5.3
G PP L (G)cX,Y ,ZXcZYCSIseparatedP (X T Z, c)
exercise 5.8
l
c
-
5.3.2 Independencies
Revisit Example 5.10 with CSI-sep (Example 5.19)
C = c L JspuriousJL L (L L J , c )C = c (L L J ,C)CSISEPspurious
12
1 2
1 c 21
21 c 2
-
5.4 Independence of Causal Influence
noisyor model
generalized linear model
-
5.4.1 The Noisy-Or Model
Letter
Letter Question Final paper
-
5.4.1 The Noisy-Or Model
2 causal mechanism
P (l q , f ) = 0.8
20%
P (l q , f ) = 0.9
10%
1 1 0
1 0 1
-
5.4.1 The Noisy-Or Model
0.2 0.1 = 0.02
P (l q , f ) = 0.981 1 1
-
5.4.1 The Noisy-Or Model
QF
= P ( q ) = 0.8 = P ( f ) = 0.9
leak probabilityLetter = 0.0001
Q q1 1
F f1 1
0
-
5.4.1 The Noisy-Or Model
Definition 5.8 noisyor CPD
Yk2X , ...,X 2P (y X , ...,X ) = (1 ) (1 )P (y X , ...,X ) = 1 [(1 ) (1 )]k + 1 , , ..., PCD P (Y X , ...,X )noisyor
x 1, x 0
P (y X , ...,X ) = (1 ) (1 )
1 k0
1 k 0 i:X =xi i1 i1
1 k 0 i:X =xi i1 i0 1 k
1 k
i1
i0
01 k 0
i=1
i xi
-
5.4.1 The Noisy-Or Model
noisyorXP (Y )
a: = 0
b: = 0.50
0
-
5.4.2 Generalized Linear Models
causal influenceGeneralized Linear ModelsYP (Y X , ...X )
1 k
-
5.4.2 Generalized Linear Models
5.4.2.1 2burdentotal burden ...
f(X , ...,X ) = w X
w
f(X , ...,X )f(X , ...,X ) = w + w X
w =
1 k i=1k
i i
i
1 k
1 k 0 i=1k
i i
0
-
5.4.2 Generalized Linear Models
5.4.2.1 2
sigmoid(z) = 1+ezez
-
5.4.2 Generalized Linear Models
5.4.2.1 2
Definition 5.9 logistic CPD
YkX , ...,X 2P (y X , ...,X ) = sigmoid(w + w X )k + 1w ,w , ...,w CPD P (Y X , ...,X )logistic CPD
wY
2y y O(X) = = = e
X
= = e
1 k1
1 k 0 i=1k
i i
0 1 k
1 k
1 0
P (y X ,...,X )0 1 kP (y X ,...,X )1 1 k
1/(1+e )ze /(1+e )z z z
j
O(X ,x )j j0
O(X ,x )j j1
exp(w + w X )0 ij i i
exp(w + w X +w )0 ij i i j wj
-
5.4.2 Generalized Linear Models
5.4.2.1 2
wlogical CPDb: w = 0, c: w = 5, d: ww 10
bnoisyorw
logistic CPDw X Y
0 0 0
i i
-
5.4.2 Generalized Linear Models
5.4.2.2 Yy , ..., y logistic CPDY"winnertakesall"y 10
Definition 5.10 multinomial logistic PCD
YkX , ...,X mj = 1, ...,ml (X , ...,X ) = w + w XP (y X , ...,X ) =
k + 1w ,w , ...,w CPD P (Y X , ...,X )multinomial logistic CPD
1 n
i
1 k
i 1 k j,0 i=1k
j,i ij
1 k exp(l (X ,...,X ))j =1m
j 1 k
exp(l (X ,...,X ))j 1 k
j,0 j,1 j,k
1 k
-
5.4.2 Generalized Linear Models
5.4.2.2
X ,X 3YX 2
X = x , ...,x X = jX = x 2X , ...,X mX2Ym+ 1P (y X) = sigmoid(w + w 1{X = x })
1 2
i
i i1
im
i i,j i,j1
i,1 i,m
10 j=1
mj
j
chapter5_01chapter5_02