junction tree algorithm 10-708:probabilistic graphical models recitation: 10/04/07 ramesh nallapati

Junction tree Algorithm

10-708:Probabilistic Graphical Models

Recitation: 10/04/07

Ramesh Nallapati

Cluster Graphs

A cluster graph K for a set of factors F is an undirected graph with the following properties: Each node i is associated with a subset Ci ½ X Family preserving property: each factor is such that

scope[] µ Ci

Each edge between Ci and Cj is associated with a sepset Sij = Ci Å Cj

Execution of variable elimination defines a cluster-graph Each factor used in elimination becomes a cluster-node An edge is drawn between two clusters if a message is

passed between them in elimination Example: Next slide

Variable Elimination to Junction Trees:

Original graph

Difficulty Intelligence

Coherence

Grade SAT

Happy

Letter

Job


Moralized graph


Coherence

Grade SAT

Happy

Letter

Job


Triangulated graph


Coherence

Grade SAT

Happy

Letter

Job


Elimination ordering: C, D, I, H, G, S, L


Coherence

Grade SAT

Happy

Letter

Job

C,D

D



Coherence

Grade SAT

Happy

Letter

Job

C,D

D,I,G

D

G,I





Coherence

Grade SAT

Happy

Letter

Job

C,D

D,I,G

D

G,I,S

G,I

G,S




Coherence

Grade SAT

Happy

Letter

Job

C,D

D,I,G

D

G,I,S

G,I

H,G,J

G,S

G,J




Coherence

Grade SAT

Happy

Letter

Job

C,D

D,I,G

D

G,I,S

G,I

H,G,J

G,J

G,J,S,L

G,SJ,S,L




Coherence

Grade SAT

Happy

Letter

Job

C,D

D,I,G

D

G,I,S

G,I

H,G,J

G,J

G,J,S,L

G,S

J,S,LJ,S,L

L,J




Coherence

Grade SAT

Happy

Letter

Job

C,D

D,I,G

D

G,I,S

G,I

H,G,J

G,J

G,J,S,L

G,S

J,S,LJ,S,L

L,J

L,J

Properties of Junction Tree

Cluster-graph G induced by variable elimination is necessarily a tree Reason: each intermediate factor is used atmost

once G satisfies Running Intersection Property (RIP)

(X 2 Ci & X in Cj) ) X 2 CK where Ck is in the path of Ci and Cj

If Ci and Cj are neighboring clusters, and Ci passes message mij to Cj, then scope[mij] = Si,j

Let F be set of factors over X. A cluster tree over F that satisfies RIP is called a junction tree

One can obtain a minimal junction tree by eliminating the sub-cliques No redundancies

C,D

D,I,G

D

G,I,S

G,I

H,G,J

G,J

G,J,S,L

G,S

J,S,LJ,S,L

L,J

L,J

Junction Trees to Variable elimination:

Now we will assume a junction tree and show how to do variable elimination


Coherence

Grade SAT

Happy

Letter

Job

1: C,D

2: G,I,D

3: G,S,I

4: G,J,S,L

5: H,G,J

D

G,I

G,S

G,J

Junction Trees to Variable Elimination:

Initialize potentials first:


Coherence

Grade SAT

Happy

Letter

Job

1: C,D

2: G,I,D

3: G,S,I

4:G,J,S,L

5:H,G,J

D

G,I

G,S

G,J

01(C,D) =

P(C)P(D|C)

02(G,I,D) = P(G|

D,I)

03(G,S,I) =

P(I)P(S|I)

04(G,J,S,L) = P(L|

G)P(J|S,L)

05(H,G,J) = P(H|

G,J)

Junction Trees to Variable Elimination:

Pass messages: (C4 is the root)

1: C,D

2: G,I,D

3: G,S,I

4:G,J,S,L

5:H,G,J

D

G,I

G,S

G,J

01(C,D) =

P(C)P(D|C)

02(G,I,D) = P(G|

D,I)

03(G,S,I) =

P(I)P(S|I)

04(G,J,S,L) = P(L|

G)P(J|S,L)

05(H,G,J) = P(H|

G,J)

1! 2(D) = C 01(C,D)

2! 3(G,I) = D 0

2(G,I,D)1! 2(D)

3! 4(G,S) = I 0

3(G,S,I)2! 3(G,I)

5! 4(G,J) = H 05(H,G,J)

4(G,J,S,L) = 3 ! 4(G,S)5 ! 4(G,J)0

4(G,J,S,L)

Junction Tree calibration

Aim is to compute marginals of each node using least computation Similar to the 2-pass sum-product algorithm

Ci transmits a message to its neighbor Cj after it receives messages from all other neighbors

Called “Shafer-Shenoy” clique tree algorithm

1: C,D 2: G,I,D 3: G,S,I 4:G,J,S,L 5:H,G,J

Message passing with division

Consider calibrated potential at node Ci

whose neighbor is Cj

Consider message from Ci to C

j

Hence, one can write:

Ci

Cj

Message passing with division

Belief-update or Lauritzen-Speigelhalter algorithm Each cluster Ci maintains its fully updated current

beliefs i

Each sepset sij maintains ij, the previous message passed between Ci-Cj regardless of direction

Any new message passed along Ci-Cj is divided by ij

Belief Update message passingExample

1: A,B 2: B,C 3: C,DB C

12 = 1 ! 2(B) 23 = 3 ! 2(C)

2! 1(B)

This is what we expect to send in the regular message passing!

Actual message

Belief Update message passingAnother Example

1: A,B 2: B,C 3: C,DB C

2 ! 3(C) = 023

3 ! 2(C) = 123

This is exactly the message C2 would have received from C3 if C2 didn’t send an uninformed message: Order of messages doesn’t matter!

Belief Update message passingJunction tree invariance

Recall: Junction Tree measure:

A message from Ci to Cj changes only j and ij:

Thus the measure remains unchanged for updated potentials too!

Junction trees from Chordal graphs

Recall: A junction tree can be obtained by the induced graph from variable elimination

Alternative approach: using chordal graphs Recall:

Any chordal graph has a clique tree Can obtain chordal graphs through triangulation

Finding a minimum triangulation, where largest clique has minimum size is NP-hard

Junction trees from Chordal graphsMaximum spanning tree algorithm

Original Graph


Coherence

Grade SAT

Happy

Letter

Job


Undirected moralized graph


Coherence

Grade SAT

Happy

Letter

Job


Chordal (Triangulated) graph


Coherence

Grade SAT

Happy

Letter

Job


Cluster graph


Coherence

Grade SAT

Happy

Letter

Job

C,D

D,I,G

1

G,I,S

2

L,S,J

2

G,S,L

2

G,H1 1

1

1

1


Junction tree


Coherence

Grade SAT

Happy

Letter

Job

C,D

D,I,G

D

G,I,S

G,I

L,S,J

S,L

G,S,L

G,S

G,HG

Summary

Junction tree data-structure for exact inference on general graphs

Two methods Shafer-Shenoy Belief-update or Lauritzen-Speigelhalter

Constructing Junction tree from chordal graphs Maximum spanning tree approach

junction tree algorithm 10-708:probabilistic graphical models recitation: 10/04/07 ramesh nallapati

Documents