mean-field theory and its applications in computer vision1 1
TRANSCRIPT
Mean-Field Theory and Its Applications In Computer Vision1
1
Introduction
2
• Problem formulation
• Mean-field based inference method
• Strategy for incorporating different costs
Labelling problem
3Stereo Object detection
Assign a label to each image pixel
Object segmentation
Problem Formulation
Find a Labelling that maximize the conditional probability
4
Inference
5
• T. Minka. Expectation Propagation for Approximate Bayesian Inference, UAI, 2001
• Murphy. Loopy Belief Propagation: An Empirical Study, UAI, 1999
• Jordan et.al. An Introduction to Variational Methods for Graphical Models, ML-1999
• J. Yedidia et al. Generalized Belief Propagation, NIPS, 2001
Message Passing• Besag. On the Statistical Analysis of Dirty
Pictures, JRSS, 1986• Boykov et al. Fast Approximate Energy
Minimization via Graph Cuts, PAMI 2001• Komodakis et al. Fast Approximate
Optimal Solutions for Single and Dynamic MRFs, CVPR, 2007
• Lempitsky et al. Fusion Moves for Markov Random Field Optimization, PAMI, 2010
Move-Making
• Chekuri et al. Approximation Algorithms for Metric Labelling, SODA, 2001
• M. Goemans et al. Improved Approximate Algorithms for Maximum-Cut, JACM, 1995
• M. Muramatsu et al. A New SOCP Relaxation for Max-Cut, JORJ, 2003
• RaviKumar et al. QP Relaxation for Metric Labelling, ICML 2006
Convex Relaxations• K. Alahari et.al. Dynamic Hybrid
Algorithms for MAP Inference, PAMI 2010
• P. Kohli et al. On Partial Optimality in Multilabel MRFs, ICML, 2008
• C. Rother et al. Optimizing Binary MRFs via Extended Roof Duality, CVPR, 2007
Other Algorithms
Inference
6
• T. Minka. Expectation Propagation for Approximate Bayesian Inference, UAI, 2001
• Murphy. Loopy Belief Propagation: An Empirical Study, UAI, 1999
• Jordan et.al. An Introduction to Variational Methods for Graphical Models, ML-99
• J. Yedidia et al. Generalized Belief Propagation, NIPS, 2001
Message Passing
• Variational message passing algorithm• We focus on mean-field based inferenceWe focus on mean-field based inference
Mean-field methods
7
• Intractable inference with distribution
P
• Approximate distribution from tractable family
• Mean-fields methods (Jordan et.al., 1999)
Variational Inference
8
• Minimize the KL-divergence between Q and P
Variational Inference
9
• Minimize the KL-divergence between Q and P
Variational Inference
10
• Minimize the KL-divergence between Q and P
Variational Inference
11
• Minimize the KL-divergence between Q and P
Markov Random Field (MRF)
12
• Graph:
• A simple MRF
Product of potentials defined over cliques
Markov Random Field (MRF)
13
• Graph:
• In general
Un-normalized part
Energy minimization
14
• Potential and energy
Variational Inference
15
Entropy of Q
Expectation of costunder Q distribution
Naïve Mean Field
16
• Family : assume all variables are independent
Variational Inference
17
• Shannon’s entropy decomposes
Variational Inference
18
• Stationary point solution• Marginal update in mean-field
• Normalizing constant:
Variational Inference
19
• Marginal for variable i taking label l
Variational Inference
20
• Marginal for variable i taking label l
• An assignment of all variables in clique c
Variational Inference
21
• Marginal for variable i taking label l
• An assignment of all variables in clique c
• An assignment of all variables apart from x_i
Variational Inference
22
• Marginal for variable i taking label l
• An assignment of all variables in clique c
• An assignment of all variables apart from x_i
• Marginal distribution of all variables in c apart from x_i
Variational Inference
23
• Marginal for variable i taking label l
• An assignment of all variables in clique c
• An assignment of all variables apart from x_i
• Marginal distribution of all variables in c apart from x_i
• Summation evaluates the expected value of cost over distribution Q given that x_i takes label l
Simple Illustration
24
Naïve mean-field
approximation
Mean-field algorithm
25
• Iterative algorithm• Iterate till convergence
• Update marginals of each variable in each iteration
Q distribution
26
Max posterior marginal (MPM)
27
• MPM with approximate distribution:
• Empirically achieves very high accuracy:
• MAP solution / most likely solution
Structured Mean Field
28
• Naïve mean field can lead to poor solution• Structured (higher order) mean-field
How to make a mean-field algorithm
29
• Pick a model• Unary, pairwise, higher order cliques
• Define a cost• Potts, linear truncated, robust PN
• Calculate the marginal • Calculate the expectation of cost defined
How to make a mean-field algorithm
30
• Use this plug-in strategy in many different models• Grid pairwise CRF• Dense pairwise CRF• Higher order model• Co-occurrence model • Latent variable model• Product label space