mean-field theory and its applications in computer vision1 1

Mean-Field Theory and Its Applications In Computer Vision1

1

Introduction

2

• Problem formulation

• Mean-field based inference method

• Strategy for incorporating different costs

Labelling problem

3Stereo Object detection

Assign a label to each image pixel

Object segmentation

Problem Formulation

Find a Labelling that maximize the conditional probability

4

Inference

5

• T. Minka. Expectation Propagation for Approximate Bayesian Inference, UAI, 2001

• Murphy. Loopy Belief Propagation: An Empirical Study, UAI, 1999

• Jordan et.al. An Introduction to Variational Methods for Graphical Models, ML-1999

• J. Yedidia et al. Generalized Belief Propagation, NIPS, 2001

Message Passing• Besag. On the Statistical Analysis of Dirty

Pictures, JRSS, 1986• Boykov et al. Fast Approximate Energy

Minimization via Graph Cuts, PAMI 2001• Komodakis et al. Fast Approximate

Optimal Solutions for Single and Dynamic MRFs, CVPR, 2007

• Lempitsky et al. Fusion Moves for Markov Random Field Optimization, PAMI, 2010

Move-Making

• Chekuri et al. Approximation Algorithms for Metric Labelling, SODA, 2001

• M. Goemans et al. Improved Approximate Algorithms for Maximum-Cut, JACM, 1995

• M. Muramatsu et al. A New SOCP Relaxation for Max-Cut, JORJ, 2003

• RaviKumar et al. QP Relaxation for Metric Labelling, ICML 2006

Convex Relaxations• K. Alahari et.al. Dynamic Hybrid

Algorithms for MAP Inference, PAMI 2010

• P. Kohli et al. On Partial Optimality in Multilabel MRFs, ICML, 2008

• C. Rother et al. Optimizing Binary MRFs via Extended Roof Duality, CVPR, 2007

Other Algorithms

Inference

6

• T. Minka. Expectation Propagation for Approximate Bayesian Inference, UAI, 2001

• Murphy. Loopy Belief Propagation: An Empirical Study, UAI, 1999

• Jordan et.al. An Introduction to Variational Methods for Graphical Models, ML-99

• J. Yedidia et al. Generalized Belief Propagation, NIPS, 2001

Message Passing

• Variational message passing algorithm• We focus on mean-field based inferenceWe focus on mean-field based inference

Mean-field methods

7

• Intractable inference with distribution

P

• Approximate distribution from tractable family

• Mean-fields methods (Jordan et.al., 1999)

Variational Inference

8

• Minimize the KL-divergence between Q and P

Variational Inference

9

• Minimize the KL-divergence between Q and P

Variational Inference

10

• Minimize the KL-divergence between Q and P

Variational Inference

11

• Minimize the KL-divergence between Q and P

Markov Random Field (MRF)

12

• Graph:

• A simple MRF

Product of potentials defined over cliques

Markov Random Field (MRF)

13

• Graph:

• In general

Un-normalized part

Energy minimization

14

• Potential and energy

Variational Inference

15

Entropy of Q

Expectation of costunder Q distribution

Naïve Mean Field

16

• Family : assume all variables are independent

Variational Inference

17

• Shannon’s entropy decomposes

Variational Inference

18

• Stationary point solution• Marginal update in mean-field

• Normalizing constant:

Variational Inference

19

• Marginal for variable i taking label l

Variational Inference

20

• Marginal for variable i taking label l

• An assignment of all variables in clique c

Variational Inference

21

• Marginal for variable i taking label l

• An assignment of all variables in clique c

• An assignment of all variables apart from x_i

Variational Inference

22

• Marginal for variable i taking label l

• An assignment of all variables in clique c

• An assignment of all variables apart from x_i

• Marginal distribution of all variables in c apart from x_i

Variational Inference

23

• Marginal for variable i taking label l

• An assignment of all variables in clique c

• An assignment of all variables apart from x_i

• Marginal distribution of all variables in c apart from x_i

• Summation evaluates the expected value of cost over distribution Q given that x_i takes label l

Simple Illustration

24

Naïve mean-field

approximation

Mean-field algorithm

25

• Iterative algorithm• Iterate till convergence

• Update marginals of each variable in each iteration

Q distribution

26

Max posterior marginal (MPM)

27

• MPM with approximate distribution:

• Empirically achieves very high accuracy:

• MAP solution / most likely solution

Structured Mean Field

28

• Naïve mean field can lead to poor solution• Structured (higher order) mean-field

How to make a mean-field algorithm

29

• Pick a model• Unary, pairwise, higher order cliques

• Define a cost• Potts, linear truncated, robust PN

• Calculate the marginal • Calculate the expectation of cost defined

How to make a mean-field algorithm

30

• Use this plug-in strategy in many different models• Grid pairwise CRF• Dense pairwise CRF• Higher order model• Co-occurrence model • Latent variable model• Product label space