learning specific-class segmentation from diverse data m. pawan kumar, haitherm turki, dan preston...
TRANSCRIPT
Learning Specific-Class Segmentation from Diverse Data
M. Pawan Kumar, Haitherm Turki, Dan Preston and Daphne Koller at ICCV 2011
VGG reading group, 29 Nov 2011, presented by Varun Gulshan
Semantic image segmentation
Main idea
• High level: Getting fully labelled data for training is expensive, use other easily available ‘diverse’ data for learning (bounding boxes, classification labels for image).
Tags: Car, peoplePerson bounding box
Implementing the idea
• The bounding box/image classification data is incomplete for segmentation, fill in the missing information using latent variables.
• Setup the training cost function using latent variables. Use their self-paced learning algorithm for Latent-SVM’s [NIPS2010] to optimise the training cost function.
• While inferring latent variables, make sure latent variable estimation is consistent with the weak annotation. Setting up the inference problems to ensure this condition.
Energy function without latent variables
Notation:
Image
Parameters to be trained
Joint feature vector (essentially the terms of a CRF)
Structured output training
Ground truth labels
Loss function
Introducing latent variables
Introducing latent variables
But we don’t know what hk is (its latent), so maximise it out.
Introducing latent variables
Self-paced optimisation
Self-paced optimisation
Indicator variable to switch off the harder cases.
Second idea: Latent variable estimation
The algorithm involves estimating annotation consistent latent variables in the following equation:
More precisely
Move to white-board
Me
You
Beware of Equations