discrete optimization in computer visionimagine.enpc.fr/~marletr/enseignement/mathimage/... ·...
TRANSCRIPT
![Page 1: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/1.jpg)
Discrete Optimization in
Computer Vision
Nikos Komodakis
Ecole des Ponts ParisTech LIGM
Traitement de l’information et vision artificielle
![Page 2: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/2.jpg)
Goal of lectures on optimization
Explain some basic principles and ideas behind
discrete optimization as used in vision
Present some state-of-the-art techniques
Understand how to apply them in practice to
problems in image analysis
![Page 3: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/3.jpg)
Discrete optimization algorithms
Widely used in computer vision/image analysis
(but also in many other domains of applied
mathematics)
principled approach to solving a wide variety of
problems
Active research topic, many open problems.
![Page 4: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/4.jpg)
Why optimization ?
Image data
(observations)
Model
(constraints,
assumptions, …)
Infer solutions
![Page 5: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/5.jpg)
Why optimization ?
Noise in images
Ambiguity of visual data
…
Constraints/assumptions
never satisfied exactly
![Page 6: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/6.jpg)
Why optimization ?
Noise in images
Ambiguity of visual data
…
Constraints/assumptions
never satisfied exactly
Therefore, we must find solutions that:
- minimize the amount of violation of constraints
- are most probable based on the assumptions
![Page 7: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/7.jpg)
The sky is blue…
![Page 8: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/8.jpg)
![Page 9: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/9.jpg)
![Page 10: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/10.jpg)
Discrete optimization
Each xi take discrete values, e.g., {0, 1}
![Page 11: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/11.jpg)
Discrete optimization
Each xi take discrete values, e.g., {0, 1}
Discrete optimization problems not necessarily
easier than continuous ones
quite often the opposite holds, e.g., LP vs integer LP
often useful connections between discrete and
continuous problems (e.g. relaxations)
![Page 12: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/12.jpg)
Discrete optimization
Each xi take discrete values, e.g., {0, 1}
Discrete optimization problems not necessarily
easier than continuous ones
quite often the opposite holds, e.g., LP vs integer LP
often useful connections between discrete and
continuous problems (e.g. relaxations)
Also, often close connection between discrete
optimization and combinatorial algorithms on graphs
![Page 13: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/13.jpg)
Using discrete optimization in vision
For any problem, two stages required:
modeling/formulation
optimization algorithm
Both stages are important and also interrelated
Trade-offs between them are often required
Let’s take a look at a simple example
![Page 14: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/14.jpg)
Stereo matching
(one of the so-called “early vision”
problems)
![Page 15: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/15.jpg)
Stereo matching
Input: left and right images
Goal: extract disparity (equivalently depth)
Many practical applications
![Page 16: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/16.jpg)
![Page 17: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/17.jpg)
![Page 18: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/18.jpg)
Ground truth disparity
![Page 19: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/19.jpg)
How can we model stereo-matching as a
discrete optimization problem?
![Page 20: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/20.jpg)
Stereo matching
A first attempt:
simply compare the intensity/color of pixels in the
left and right images
What is the resulting objective function in this case ?
Is this enough ?
![Page 21: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/21.jpg)
![Page 22: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/22.jpg)
Stereo matching
Any additional knowledge to encode ?
![Page 23: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/23.jpg)
Stereo matching
Any additional knowledge to encode ?
- Depth maps (disparity fields) are smooth
- Neighboring pixels will have similar disparity
Resulting objective function ?
![Page 24: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/24.jpg)
Stereo matching
Is this enough ?
Any further knowledge to encode ?
![Page 25: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/25.jpg)
Stereo matching
Is this enough ?
Any further knowledge to encode ?
Yes:
disparity field should not be smooth everywhere,
e.g., not along object boundaries.
How to encode this in the objective function ?
![Page 26: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/26.jpg)
Result when using the improved objective function
![Page 27: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/27.jpg)
Stereo matching
Are we completely done ?
No, we could model/encode even more things in the
objective function.
E.g., what about:
- Occlusions
- Specular reflections
- …
![Page 28: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/28.jpg)
![Page 29: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/29.jpg)
![Page 30: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/30.jpg)
![Page 31: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/31.jpg)
![Page 32: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/32.jpg)
Challenges of using the optimization
paradigm in computer vision and
image analysis
![Page 33: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/33.jpg)
Challenges (optimization)
– Problems are often of very large scale (e.g. millions of variables)
– Solution lives in very high-dimensional spaces
![Page 34: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/34.jpg)
Challenges (optimization)
– Highly non-convex objective functions in most cases (NP-hard problems)
– Problems are often of very large scale (e.g. millions of variables)
– Solution lives in very high-dimensional spaces
![Page 35: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/35.jpg)
Challenges (optimization)
– Highly non-convex objective functions in most cases (NP-hard problems)
– Problems are often of very large scale (e.g. millions of variables)
– Solution lives in very high-dimensional spaces
– Computational efficiency extremely important
– Polynomial time not enough
![Page 36: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/36.jpg)
Challenges (modeling)
Great diversity of visual tasks
![Page 37: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/37.jpg)
Challenges (modeling)
Great diversity of visual tasks
![Page 38: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/38.jpg)
Challenges (modeling)
Great diversity of visual tasks
Huge variability of visual data
![Page 39: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/39.jpg)
Challenges (modeling)
Great diversity of visual tasks
Huge variability of visual data
![Page 40: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/40.jpg)
Challenges (modeling)
Great diversity of visual tasks
Huge variability of visual data
Ambiguities of the visual world
![Page 41: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/41.jpg)
Semantic gap
What we see What a computer sees
![Page 42: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/42.jpg)
How to deal with these challenges?
• Powerful mathematical models and algorithms are required
![Page 43: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/43.jpg)
How to deal with these challenges?
• Powerful mathematical models and algorithms are required
• Key characteristics
– robustness
– generality
– flexibility
– ability to efficiently encode prior knowledge
![Page 44: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/44.jpg)
Markov Random Fields (MRFs)
• Very general probabilistic graphical models
– great descriptive/representational power
• Ubiquitous in image analysis
• Applications in many domains – medical imaging, computer vision, statistical physics,
computational biology, digital communications, natural language processing, …
• Key advantages/properties – encode dependencies, constraints, uncertainties, priors
– modular and flexible
![Page 45: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/45.jpg)
Discrete Markov Random Fields vertices G = set of objects
![Page 46: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/46.jpg)
Discrete Markov Random Fields vertices G = set of objects
edges E = object relationships
![Page 47: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/47.jpg)
Discrete Markov Random Fields vertices G = set of objects
edges E = object relationships set L = discrete set of labels
![Page 48: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/48.jpg)
Discrete Markov Random Fields vertices G = set of objects
edges E = object relationships set L = discrete set of labels
Vp(xp) = cost of assigning label xp to vertex p (also called single node potential)
![Page 49: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/49.jpg)
Discrete Markov Random Fields vertices G = set of objects
edges E = object relationships set L = discrete set of labels
Vp(xp) = cost of assigning label xp to vertex p (also called single node potential)
Vpq(xp,xq) = cost of assigning labels (xp,xq) to neighboring vertices (p,q) (also called pairwise potential)
![Page 50: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/50.jpg)
Discrete Markov Random Fields vertices G = set of objects
edges E = object relationships set L = discrete set of labels
Vp(xp) = cost of assigning label xp to vertex p (also called single node potential)
Vpq(xp,xq) = cost of assigning labels (xp,xq) to neighboring vertices (p,q) (also called pairwise potential)
Find labels that minimize the MRF energy (i.e., the
sum of all potentials):
![Page 51: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/51.jpg)
edges objects
Markov Random Fields (MRFs)
{Vp(.), Vpq(.,.)} = MRF potentials
vertices G = set of objects edges E = object relationships
set L = discrete set of labels
![Page 52: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/52.jpg)
edges objects
Markov Random Fields (MRFs)
{Vp(.), Vpq(.,.)} = MRF potentials
vertices G = set of objects edges E = object relationships
set L = discrete set of labels
Can model very broad class of problems in image analysis image segmentation
registration
image restoration
optical flow
stereo matching
object detection
scene understanding
image completion ...
Segmentation
Stereo
Registration
Resto ration
![Page 53: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/53.jpg)
Stereo matching as MRF optimization problem
(see earlier slides)
![Page 54: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/54.jpg)
The Image Completion Problem
Based only on the observed part of an incomplete image, fill its missing part in a visually plausible way
We want to be able to handle: complex natural images with (possibly) large missing regions in an automatic way (i.e. without user intervention)
Many applications: photo editing, film post-production, object removal, text removal, image repairing etc.
![Page 55: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/55.jpg)
Image Completion as a Discrete Global Optimization Problem
S T
sample labels
![Page 56: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/56.jpg)
Image Completion as a Discrete Global Optimization Problem
Labels L = all wxh patches from source region S
S T
sample labels
![Page 57: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/57.jpg)
Image Completion as a Discrete Global Optimization Problem
Labels L = all wxh patches from source region S
MRF nodes = all lattice points whose neighborhood intersects target region T
S
T
sample labels
![Page 58: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/58.jpg)
Image Completion as a Discrete Global Optimization Problem
Labels L = all wxh patches from source region S
MRF nodes = all lattice points whose neighborhood intersects target region T
potential Vp(xp) = how well source patch xp agrees with source region around p
potential Vpq(xp,xq) = how well source patches xp, xq
agree on their overlapping region
S
T
sample labels
![Page 59: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/59.jpg)
Image Completion as a Discrete Global Optimization Problem
Image completion reduces to finding labeling with minimum total energy:
Intuitively, it’s like assembling a huge jigsaw puzzle
^ x
![Page 60: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/60.jpg)
Image completion via global MRF optimization [TIP’07]
![Page 61: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/61.jpg)
Image completion via global MRF optimization [TIP’07]
![Page 62: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/62.jpg)
Image completion via global MRF optimization [TIP’07]
![Page 63: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/63.jpg)
Image completion via global MRF optimization [TIP’07]
![Page 64: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/64.jpg)
Image completion via global MRF optimization [TIP’07]
![Page 65: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/65.jpg)
Image completion via global MRF optimization [TIP’07]
![Page 66: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/66.jpg)
The Texture Synthesis Problem In texture synthesis, we are given as input a small
texture and we want to generate a larger texture of arbitrary size (specified by the user)
![Page 67: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/67.jpg)
Synthesizing textures input
texture
visiting order during 1st
forward pass output texture
![Page 68: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/68.jpg)
Synthesizing textures
input texture
visiting order during 1st forward pass
output texture
![Page 69: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/69.jpg)
Synthesizing textures
![Page 70: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/70.jpg)
Synthesizing textures
![Page 71: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/71.jpg)
Synthesizing textures
![Page 72: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/72.jpg)
Synthesizing textures
![Page 73: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/73.jpg)
Synthesizing textures
This doctored photo used in a Bush campaign ad features cloned U.S. troops
![Page 74: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/74.jpg)
Interactive image segmentation
![Page 75: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/75.jpg)
Interactive image segmentation
![Page 76: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/76.jpg)
Interactive image segmentation
![Page 77: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/77.jpg)
MRFs
In image analysis applications we often use grid-like graphs
In general, however, any graph can be used (this depends on the problem at hand)
A grid-structured graph for a 4x4 image
![Page 78: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/78.jpg)
MRFs
Unary and pairwise potentials of MRF model often
encode data and prior terms:
Data and prior terms serve contradictory roles
(fit to observations)
(contextual constraints)
![Page 79: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/79.jpg)
MRFs
MRFs provide an EFFICIENT way for modeling spatial
dependence between pixels
Non-neighboring pixels are not independent
An efficient way for modeling global and far-away
dependencies through local dependencies
Trying to directly model global dependencies is usually
intractable. Why?
![Page 80: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/80.jpg)
MRFs and probabilities
In a nutshell, MRFs can be thought of as a neat way of specifying probability distributions.
Think of x={xp} as a random vector
We want to define a probability on x
QUESTION: How can we do that efficiently when x is a high-dimensional random vector?
![Page 81: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/81.jpg)
MRFs to the rescue…
![Page 82: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/82.jpg)
MRFs to the rescue…
![Page 83: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/83.jpg)
MRFs to the rescue…
Another example:
![Page 84: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/84.jpg)
MRFs to the rescue…
Equivalent definition for MRFs:
![Page 85: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/85.jpg)
MRFs to the rescue…
Equivalent definition for MRFs:
E.g, for 1-dimensional MRFs, this property simply says:
past and future are independent given present
![Page 86: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/86.jpg)
MRFs to the rescue…
Theorem: If the above mentioned conditional independence
assumptions hold true then:
In the above formula:
The sum is taken over all cliques of the graph
Z is a normalizing constant
(also known as the “partition function”)
![Page 87: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/87.jpg)
MRFs to the rescue… What is the form of the probability distribution for the
graph below?
![Page 88: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/88.jpg)
MRFs to the rescue… What is the form of the probability distribution for the
graph below?
![Page 89: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/89.jpg)
MRFs to the rescue… What is the form of the probability distribution for the
graph below?
What is the form of the probability distribution if we
assume that the potentials for all cliques of size larger than
2 is zero?
![Page 90: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/90.jpg)
MRFs to the rescue… What is the form of the probability distribution for the
graph below?
What is the form of the probability distribution if we
assume that the potentials for all cliques of size larger than
2 is zero?
What is the form of the probability distribution if, in
addition, we assume that some of the random variables are
observed?
![Page 91: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/91.jpg)
MRFs and probabilities
MRFs are just a special class of the so called
probabilistic graphical models (combination of
graph theory + probability)
MRF graphs contain only undirected edges
Other big class: probabilistic graphical models with
directed edges
![Page 92: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/92.jpg)
MRF optimization algorithms
![Page 93: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/93.jpg)
MRF optimization algorithms
• Local optimization algorithms
– yield just a local minimum
• Global optimization algorithms – yield solution with energy close to the global
minimum energy
• Today, we will look at a very simple (but not very effective) local optimization algorithm
![Page 94: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/94.jpg)
The Iterated Conditional Modes
(ICM) algorithm
![Page 95: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/95.jpg)
Iterated Conditional Modes (ICM)
1. start with an initial solution x(k)
2. For each node p in the MRF graph do
3. estimate local energy at p for all possible labels (assuming that all neighboring labels are fixed)
4. assign to p the label with minimum local energy
5. end for
6. if not converged goto 2
![Page 96: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/96.jpg)
ICM algorithm
ICM essentially corresponds to a greedy coordinate-wise descent
Advantages
each iteration is fast
Disadvantages?
![Page 97: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/97.jpg)
ICM algorithm
ICM essentially corresponds to a greedy coordinate-wise descent
Advantages
each iteration is fast
Disadvantages?
very sensitive to initialization (trapped to nearest local minima)
In practice, this is what happens for difficult problems
(bad solutions)
No guarantees about quality of solutions
Also, convergence may be slow (why?)
![Page 98: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/98.jpg)
Example: denoising of 1D signals
(Gaussian noise)
Input: noisy signal y
Goal: find true signal x
Assumptions:
Noise is Gaussian
True signal is smooth. We will also assume a quadratic smoothness prior.
Under these assumptions, the problem becomes very easy (convex) and we will apply ICM to solve it
![Page 99: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/99.jpg)
Example: denoising of 1D signals (Gaussian noise) What is the MRF energy we want to minimize?
![Page 100: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/100.jpg)
Example: denoising of 1D signals (Gaussian noise) What is the MRF energy we want to minimize?
(what is the role of parameter λ?)
![Page 101: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/101.jpg)
Example: denoising of 1D signals (Gaussian noise) What is the MRF energy we want to minimize?
What is the local energy at node i?
(what is the role of parameter λ?)
![Page 102: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/102.jpg)
Example: denoising of 1D signals (Gaussian noise) What is the MRF energy we want to minimize?
What is the local energy at node i?
(what is the role of parameter λ?)
![Page 103: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/103.jpg)
Example: denoising of 1D signals (Gaussian noise) What is the MRF energy we want to minimize?
What is the local energy at node i?
What is the minimizer of the local energy?
(what is the role of parameter λ?)
![Page 104: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/104.jpg)
Example: denoising of 1D signals (Gaussian noise) What is the MRF energy we want to minimize?
What is the local energy at node i?
What is the minimizer of the local energy?
(what is the role of parameter λ?)
![Page 105: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/105.jpg)
λ=10 λ=100 λ=200
![Page 106: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/106.jpg)
![Page 107: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/107.jpg)
![Page 108: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/108.jpg)
![Page 109: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/109.jpg)
(convergence threshold = 0.001)
![Page 110: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/110.jpg)
(convergence threshold = 0.0001)
![Page 111: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/111.jpg)
Example: denoising of 1D signals (Poisson noise) What is the MRF energy we want to minimize?
![Page 112: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/112.jpg)
Example: denoising of 1D signals (Poisson noise) What is the MRF energy we want to minimize?
What is the local energy at node i?
![Page 113: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/113.jpg)
Example: denoising of 1D signals (Poisson noise) What is the MRF energy we want to minimize?
What is the local energy at node i?
![Page 114: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/114.jpg)
Example: denoising of 1D signals (Poisson noise) What is the MRF energy we want to minimize?
What is the local energy at node i?
What is the minimizer of the local energy?
![Page 115: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/115.jpg)
![Page 116: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/116.jpg)
![Page 117: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/117.jpg)
![Page 118: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/118.jpg)
![Page 119: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/119.jpg)
![Page 120: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/120.jpg)
Example: denoising of 1D signals (Gaussian noise + linear prior) What is the MRF energy we want to minimize?
![Page 121: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/121.jpg)
Example: denoising of 1D signals (Gaussian noise + linear prior) What is the MRF energy we want to minimize?
What is the local energy at node i?
![Page 122: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/122.jpg)
Example: denoising of 1D signals (Gaussian noise + linear prior) What is the MRF energy we want to minimize?
What is the local energy at node i?
![Page 123: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/123.jpg)
Example: denoising of 1D signals (Gaussian noise + linear prior) What is the MRF energy we want to minimize?
What is the local energy at node i?
What is the minimizer of the local energy?
![Page 124: Discrete Optimization in Computer Visionimagine.enpc.fr/~marletr/enseignement/mathimage/... · Challenges of using the optimization paradigm in computer vision and image analysis](https://reader033.vdocuments.net/reader033/viewer/2022050106/5f449fd25b2c6345967321f6/html5/thumbnails/124.jpg)
Example: denoising of 1D signals (Gaussian noise + linear prior) What is the MRF energy we want to minimize?
What is the local energy at node i?
What is the minimizer of the local energy?