image blending and compositing
TRANSCRIPT
![Page 1: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/1.jpg)
Image Blending and Compositing
© NASA
CS194: Image Manipulation & Computational Photography Alexei Efros, UC Berkeley, Fall 2016
![Page 2: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/2.jpg)
Image Compositing
![Page 3: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/3.jpg)
Compositing Procedure 1. Extract Sprites (e.g using Intelligent Scissors in Photoshop)
Composite by David Dewey
2. Blend them into the composite (in the right order)
![Page 4: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/4.jpg)
Alpha Blending / Feathering
0 1
0 1
+
= Iblend = αIleft + (1-α)Iright
![Page 5: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/5.jpg)
Pyramid Blending
![Page 6: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/6.jpg)
Gradient Domain vs. Frequency Domain In Pyramid Blending, we decomposed our
images into several frequency bands, and transferred them separately • But boundaries appear across multiple bands
But what about representation based on derivatives (gradients) of the image?: • Represents local change (across all frequences) • No need for low-res image
– captures everything (up to a constant) • Blending/Editing in Gradient Domain:
– Differentiate – Copy / Blend / edit / whatever – Reintegrate
![Page 7: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/7.jpg)
Gradients vs. Pixels
![Page 8: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/8.jpg)
Gilchrist Illusion (c.f. Exploratorium)
![Page 9: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/9.jpg)
![Page 10: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/10.jpg)
White?
![Page 11: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/11.jpg)
White?
![Page 12: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/12.jpg)
White?
![Page 13: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/13.jpg)
![Page 14: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/14.jpg)
Drawing in Gradient Domain
James McCann & Nancy Pollard
Real-Time Gradient-Domain Painting, SIGGRAPH 2009
(paper came out of this class!) http://www.youtube.com/watch?v=RvhkAfrA0-w&feature=youtu.be
![Page 15: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/15.jpg)
Gradient Domain blending (1D)
Two signals
Regular blending
Blending derivatives
bright
dark
![Page 16: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/16.jpg)
Gradient hole-filling (1D)
source target
![Page 17: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/17.jpg)
It is impossible to faithfully preserve the gradients
source target
![Page 18: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/18.jpg)
Gradient Domain Blending (2D)
Trickier in 2D: • Take partial derivatives dx and dy (the gradient field) • Fiddle around with them (copy, blend, smooth, feather, etc) • Reintegrate
– But now integral(dx) might not equal integral(dy) • Find the most agreeable solution
– Equivalent to solving Poisson equation – Can be done using least-squares (\ in Matlab)
![Page 19: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/19.jpg)
Example
Source: Evan Wallace
Gradient Visualization
![Page 20: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/20.jpg)
Source: Evan Wallace
+
Specify object region
![Page 21: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/21.jpg)
Poisson Blending Algorithm A good blend should preserve gradients of source
region without changing the background Treat pixels as variables to be solved
– Minimize squared difference between gradients of foreground region and gradients of target region
– Keep background pixels constant
Perez et al. 2003
![Page 22: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/22.jpg)
Examples Gradient domain processing
20 20 20 20
20 80 20 20
20 20 80 20
20 20 20 20
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
10 10 10 10
10 10 10 10
10 10 10 10
10 10 10 10
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
source image background image target image 10 10 10 10
10 v1 v3 10
10 v2 v4 10
10 10 10 10
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
![Page 23: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/23.jpg)
Gradient-domain editing
Creation of image = least squares problem in terms of: 1) pixel intensities; 2) differences of pixel intensities
Least Squares Line Fit in 2 Dimensions
( )( )2
2
minargˆ
minargˆ
bAvv
vav
v
v
−=
−= ∑i
iTi b
Use Matlab least-squares solvers for numerically stable solution with sparse A
![Page 24: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/24.jpg)
Perez et al., 2003
![Page 25: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/25.jpg)
source target mask
no blending gradient domain blending
Slide by Mr. Hays
![Page 26: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/26.jpg)
What’s the difference?
no blending gradient domain blending
- =
Slide by Mr. Hays
![Page 27: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/27.jpg)
Perez et al, 2003
Limitations: • Can’t do contrast reversal (gray on black -> gray on white) • Colored backgrounds “bleed through” • Images need to be very well aligned
editing
![Page 28: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/28.jpg)
Gradient Domain as Image Representation
See GradientShop paper as good example: http://www.gradientshop.com/
![Page 29: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/29.jpg)
Can be used to exert high-level control over images
![Page 30: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/30.jpg)
Can be used to exert high-level control over images gradients – low level image-features
![Page 31: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/31.jpg)
Can be used to exert high-level control over images gradients – low level image-features
+100 pixel gradient
![Page 32: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/32.jpg)
Can be used to exert high-level control over images gradients – low level image-features gradients – give rise to high level image-features
+100 pixel gradient
![Page 33: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/33.jpg)
Can be used to exert high-level control over images gradients – low level image-features gradients – give rise to high level image-features
+100
+100
+100
+100
+100
+100 pixel gradient
+100
+100
+100
+100
+100
+100 pixel gradient
![Page 34: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/34.jpg)
Can be used to exert high-level control over images gradients – low level image-features gradients – give rise to high level image-features
+100
+100
+100
+100
+100
+100 pixel gradient
+100
+100
+100
+100
+100
+100 pixel gradient
image edge image edge
![Page 35: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/35.jpg)
Can be used to exert high-level control over images gradients – low level image-features gradients – give rise to high level image-features manipulate local gradients to
manipulate global image interpretation
+100
+100
+100
+100
+100
+100 pixel gradient
+100
+100
+100
+100
+100
+100 pixel gradient
![Page 36: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/36.jpg)
Can be used to exert high-level control over images gradients – low level image-features gradients – give rise to high level image-features manipulate local gradients to
manipulate global image interpretation
+255
+255
+255
+255
+255
pixel gradient
![Page 37: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/37.jpg)
Can be used to exert high-level control over images gradients – low level image-features gradients – give rise to high level image-features manipulate local gradients to
manipulate global image interpretation
+255
+255
+255
+255
+255
pixel gradient
![Page 38: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/38.jpg)
Can be used to exert high-level control over images gradients – low level image-features gradients – give rise to high level image-features manipulate local gradients to
manipulate global image interpretation
+0
+0
+0
+0
+0
pixel gradient
![Page 39: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/39.jpg)
Can be used to exert high-level control over images gradients – low level image-features gradients – give rise to high level image-features manipulate local gradients to
manipulate global image interpretation
+0
+0
+0
+0
+0
pixel gradient
![Page 40: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/40.jpg)
Can be used to exert high-level control over images
![Page 41: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/41.jpg)
Optimization framework
Pravin Bhat et al
![Page 42: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/42.jpg)
Optimization framework Input unfiltered image – u
![Page 43: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/43.jpg)
Optimization framework Input unfiltered image – u Output filtered image – f
![Page 44: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/44.jpg)
Optimization framework Input unfiltered image – u Output filtered image – f Specify desired pixel-differences – (gx, gy)
min (fx – gx)2 + (fy – gy)2
f
Energy function
![Page 45: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/45.jpg)
Optimization framework Input unfiltered image – u Output filtered image – f Specify desired pixel-differences – (gx, gy) Specify desired pixel-values – d
min (fx – gx)2 + (fy – gy)2 + (f – d)2
f
Energy function
![Page 46: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/46.jpg)
Optimization framework Input unfiltered image – u Output filtered image – f Specify desired pixel-differences – (gx, gy) Specify desired pixel-values – d Specify constraints weights – (wx, wy, wd)
min wx(fx – gx)2 + wy(fy – gy)2 + wd(f – d)2
f
Energy function
![Page 47: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/47.jpg)
![Page 48: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/48.jpg)
![Page 49: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/49.jpg)
![Page 50: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/50.jpg)
Pseudo image relighting change scene illumination
in post-production example
input
![Page 51: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/51.jpg)
Pseudo image relighting change scene illumination
in post-production example
manual relight
![Page 52: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/52.jpg)
Pseudo image relighting change scene illumination
in post-production example
input
![Page 53: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/53.jpg)
Pseudo image relighting change scene illumination
in post-production example
GradientShop relight
![Page 54: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/54.jpg)
Pseudo image relighting change scene illumination
in post-production example
GradientShop relight
![Page 55: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/55.jpg)
Pseudo image relighting change scene illumination
in post-production example
GradientShop relight
![Page 56: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/56.jpg)
Pseudo image relighting change scene illumination
in post-production example
GradientShop relight
![Page 57: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/57.jpg)
Pseudo image relighting
u
f
o
![Page 58: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/58.jpg)
Pseudo image relighting
min wx(fx – gx)2 + f wy(fy – gy)2 + wd(f – d)2
Energy function
u
f
o
![Page 59: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/59.jpg)
Pseudo image relighting
min wx(fx – gx)2 + f wy(fy – gy)2 + wd(f – d)2
Energy function
Definition: d = u
u
f
o
![Page 60: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/60.jpg)
Pseudo image relighting
min wx(fx – gx)2 + f wy(fy – gy)2 + wd(f – d)2
Energy function
u
f
o Definition: d = u gx(p) = ux(p) * (1 + a(p)) a(p) = max(0, - u(p).o(p))
![Page 61: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/61.jpg)
Pseudo image relighting
min wx(fx – gx)2 + f wy(fy – gy)2 + wd(f – d)2
Energy function
u
f
o Definition: d = u gx(p) = ux(p) * (1 + a(p)) a(p) = max(0, - u(p).o(p))
![Page 62: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/62.jpg)
Sparse data interpolation Interpolate scattered data
over images/video
![Page 63: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/63.jpg)
Sparse data interpolation Interpolate scattered data
over images/video Example app: Colorization*
input output *Levin et al. – SIGRAPH 2004
![Page 64: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/64.jpg)
u
f
user data
Sparse data interpolation
![Page 65: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/65.jpg)
u
f
user data
Sparse data interpolation
min wx(fx – gx)2 + f wy(fy – gy)2 + wd(f – d)2
Energy function
![Page 66: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/66.jpg)
u
f
user data
Sparse data interpolation
min wx(fx – gx)2 + f wy(fy – gy)2 + wd(f – d)2
Energy function
Definition: d = user_data
![Page 67: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/67.jpg)
u
f
user data
Sparse data interpolation
min wx(fx – gx)2 + f wy(fy – gy)2 + wd(f – d)2
Energy function
Definition: d = user_data if user_data(p) defined
wd(p) = 1 else wd(p) = 0
![Page 68: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/68.jpg)
u
f
user data
Sparse data interpolation
min wx(fx – gx)2 + f wy(fy – gy)2 + wd(f – d)2
Energy function
Definition: d = user_data if user_data(p) defined
wd(p) = 1 else wd(p) = 0
gx(p) = 0; gy(p) = 0
![Page 69: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/69.jpg)
u
f
user data
Sparse data interpolation
min wx(fx – gx)2 + f wy(fy – gy)2 + wd(f – d)2
Energy function
Definition: d = user_data if user_data(p) defined
wd(p) = 1 else wd(p) = 0
gx(p) = 0; gy(p) = 0 wx(p) = 1/(1 + c*|ux(p)|)
wy(p) = 1/(1 + c*|uy(p)|)
![Page 70: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/70.jpg)
Don’t blend, CUT!
So far we only tried to blend between two images. What about finding an optimal seam?
Moving objects become ghosts
![Page 71: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/71.jpg)
Davis, 1998 Segment the mosaic
• Single source image per segment • Avoid artifacts along boundries
– Dijkstra’s algorithm
![Page 72: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/72.jpg)
min. error boundary
Minimal error boundary
overlapping blocks vertical boundary
_ = 2
overlap error
![Page 74: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/74.jpg)
Seam Carving
• Basic Idea: remove unimportant pixels from the image
– Unimportant = pixels with less “energy”
• Intuition for gradient-based energy: – Preserve strong contours – Human vision more sensitive to edges – so try remove content from
smoother areas – Simple, enough for producing some nice results – See their paper for more measures they have used
Michael Rubinstein — MIT CSAIL – [email protected]
![Page 76: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/76.jpg)
The Optimal Seam
)(minarg* sEsS
=||||)( III yxE∂∂+
∂∂=
Michael Rubinstein — MIT CSAIL – [email protected]
![Page 77: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/77.jpg)
Dynamic Programming
• Invariant property: – M(i,j) = minimal cost of a seam going through (i,j) (satisfying the seam
properties)
5 8 12 3
9 2 3 9
7 3 4 2
4 5 7 8
Michael Rubinstein — MIT CSAIL – [email protected]
![Page 78: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/78.jpg)
Dynamic Programming
5 8 12 3
9 2+5 3 9
7 3 4 2
4 5 7 8
( ))1,1(),,1(),1,1(min),(),( +−−−−+= jijijijiEji MMMM
Michael Rubinstein — MIT CSAIL – [email protected]
![Page 79: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/79.jpg)
Dynamic Programming
5 8 12 3
9 7 3+3 9
7 3 4 2
4 5 7 8
( ))1,1(),,1(),1,1(min),(),( +−−−−+= jijijijiEji MMMM
Michael Rubinstein — MIT CSAIL – [email protected]
![Page 80: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/80.jpg)
Dynamic Programming
5 8 12 3
9 7 6 12
14 9 10 8
14 14 15 8+8
( ))1,1(),,1(),1,1(min),(),( +−−−−+= jijijijiEji MMMM
Michael Rubinstein — MIT CSAIL – [email protected]
![Page 81: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/81.jpg)
Searching for Minimum
• Backtrack (can store choices along the path, but do not have to)
5 8 12 3
9 7 6 12
14 9 10 8
14 14 15 16
Michael Rubinstein — MIT CSAIL – [email protected]
![Page 82: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/82.jpg)
Backtracking the Seam
5 8 12 3
9 7 6 12
14 9 10 8
14 14 15 16
Michael Rubinstein — MIT CSAIL – [email protected]
![Page 83: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/83.jpg)
Backtracking the Seam
5 8 12 3
9 7 6 12
14 9 10 8
14 14 15 16
Michael Rubinstein — MIT CSAIL – [email protected]
![Page 84: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/84.jpg)
Backtracking the Seam
5 8 12 3
9 7 6 12
14 9 10 8
14 14 15 16
Michael Rubinstein — MIT CSAIL – [email protected]
![Page 85: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/85.jpg)
Graphcuts What if we want similar “cut-where-things-
agree” idea, but for closed regions? • Dynamic programming can’t handle loops
![Page 86: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/86.jpg)
Graph cuts – a more general solution
n-links
s
t a cut hard constraint
hard constraint
Minimum cost cut can be computed in polynomial time (max-flow/min-cut algorithms)
![Page 87: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/87.jpg)
e.g. Lazy Snapping
Interactive segmentation using graphcuts Also see the original Boykov&Jolly, ICCV’01, “GrabCut”, etc, etc ,etc.
![Page 88: Image Blending and Compositing](https://reader030.vdocuments.net/reader030/viewer/2022012717/61af4cc48b53ac154e684f47/html5/thumbnails/88.jpg)
Putting it all together Compositing images
• Have a clever blending function – Feathering – blend different frequencies differently – Gradient based blending
• Choose the right pixels from each image – Dynamic programming – optimal seams – Graph-cuts
Now, let’s put it all together:
• Interactive Digital Photomontage, 2004 (video)