1. Edge-Based Image Coarsening
SIGGRAPH 2010
RaananFattal
Hebrew University of Jerusalem, Israel
Robert Carroll
University of California, Berkeley
ManeeshAgrawala
University of California, Berkeley

2. Abstract
A new dimensionally-reduced linear image space
Pixel-by-pixel image sparse grid kernel
High performance on gradient-based tone mapping techniques
Useful for energy-minimization method
3. Outline
Introduction
Image coarsening
Pixel-by-pixel spanned by spare kernel (BIG)
Scale-Adaptive corasening
Projection operations
Image/Gradient projection
Gradient projection
Results applied on
Shadow Removal
Sparse-Error Norm
AlphaMatting
Joint Bilateral upsampling
Conclusion
4. Introduction
State-of-art image editing tech.
Bilateral filter based
Gradient based manipulation
5. Bilateral filter [Tomasi and Manduchi 1998]
A nonlinear filter that locally gathers information from similar pixels
Preserve edge
Noise removal
6. Bilateral filter [Tomasi and Manduchi 1998]
Averaging over the 2N+1 neighborhood
The weight
The neighbor sample
The result at the kth sample
Y[j]
Normalization of the weighting
j
k
6/59
Michael Elad, Algorithms for Noise Removal and the Bilateral Filter.ppt, 2002
7. Bilateral filter [Tomasi and Manduchi 1998]
Michael Elad, Algorithms for Noise Removal and the Bilateral Filter.ppt, 2002
8. Bilateral filter [Tomasi and Manduchi 1998]
Center Sample
Neighborhood
It is clear that in weighting this neighborhood, we would like to preserve the step
8/59
Michael Elad, Algorithms for Noise Removal and the Bilateral Filter.ppt, 2002
9. Bilateral filter [Tomasi and Manduchi 1998]
Ws
WR
9/59
Michael Elad, Algorithms for Noise Removal and the Bilateral Filter.ppt, 2002
10. Bilateral filter [Tomasi and Manduchi 1998]
It appears that the weight is inversely prop. to the Total-Distance (both horizontal and vertical) from the center sample.
Michael Elad, Algorithms for Noise Removal and the Bilateral Filter.ppt, 2002
11. Bilateral filter [Tomasi and Manduchi 1998]
A nonlinear filter that locally gathers information from similar pixels
Preserve edge
Noise removal
12. Bilateral filter
Decompose images into a pixelwise-smooth layer [Durand and Dorsey 02]
Realtime bilateral filter [Chen et al. 07]
Bilateral image decomposition [Durand and Dorsey 02]
Flash/no-flash enhancement [Eisemann and Durand 2004; Petschnigg et al. 04]
Tone management [Bae et al. 06]
NPR [Fattal et al. 07]
Upsampling[Kopf et al. 07].
13. Gradient based image editing
Propagate locate image editing operation according to the gradient field
14. The gradient domain provides a natural setting for image manipulation tech-
niques, including dynamic range compression [Fattal et al. 2002], seamless image
stitching [Levin et al. 2004], image editing [Perez et al. 2003], alpha matte extrac-
tion [Sun et al. 2004], and shadow removal [Finlayson et al. 2006; Xu et al. 2006].
Solving the Poisson equation amounts to performing an L2 minimization in which
the image gradients are weighted uniformly in space. More recent gradient based
methods such as colorization [Levin et al. 2004], interactive tone mapping [Lischin-
ski et al. 2006] and alpha matting [Levin et al. 2006], propagate local image editing
operations throughout the image according to the underlying gradient field. These
approaches require solving a similar optimization problem, but in this case the
output image gradients are weighted in a spatially-dependent manner.
15. Should we need a coarse representation of an image ?
Why always pixel-wise kernel of bilateral filtering ?
A sparse bilateral kernel ?
Coarse but keep edge ?
16. A new dimensionally-reduced linear image
A coarse image representation consisting of elementary basis functions derived from the bilateral filter kernels
scale-adaptive coarsened representation
detail
coarse
binding together smooth regions but also shaped by strong edges
17. 1D kernel construction steps 1D input I(x)
18. 1D kernel construction steps Grid kernels
I(y)
y
S(x,y)
y
Kernel S of bilateral filter at grid y
19. 1D kernel construction steps Grid kernels
I(y)
y
S(x,y)
y
Kernel S of bilateral filter at grid y
20. 1D kernel construction steps Grid kernels
I(y)
S(x,y)
Kernel S of bilateral filter at grid y
21. 1D kernel construction steps Grid kernels
I(y)
y
S(x,y)
y
Kernel S of bilateral filter at grid y
22. 1D kernel construction steps Grid kernels
I(y)
y
S(x,y)
y
Kernel S of bilateral filter at grid y
23. 1D kernel construction steps.
I(y)
y
S(x,y)
y
Kernel S of bilateral filter at grid y
24. 1D kernel construction steps-Center adjustment
kernel centers are shifted away from edges.
25. 1D kernel construction steps-add Islands
Island kernel are added
Island kernel is where surrounding kernel is less than
26. 1D kernel construction steps normalized grid kernel Ki(x)
C(x) : a per-pixel normalization factor
xi for i=1..n are the kernel centers
27. Normalized grid kernel Ki(x) :case C(x)=1
28. Bilateral Image Coarsening (BIG), J- spanned by Ki(x)
29. Bilateral Image Coarsening (BIG), J- spanned by Ki(x), Pi(x)
Pi(x)
30. Bilateral Image Coarsening space (BIG), J
31. Image kernels

grid kernel center

+ shifted kernel center
+ island center
non-overlapping kernel
32. Processing
Down-sample the input pixels by 2k
Construct grid kernel center S(xi,x)
Fine-tune
Local shift xi to xi where minimal |I(x)| within a window of k-2 by k-2 across xi
Add island kernel where S(xi,x) <
Normalized kernel Ki (x)
A set of construction polynomial (CPs)
Build the Bilateral Image Coarsening (BIC) space
aij : degree of freedom of CPs
33. Pixel normalized factor C

set C(x) = I(x)

34. J is a subspace surrounding the input image. 35. set C(x) = 1 36. J becomes a space of piecewise-smooth functions