a gentle introduction -...
TRANSCRIPT
![Page 1: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/1.jpg)
![Page 2: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/2.jpg)
A Gentle Introduction to Bilateral Filtering and its Applications
A Gentle Introduction to Bilateral Filtering and its Applications
07/10: Novel Variantsof the Bilateral Filter
Jack Tumblin – EECS, Northwestern University
![Page 3: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/3.jpg)
Review: Bilateral FilterReview: Bilateral Filter
A 2A 2--D filter window: weights vary with intensityD filter window: weights vary with intensity
cc
ss
DomainDomain
RangeRangef(x)f(x)
xx
2 Gaussian Weights:2 Gaussian Weights:product = product = ellisoidalellisoidal footprintfootprint
Normalize weights toNormalize weights toalways sum to 1.0always sum to 1.0
![Page 4: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/4.jpg)
Review: Review: Bilateral StrengthsBilateral StrengthsPiecewise smooth result Piecewise smooth result
–– averages local small details, ignores outliersaverages local small details, ignores outliers–– preserves steps, largepreserves steps, large--scale ramps, and curves,...scale ramps, and curves,...
•• Some equivalence to anisotropic diffusion, robust statisticsSome equivalence to anisotropic diffusion, robust statistics[Black98,Elad02,Durand02][Black98,Elad02,Durand02]
•• Simple & Fast Simple & Fast (esp. w/ (esp. w/ [Durand02][Durand02], , [Paris06][Paris06], , [Porikli08][Porikli08] other speedup methods)other speedup methods)
cc
ss
![Page 5: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/5.jpg)
Review: Bilateral FilterReview: Bilateral FilterWhy it works: graceful segmentationWhy it works: graceful segmentation•• Smoothing for Smoothing for ‘‘similarsimilar’’ parts parts ONLYONLY•• Range Gaussian Range Gaussian ss acts as a acts as a ‘‘filtered regionfiltered region’’ finderfinder DomainDomain
RangeRangef(x)f(x)
xx
cc
ss
cc
ss
![Page 6: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/6.jpg)
Bilateral FilterBilateral Filter VariantsVariants•• Before the Before the ‘‘BilateralBilateral’’ name :name :
–– YaroslavskyYaroslavsky (1985): T.D.R.I.M.(1985): T.D.R.I.M.–– Smith & Brady (1997): SUSANSmith & Brady (1997): SUSAN
And now, a growing set of extended variants:And now, a growing set of extended variants:•• ‘‘TrilateralTrilateral’’ Filter (Filter (ChoudhuryChoudhury et al., EGSR 2003)et al., EGSR 2003)•• CrossCross--Bilateral (Petschnigg04, Eisemann04) Bilateral (Petschnigg04, Eisemann04) •• NLNL--Means (Buades05)Means (Buades05)•• Bilateral Retinex(Elad05), JointBilateral Retinex(Elad05), Joint--Bilateral Bilateral
UpsamplingUpsampling (Kopf07), many more exist(Kopf07), many more exist……And many more coming: application drivenAnd many more coming: application driven……
![Page 7: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/7.jpg)
Who was first? Who was first? ManyMany PioneersPioneers•• Elegant, Simple, Broadly useful Idea Elegant, Simple, Broadly useful Idea
‘‘InventedInvented’’ several timesseveral times
•• Different Approaches, Increasing ClarityDifferent Approaches, Increasing Clarity–– Yaroslavsky(1985): Yaroslavsky(1985):
‘‘Transform Domain Image Restoration MethodsTransform Domain Image Restoration Methods’’–– Smith & Brady (1995): Smith & Brady (1995): ‘‘SUSANSUSAN’’
““SSmallest mallest UUnivaluenivalueSSegmentegment--AAssimilating ssimilating NNucleusucleus””
–– TomasiTomasi & Manduchi(1998): & Manduchi(1998): ‘‘Bilateral FilterBilateral Filter’’
![Page 8: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/8.jpg)
New Idea!New Idea!1985 1985 YaroslavskyYaroslavsky: :
A 2A 2--D filter window: D filter window: weights vary with intensity ONLYweights vary with intensity ONLY
cc
ss
DomainDomain
RangeRangef(x)f(x)
xx
Square neighborhood,Square neighborhood,Gaussian WeightedGaussian Weighted‘‘similaritysimilarity’’
Normalize weights toNormalize weights toalways sum to 1.0always sum to 1.0
![Page 9: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/9.jpg)
New Idea!New Idea!1995 Smith: 1995 Smith: ‘‘SUSANSUSAN’’ FilterFilter
A 2A 2--D filter window: weights vary with intensityD filter window: weights vary with intensity
cc
ss
DomainDomain
RangeRangef(x)f(x)
xx
2 Gaussian Weights:2 Gaussian Weights:product = product = ellisoidalellisoidal footprintfootprint
Normalize weights toNormalize weights toalways sum to 1.0always sum to 1.0
![Page 10: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/10.jpg)
Output at is Output at is average of a average of a tiny regiontiny region
Bilateral Filter:Bilateral Filter: 3 Difficulties3 Difficulties
•• Poor Smoothing in Poor Smoothing in High Gradient RegionsHigh Gradient Regions
•• Smoothes and bluntsSmoothes and bluntscliffs, valleys & ridgescliffs, valleys & ridges
•• Can combine disjoint Can combine disjoint signal regions signal regions
cc
ss
![Page 11: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/11.jpg)
Bilateral Filter:Bilateral Filter: 3 Difficulties3 Difficulties
•• Poor Smoothing in Poor Smoothing in High Gradient RegionsHigh Gradient Regions
•• Smoothes and bluntsSmoothes and bluntscliffs, valleys & ridgescliffs, valleys & ridges
•• Can combine disjoint Can combine disjoint signal regions
c
ss
signal regions
![Page 12: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/12.jpg)
Bilateral Filter:Bilateral Filter: 3 Difficulties3 Difficulties
•• Poor Smoothing in Poor Smoothing in High Gradient RegionsHigh Gradient Regions
•• Smoothes and bluntsSmoothes and bluntscliffs, valleys & ridgescliffs, valleys & ridges
•• Disjoint regions Disjoint regions can blend together can blend together
c
ss
![Page 13: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/13.jpg)
‘‘Blunted CornersBlunted Corners’’ Weak HalosWeak Halos
Bilateral :Bilateral :
What What we getwe get
![Page 14: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/14.jpg)
‘‘Blunted CornersBlunted Corners’’ Weak HalosWeak Halos
What What we wantwe want
‘‘TrilateralTrilateral’’resultresult
![Page 15: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/15.jpg)
Try to fix this:Try to fix this:Trilateral Filter (Trilateral Filter (ChoudhuryChoudhury 2003)2003)
Goal:Goal:Piecewise linear smoothing, not piecewise constantPiecewise linear smoothing, not piecewise constant
Method:Method:‘‘SteerSteer’’ Bilateral Filter with smoothed gradients Bilateral Filter with smoothed gradients
PositionPosition
IntensityIntensity
EXAMPLE:EXAMPLE: remove noise from a piecewise linear remove noise from a piecewise linear scanlinescanline
![Page 16: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/16.jpg)
Outline: BilateralOutline: Bilateral Trilateral FilterTrilateral FilterThree Key Ideas:Three Key Ideas:•• TiltTilt the filter windowthe filter window
according to bilaterallyaccording to bilaterally--smoothed gradientssmoothed gradients
•• LimitLimit the filter windowthe filter windowto connected regions to connected regions of similar smoothed gradient.of similar smoothed gradient.
•• AdjustAdjust Parameters Parameters from measurements from measurements of the windowed signalof the windowed signal
ccss
![Page 17: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/17.jpg)
Outline: BilateralOutline: Bilateral Trilateral FilterTrilateral FilterKey Ideas:Key Ideas:•• TiltTilt the filter windowthe filter window
according to bilaterallyaccording to bilaterally--smoothed gradientssmoothed gradients
•• LimitLimit the filter windowthe filter windowto connected regions to connected regions of similar smoothed gradient.of similar smoothed gradient.
•• AdjustAdjust Parameters Parameters from measurements from measurements of the windowed signalof the windowed signal
ccss
![Page 18: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/18.jpg)
Outline: BilateralOutline: Bilateral Trilateral FilterTrilateral FilterKey Ideas:Key Ideas:•• TiltTilt the filter windowthe filter window
according to bilaterallyaccording to bilaterally--smoothed gradientssmoothed gradients
•• LimitLimit the filter windowthe filter windowto connected regions to connected regions of similar smoothed gradient.of similar smoothed gradient.
•• AdjustAdjust Parameters Parameters from measurements from measurements of the windowed signalof the windowed signal
ccss
![Page 19: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/19.jpg)
....
BilateralBilateral
![Page 20: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/20.jpg)
....
TrilateralTrilateral
![Page 21: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/21.jpg)
. . •• ,,
![Page 22: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/22.jpg)
Trilateral Filter (Trilateral Filter (ChoudhuryChoudhury 2003)2003)•• StrengthsStrengths
–– Sharpens Sharpens cornerscorners–– Smoothes similar Smoothes similar gradientsgradients–– Automatic Automatic parameter parameter settingsetting–– 33--D D mesh demesh de--noisingnoising, too!, too!
•• WeaknessesWeaknesses–– SS--LL--OO--W;W; very costly connectedvery costly connected--region finderregion finder–– Shares Shares BilateralBilateral’’ss ‘‘Lonely Outlier PixelLonely Outlier Pixel’’ artifactsartifacts–– Noise ToleranceNoise Tolerance limits; disrupts limits; disrupts ‘‘tilttilt’’ estimatesestimates
![Page 23: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/23.jpg)
NEW IDEA : NEW IDEA : ‘‘JointJoint’’ or or ‘‘CrossCross’’ BilateralBilateral’’Petschnigg(2004) and Eisemann(2004)Petschnigg(2004) and Eisemann(2004)
Bilateral Bilateral two kindstwo kinds of weightsof weightsNEW :NEW : get them from get them from two kindstwo kinds of images.of images.
•• Smooth image Smooth image AA pixels locally, butpixels locally, but•• Limit to Limit to ““similar regionssimilar regions”” found in image found in image BB
Why do this?Why do this? To get To get ‘‘best of both imagesbest of both images’’
![Page 24: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/24.jpg)
OrdinaryOrdinary BilateralBilateral FilterFilterBilateral Bilateral two kindstwo kinds of weights, one of weights, one image A image A ::
( ) ( )∑∈
−−=S
AAAGGW
ABFq
qqpp
p qp ||||||1][rs σσ
cc
ss
Image A:
DomainDomain
RangeRangef(x)f(x)
xx
![Page 25: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/25.jpg)
‘‘JointJoint’’ or or ‘‘CrossCross’’ BilateralBilateral FilterFilterNEW:NEW: two kindstwo kinds of weights, of weights, twotwo imagesimages
( ) ( )∑∈
−−=S
ABBGGW
ABFq
qqpp
p qp ||||||1][rs σσ
cc
ss
A: Noisy, dim(ambient image)
cc
ss
B: Clean,strong(Flash image)
![Page 26: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/26.jpg)
Image A:Image A: Warm, shadows, but too NoisyWarm, shadows, but too Noisy(too dim for a good quick photo)(too dim for a good quick photo)
No-flash
![Page 27: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/27.jpg)
Image B:Image B: Cold, ShadowCold, Shadow--free, Cleanfree, Clean(flash: simple light, ALMOST no shadows)(flash: simple light, ALMOST no shadows)
![Page 28: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/28.jpg)
MERGE BEST OF BOTH:MERGE BEST OF BOTH: applyapply‘‘Cross BilateralCross Bilateral’’ or or ‘‘Joint BilateralJoint Bilateral’’
![Page 29: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/29.jpg)
(it really is (it really is muchmuch better!)better!)
![Page 30: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/30.jpg)
Recovers Weak Signals Hidden by NoiseRecovers Weak Signals Hidden by Noise
Noisy but StrongNoisy but Strong……
Noisy and Weak…
+ Noise =
+ Noise =
![Page 31: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/31.jpg)
Ordinary Bilateral Filter? Ordinary Bilateral Filter?
Noisy but StrongNoisy but Strong……
Noisy and Weak…
BF
BF
Step feature GONE!!Step feature GONE!!
![Page 32: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/32.jpg)
Ordinary Bilateral Ordinary Bilateral
Noisy but StrongNoisy but Strong……
Noisy and Weak…
Signal too weak to rejectSignal too weak to reject
Range filter preserves signalRange filter preserves signal
![Page 33: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/33.jpg)
‘‘CrossCross’’ or or ‘‘JointJoint’’ Bilateral Idea:Bilateral Idea:
Noisy but StrongNoisy but Strong……
Noisy and Weak…
Range filter preserves signalRange filter preserves signal
Use stronger signalUse stronger signal’’s range s range to set otherto set other’’s filter weightss filter weights……
![Page 34: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/34.jpg)
‘‘JointJoint’’ or or ‘‘CrossCross’’ BilateralBilateral FilterFilterPetschnigg(2004) and Eisemann(2004)Petschnigg(2004) and Eisemann(2004)
•• Useful Residues.Useful Residues. To transfer detailsTo transfer details,,–– CBF(A,B)CBF(A,B) to remove Ato remove A’’s noisy detailss noisy details–– CBF(B,A)CBF(B,A) to extract Bto extract B’’s clean details, and s clean details, and –– Add to Add to CBF(A,B)CBF(A,B) clean, detailed image!clean, detailed image!
••CBF(A,B)CBF(A,B):: smoothes image smoothes image AA only;only;(e.g. the (e.g. the ‘‘no flashno flash’’ image)image)
••Limits smoothing to stay within regions Limits smoothing to stay within regions where Image where Image BB is ~uniform is ~uniform (e.g. flash)(e.g. flash)
![Page 35: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/35.jpg)
New Idea:New Idea:NLNL--Means Filter (Means Filter (BuadesBuades 2005)2005)
•• Same goals: Same goals: ‘‘Smooth within Similar RegionsSmooth within Similar Regions’’
•• KEY INSIGHTKEY INSIGHT: Generalize, extend : Generalize, extend ‘‘SimilaritySimilarity’’–– Bilateral:Bilateral:
Averages neighbors with Averages neighbors with similarsimilar intensitiesintensities;;
–– NLNL--Means:Means:Averages neighbors with Averages neighbors with
similar neighborhoodssimilar neighborhoods!!
![Page 36: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/36.jpg)
NLNL--Means Method:Means Method:BuadesBuades (2005)(2005)
•• For each andFor each andevery pixel every pixel p: p:
![Page 37: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/37.jpg)
NLNL--Means Method:Means Method:BuadesBuades (2005)(2005)
•• For each andFor each andevery pixel every pixel p: p: –– Define a small, simple fixed size neighborhood;Define a small, simple fixed size neighborhood;
![Page 38: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/38.jpg)
NLNL--Means Method:Means Method:BuadesBuades (2005)(2005)
•• For each andFor each andevery pixel every pixel p: p: –– Define a small, simple fixed size neighborhood;Define a small, simple fixed size neighborhood;–– Define vector Define vector VVpp: a list of neighboring pixel : a list of neighboring pixel
values.values.
Vp = 0.740.320.410.55………
![Page 39: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/39.jpg)
NLNL--Means Method:Means Method:BuadesBuades (2005)(2005)‘‘SimilarSimilar’’ pixels pixels p, qp, q
SMALLSMALLvector distance;vector distance;
|| Vp – Vq ||2p
q
![Page 40: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/40.jpg)
NLNL--Means Method:Means Method:BuadesBuades (2005)(2005)‘‘DissimilarDissimilar’’ pixels pixels p, qp, q
LARGELARGEvector distance;vector distance;
|| Vp – Vq ||2p
q
q
![Page 41: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/41.jpg)
NLNL--Means Method:Means Method:BuadesBuades (2005)(2005)‘‘DissimilarDissimilar’’ pixels pixels p, qp, q
LARGELARGEvector distance;vector distance;
Filter with this!Filter with this!
tiny tiny distdistance ance big big wweighteightbig big distdistance ance tiny tiny wweighteight
|| Vp – Vq ||2p
q
q
wt
dist
![Page 42: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/42.jpg)
NLNL--Means Method:Means Method:BuadesBuades (2005)(2005)p, q p, q neighbors defineneighbors definea vector distance;a vector distance;
Filter with this:Filter with this:No spatial term!No spatial term!
|| Vp – Vq ||2 pq
( ) ( )∑∈
−−=S
IVVGGW
INLMFq
qqpp
p qp 2||||||||1][rs
rrσσ
![Page 43: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/43.jpg)
NLNL--Means Method:Means Method:BuadesBuades (2005)(2005)pixels pixels p, q p, q neighborsneighbors
Set a vector distance;Set a vector distance;
Vector Distance to Vector Distance to pp sets sets weight for each pixel weight for each pixel qq
|| Vp – Vq ||2 pq
( )∑∈
−=S
IVVGW
INLMFq
qqpp
p2||||1][
r
rrσ
![Page 44: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/44.jpg)
NLNL--Means Filter (Means Filter (BuadesBuades 2005)2005)•• NoisyNoisy
sourcesourceimage:image:
![Page 45: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/45.jpg)
NLNL--Means Filter (Means Filter (BuadesBuades 2005)2005)•• GaussianGaussian
FilterFilter
Low noise,Low noise,Low detailLow detail
![Page 46: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/46.jpg)
NLNL--Means Filter (Means Filter (BuadesBuades 2005)2005)•• AnisotropicAnisotropic
Diffusion:Diffusion:
(Note (Note ‘‘stairstepsstairsteps’’::~ piecewise~ piecewiseconstant)constant)
![Page 47: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/47.jpg)
NLNL--Means Filter (Means Filter (BuadesBuades 2005)2005)•• BilateralBilateral
Filter:Filter:
(better, but(better, butsimilarsimilar‘‘stairstepsstairsteps’’::
![Page 48: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/48.jpg)
NLNL--Means Filter (Means Filter (BuadesBuades 2005)2005)•• NLNL--Means:Means:
Sharp,Sharp,Low noise,Low noise,Few artifacts.Few artifacts.
![Page 49: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/49.jpg)
NonNon--Local Similarity (You, 2008)Local Similarity (You, 2008)•• BuadesBuades NL Means: vector similarity helps, NL Means: vector similarity helps,
but is only shiftbut is only shift--invariantinvariant……•• You: expand to rotation & scale invariance;You: expand to rotation & scale invariance;
exploit SIFT for similarity findingexploit SIFT for similarity finding……
![Page 50: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/50.jpg)
Weighted LeastWeighted Least--Squares OptimizationSquares Optimizationhttp://www.cs.huji.ac.il/~danix/epd/
•• Improved lowImproved low--halo detail scaleshalo detail scales……..
SIGGRAPH2008 SIGGRAPH2008 ““EdgeEdge--Preserving Decompositions for MultiPreserving Decompositions for Multi--Scale Tone and Detail ManipulationScale Tone and Detail Manipulation””Z.FarbmanZ.Farbman, R. , R. Fattal,lDFattal,lD. . LischinskiLischinski R. R. SzeliskiSzeliski
![Page 51: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/51.jpg)
Many More Possibilities: Many More Possibilities: EXPERIMENT!EXPERIMENT!
•• Bilateral goals are Bilateral goals are subjectivesubjective;;----‘‘Local smoothing within similar regionsLocal smoothing within similar regions’’----‘‘EdgeEdge--preserving smoothingpreserving smoothing’’----‘‘Separate large structure & fine detailSeparate large structure & fine detail’’----‘‘Eliminate outliersEliminate outliers’’----‘‘Filter within edges, not across themFilter within edges, not across them’’
•• ItIt’’s simplicity s simplicity invites new & inventive answers.invites new & inventive answers.
![Page 52: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/52.jpg)
![Page 53: A Gentle Introduction - Peoplepeople.csail.mit.edu/sparis/siggraph07_course/slides08/07_variants.… · A Gentle Introduction to Bilateral Filtering and its Applications 07/10: Novel](https://reader033.vdocuments.net/reader033/viewer/2022050105/5f435a8af852e419b72d996e/html5/thumbnails/53.jpg)
VariantsVariants