learning to detect natural image boundaries using local brightness, color and texture cues by david...
Post on 22-Dec-2015
220 views
TRANSCRIPT
![Page 1: Learning to Detect Natural Image Boundaries Using Local Brightness, Color and Texture Cues by David R. Martin, Charless C. Fowlkes, Jitendra Malik Heather](https://reader038.vdocuments.net/reader038/viewer/2022103022/56649d7f5503460f94a6234d/html5/thumbnails/1.jpg)
Learning to Detect Natural Image Boundaries Using Local
Brightness, Color and Texture Cues
by David R. Martin, Charless C. Fowlkes, Jitendra Malik
Heather Dunlop16-721: Advanced Perception
January 25, 2006
![Page 2: Learning to Detect Natural Image Boundaries Using Local Brightness, Color and Texture Cues by David R. Martin, Charless C. Fowlkes, Jitendra Malik Heather](https://reader038.vdocuments.net/reader038/viewer/2022103022/56649d7f5503460f94a6234d/html5/thumbnails/2.jpg)
What is a Boundary?
CannyMartin,
2002
Human
![Page 3: Learning to Detect Natural Image Boundaries Using Local Brightness, Color and Texture Cues by David R. Martin, Charless C. Fowlkes, Jitendra Malik Heather](https://reader038.vdocuments.net/reader038/viewer/2022103022/56649d7f5503460f94a6234d/html5/thumbnails/3.jpg)
Dataset“You will be presented a photographic image. Divide the image into some number of segments, where the segments represent ‘things’ or ‘parts of things’ in the scene. The number of segments is up to you, as it depends on the image. Something between 2 and 30 is likely to be appropriate. It is important that all of the segments have approximately equal importance.”
![Page 4: Learning to Detect Natural Image Boundaries Using Local Brightness, Color and Texture Cues by David R. Martin, Charless C. Fowlkes, Jitendra Malik Heather](https://reader038.vdocuments.net/reader038/viewer/2022103022/56649d7f5503460f94a6234d/html5/thumbnails/4.jpg)
DatasetDatabase of over 1000 images and 5-10 segmentations for each
Martin, 2002
![Page 5: Learning to Detect Natural Image Boundaries Using Local Brightness, Color and Texture Cues by David R. Martin, Charless C. Fowlkes, Jitendra Malik Heather](https://reader038.vdocuments.net/reader038/viewer/2022103022/56649d7f5503460f94a6234d/html5/thumbnails/5.jpg)
Boundaries
Intensity
Texture
Brightness
Color
Non-boundaries Boundaries
Martin, 2002
![Page 6: Learning to Detect Natural Image Boundaries Using Local Brightness, Color and Texture Cues by David R. Martin, Charless C. Fowlkes, Jitendra Malik Heather](https://reader038.vdocuments.net/reader038/viewer/2022103022/56649d7f5503460f94a6234d/html5/thumbnails/6.jpg)
Method
ImageOptimized Cues
Boundary Strength
Brightness
Color
Texture
Benchmark
Human Segmentations
Cue Combination
Model
Martin, 2002
Goal: learn the probability of a boundary, Pb(x,y,θ)
![Page 7: Learning to Detect Natural Image Boundaries Using Local Brightness, Color and Texture Cues by David R. Martin, Charless C. Fowlkes, Jitendra Malik Heather](https://reader038.vdocuments.net/reader038/viewer/2022103022/56649d7f5503460f94a6234d/html5/thumbnails/7.jpg)
Image FeaturesCIE L*a*b* color space (luminance, red-green, yellow-blue)Oriented Energy:
fe: Gaussian second derivativefo: Its Hilbert transform
BrightnessL* distribution
Colora* and b* distributions (joint or marginal)
Texture
2,
2
,,oe fIfIOE
![Page 8: Learning to Detect Natural Image Boundaries Using Local Brightness, Color and Texture Cues by David R. Martin, Charless C. Fowlkes, Jitendra Malik Heather](https://reader038.vdocuments.net/reader038/viewer/2022103022/56649d7f5503460f94a6234d/html5/thumbnails/8.jpg)
TextureConvolve with a filter bank:
Gaussian second derivativeIts Hilbert transformDifference of Gaussians
Filter responses give a measure of texture
![Page 9: Learning to Detect Natural Image Boundaries Using Local Brightness, Color and Texture Cues by David R. Martin, Charless C. Fowlkes, Jitendra Malik Heather](https://reader038.vdocuments.net/reader038/viewer/2022103022/56649d7f5503460f94a6234d/html5/thumbnails/9.jpg)
Other Filter BanksLeung-Malik filter set: Schmid filter set:
Maximum Response 8 filter set:
![Page 10: Learning to Detect Natural Image Boundaries Using Local Brightness, Color and Texture Cues by David R. Martin, Charless C. Fowlkes, Jitendra Malik Heather](https://reader038.vdocuments.net/reader038/viewer/2022103022/56649d7f5503460f94a6234d/html5/thumbnails/10.jpg)
TextonsConvolve image with filter bankCluster filter responses to form textons
Adapted from Martin, 2002 and Varma, Zisserman, 2005
![Page 11: Learning to Detect Natural Image Boundaries Using Local Brightness, Color and Texture Cues by David R. Martin, Charless C. Fowlkes, Jitendra Malik Heather](https://reader038.vdocuments.net/reader038/viewer/2022103022/56649d7f5503460f94a6234d/html5/thumbnails/11.jpg)
Texton DistributionAssign each pixel to nearest textonForm distribution of textons
Adapted from Martin, 2002 and Varma, Zisserman, 2005
![Page 12: Learning to Detect Natural Image Boundaries Using Local Brightness, Color and Texture Cues by David R. Martin, Charless C. Fowlkes, Jitendra Malik Heather](https://reader038.vdocuments.net/reader038/viewer/2022103022/56649d7f5503460f94a6234d/html5/thumbnails/12.jpg)
Gradient-based FeaturesBrightness (BG), color (CG), texture (TG) gradientsHalf-disc regions described by histogramsCompare distributions with χ2 statistic
r(x,y)
i ii
ii
hg
hghg
22 )(
2
1),(
![Page 13: Learning to Detect Natural Image Boundaries Using Local Brightness, Color and Texture Cues by David R. Martin, Charless C. Fowlkes, Jitendra Malik Heather](https://reader038.vdocuments.net/reader038/viewer/2022103022/56649d7f5503460f94a6234d/html5/thumbnails/13.jpg)
Texture GradientTexton distribution in two half circles
Martin, 2002
![Page 14: Learning to Detect Natural Image Boundaries Using Local Brightness, Color and Texture Cues by David R. Martin, Charless C. Fowlkes, Jitendra Malik Heather](https://reader038.vdocuments.net/reader038/viewer/2022103022/56649d7f5503460f94a6234d/html5/thumbnails/14.jpg)
LocalizationTightly localize boundariesReduce noiseCoalesce double detectionsImprove OE and TG features
OE
TG localized
OE localized
TG
Martin, Fowlkes, Malik, 2004
![Page 15: Learning to Detect Natural Image Boundaries Using Local Brightness, Color and Texture Cues by David R. Martin, Charless C. Fowlkes, Jitendra Malik Heather](https://reader038.vdocuments.net/reader038/viewer/2022103022/56649d7f5503460f94a6234d/html5/thumbnails/15.jpg)
OptimizationTexture parameters:
type of filter bankscale of filtersnumber of textonsuniversal or image-specific textons
Other possible distance/histogram comparison metricsNumber of bins for histogramsScale parameter for all cues
![Page 16: Learning to Detect Natural Image Boundaries Using Local Brightness, Color and Texture Cues by David R. Martin, Charless C. Fowlkes, Jitendra Malik Heather](https://reader038.vdocuments.net/reader038/viewer/2022103022/56649d7f5503460f94a6234d/html5/thumbnails/16.jpg)
Evaluation MethodologyPosterior probability of boundary: Pb(x,y,θ)
Evaluation measure: precision recall curveF-measure: Martin,
2002
5.0
)1(
PRPRF
![Page 17: Learning to Detect Natural Image Boundaries Using Local Brightness, Color and Texture Cues by David R. Martin, Charless C. Fowlkes, Jitendra Malik Heather](https://reader038.vdocuments.net/reader038/viewer/2022103022/56649d7f5503460f94a6234d/html5/thumbnails/17.jpg)
Cue CombinationWhich cues should be used?
OE is redundant when other cues are presentBG+CG+TG produces best results
Martin, 2002
![Page 18: Learning to Detect Natural Image Boundaries Using Local Brightness, Color and Texture Cues by David R. Martin, Charless C. Fowlkes, Jitendra Malik Heather](https://reader038.vdocuments.net/reader038/viewer/2022103022/56649d7f5503460f94a6234d/html5/thumbnails/18.jpg)
ClassifiersUntil now, only logistic regression was usedOther possible classifiers:
Density estimationClassification treesHierarchical mixtures of expertsSupport vector machines Martin,
2002
![Page 19: Learning to Detect Natural Image Boundaries Using Local Brightness, Color and Texture Cues by David R. Martin, Charless C. Fowlkes, Jitendra Malik Heather](https://reader038.vdocuments.net/reader038/viewer/2022103022/56649d7f5503460f94a6234d/html5/thumbnails/19.jpg)
Result ComparisonAlternative methods:
Matlab’s Canny edge detector with and without hysteresisSpatially-averaged second moment matrix (2MM) Martin,
2002
![Page 20: Learning to Detect Natural Image Boundaries Using Local Brightness, Color and Texture Cues by David R. Martin, Charless C. Fowlkes, Jitendra Malik Heather](https://reader038.vdocuments.net/reader038/viewer/2022103022/56649d7f5503460f94a6234d/html5/thumbnails/20.jpg)
ResultsCanny 2MM BG+CG+TG HumanImage
Martin, 2002
![Page 21: Learning to Detect Natural Image Boundaries Using Local Brightness, Color and Texture Cues by David R. Martin, Charless C. Fowlkes, Jitendra Malik Heather](https://reader038.vdocuments.net/reader038/viewer/2022103022/56649d7f5503460f94a6234d/html5/thumbnails/21.jpg)
Results
Martin, 2002
Canny 2MM BG+CG+TG HumanImage
![Page 22: Learning to Detect Natural Image Boundaries Using Local Brightness, Color and Texture Cues by David R. Martin, Charless C. Fowlkes, Jitendra Malik Heather](https://reader038.vdocuments.net/reader038/viewer/2022103022/56649d7f5503460f94a6234d/html5/thumbnails/22.jpg)
Results
Martin, 2002
Canny 2MM BG+CG+TG HumanImage
![Page 23: Learning to Detect Natural Image Boundaries Using Local Brightness, Color and Texture Cues by David R. Martin, Charless C. Fowlkes, Jitendra Malik Heather](https://reader038.vdocuments.net/reader038/viewer/2022103022/56649d7f5503460f94a6234d/html5/thumbnails/23.jpg)
ConclusionsLarge data set used for testingTexture gradients are a powerful cueSimple linear model sufficient for cue combinationOutperforms existing methodsAn approach that is useful for higher-level algorithmsCode is available online:http://www.eecs.berkeley.edu/Research/Projects/CS/vision/grouping/segbench/