Download - Pyramids - cs.cornell.edu
Pyramids
Gaussianpre-filtering• Solution:filter
theimage,thensubsample
blur
F0 H*
subsample blur subsample …F1
F1 H*
F2F0
blur
F0 H*
subsample blur subsample …F1
F1 H*
F2F0
{Gaussianpyramid
Gaussianpyramids[BurtandAdelson,1983]
• Incomputergraphics,amip map[Williams,1983]
GaussianPyramidshaveallsortsofapplicationsincomputervision
Source:S.Seitz
Gaussianpyramids- Searchingoverscales
Gaussianpyramids- Searchingoverscales
2)*( 23 ¯= gaussianGG
1G
The Gaussian Pyramid
High resolution
Low resolution
Image=0G
2)*( 01 ¯= gaussianGG
2)*( 12 ¯= gaussianGG
2)*( 34 ¯= gaussianGG
blur
blur
blur
blur
Gaussianpyramidandstack
Source: Forsyth
MemoryUsage• Whatisthesizeofthepyramid?
9
Laplacianpyramid
Re-duce
=--
-
=
=
=-
Laplacianpyramid
L3 =G3 - expand(G4)=L2 =G2 - expand(G3)=
L1 =G1 - expand(G2)=
L0 =G0 - expand(G1)=
L4 =G4 =
ReconstructingtheimagefromaLaplacianpyramid
=++
+
=
=
=+
Laplacian pyramid
Source: Forsyth
Edgedetection
Whyedges?
• Resiliencetolightingandcolor• usefulforrecognition,matchingpatchesacrossimages
Whyedges?
• Humansaresensitivetoedges• Converta2Dimageintoasetofcurves
– Extractssalientfeaturesofthescene, morecompact
Whyedges?
• Cuetoshapeandgeometry• usefulforrecognition,understanding3Dstructure
Credit:Jitendra Malik
Credit:Attneave
Whyedges?
• Groupingpixelsintoobjects(“perceptualorganization”)
Thislecture
• Edgedetectioningeneral• Edgedetectionforgrouping
Edges
• Edgesarecurvesintheimage,acrosswhichthebrightnesschanges“alot”
• Corners/Junctions
Aside
Closeupofedges
Source:D.Hoiem
Closeup ofedges
Source:D.Hoiem
Closeup ofedges
Source:D.Hoiem
Closeup ofedges
Source:D.Hoiem
Characterizingedges•Anedgeisaplaceofrapidchangeintheimageintensityfunction
imageintensityfunction
(alonghorizontalscanline) firstderivative
edgescorrespondtoextremaofderivativeSource:L.Lazebnik
Intensityprofile
Source:D.Hoiem
Derivativesandconvolution
• Differentiationislinear
• Differentiationisshift-invariant• Derivativeofshiftedsignalisshiftedderivative
• Hence,differentiationcanberepresentedasconvolution!
@(af(x) + bg(x))
@x
= a
@f(x)
@x
+ b
@g(x)
@x
• Howcanwedifferentiateadigital imageF[x,y]?– Option1:reconstructacontinuousimage,f, thencomputethederivative
– Option2:takediscretederivative(finitedifference)
1 -1
Howwouldyouimplementthisasalinearfilter?
Imagederivatives
-1
1: :
Source:S.Seitz
Thegradientpointsinthedirectionofmostrapidincreaseinintensity
Imagegradient• Thegradient ofanimage:
Theedgestrength isgivenbythegradientmagnitude:
Thegradientdirectionisgivenby:
• howdoesthisrelatetothedirectionoftheedge?Source:SteveSeitz
Imagegradient
Source:L.Lazebnik
WithalittleGaussiannoise
Gradient
Source:D.Hoiem
Effectsofnoise
Whereistheedge?Source:S.Seitz
Noisyinputimage
Effectsofnoise
• Noiseishighfrequency• Differentiationaccentuatesnoise
d sin!x
dx
= ! cos!x
Solution:smoothfirst
f
h
f * h
Source:S.SeitzTofindedges,lookforpeaksin
•Differentiationisaconvolution• Convolutionisassociative:• Thissavesusoneoperation:
Associativepropertyofconvolution
f
Source:S.Seitz
2Dedgedetectionfilters
GaussianderivativeofGaussian(x)
DerivativeofGaussianfilter
x-direction y-direction
TwoDimensionalGaussian
OrientedGaussianFirstandSecondDerivatives
Grouping
Whatisgrouping?
Whygrouping?
• Pixelspropertyofsensor,notworld• Reasoningatobjectlevel(might)makethingseasy:
• objectsatconsistentdepth• objectscanberecognized• objectsmoveasone
"I stand at the window and see a house, trees, sky. Theoretically I might say there were 327 brightnesses and nuances of colour. Do I have "327"? No. I have sky, house, and trees."Max Wertheimer
RegionsBoundaries
Isgroupingwell-defined?
• Dependsonpurpose• Objectparts• Backgroundsegmentation
A
B C
Isgroupingwell-defined?
A
B C
Image
BG L-bird R-bird
grass bush
headeye
beakfar body
headeye
beak body
Perceptualorganizationformsatree:
BA
C
D.Martin,C.Fowlkes,D.Tal,J.Malik."ADatabaseofHumanSegmentedNaturalImagesanditsApplicationtoEvaluatingSegmentationAlgorithmsandMeasuringEcologicalStatistics",ICCV,2001
Howdowegroupthings?
• Gestalt principles• Principleofproximity
https://courses.lumenlearning.com/wsu-sandbox/chapter/gestalt-principles-of-perception/
Howdowegroupthings?
• Gestaltprinciples• Principleofsimilarity
https://courses.lumenlearning.com/wsu-sandbox/chapter/gestalt-principles-of-perception/
Howdowegroupthings?
• Gestaltprinciples• Principleofcontinuityandclosure
https://courses.lumenlearning.com/wsu-sandbox/chapter/gestalt-principles-of-perception/
Howdowegroupthings?
• Gestaltprinciples• Principleofcommonfate
Gestaltprinciplesinthecontextofimages• Principleofproximity:nearbypixelsarepartofthesameobject
• Principleofsimilarity:similarpixelsarepartofthesameobject
• Lookfordifferencesincolor,intensity,ortextureacrosstheboundary
• Principleofclosureandcontinuity:contoursarelikelytocontinue
• High-levelknowledge?
RegionsBoundaries
Designingagoodboundarydetector• Differencesincolor,intensity,ortextureacrosstheboundary
• Continuityandclosure• High-levelknowledge
Criteriaforagoodboundarydetector
•Criteriaforagoodboundarydetector:• Gooddetection: Fireonlyonrealedges,notanywhereelse• Goodlocalization
• theedgesdetectedmustbeascloseaspossibletothetrueedges
• thedetectormustreturnonepointonlyforeachtrueedgepoint
Source: L. Fei-Fei
Cannyedgedetector•Theclassicedgedetector•Baselineforalllaterworkongrouping• Theoreticalmodel:step-edgescorruptedbyadditiveGaussiannoise
J.Canny,AComputationalApproachToEdgeDetection,IEEETrans.PatternAnalysisandMachineIntelligence,8:679-714,1986.
Source: L. Fei-Fei22,000 citations!
Example
originalimage
ComputeGradients(DoG)
X-Derivative of Gaussian Y-Derivative of Gaussian
Gradientmagnitudeandorientation• Orientationisundefinedatpixelswith0gradient
Orientationtheta = numpy.arctan2(gy, gx)
Magnitude
Non-maximumsuppressionforeachorientation
Atq,wehaveamaximumifthevalueislargerthanthoseatbothpandatr.Interpolatetogetthesevalues.
Source: D. Forsyth
BeforeNon-maxSuppression
AfterNon-maxSuppression
Hysteresisthresholding• Thresholdatlow/highlevelstogetweak/strongedgepixels• Doconnectedcomponents,startingfromstrongedgepixels
FinalCannyEdges
Cannyedgedetector
1. Filterimagewithx,yderivativesofGaussian2. Findmagnitudeandorientationofgradient3. Non-maximumsuppression:
• Thinmulti-pixelwide“ridges”downtosinglepixelwidth
4. Thresholding andlinking(hysteresis):• Definetwothresholds:lowandhigh• Usethehighthresholdtostartedgecurvesandthelow
thresholdtocontinuethem
Source: D. Lowe, L. Fei-Fei
DoesCannyalwayswork?
Thechallengesofedgedetection
• Texture• Low-contrastboundaries