# diffusion-geometric maximally stable component detection in deformable shapes roee litman, alexander...

Embed Size (px)

TRANSCRIPT

For SMI

Diffusion-geometricmaximally stable component detection in deformable shapesRoee Litman, Alexander Bronstein, Michael BronsteinDiffusion-geometricmaximally stable component detection in deformable shapes

1In a nutshellMSERMaximally Stable Extremal RegionDiffusion GeometryShapeMSERThe Feature Approachfor ImagesDeformable Shape AnalysisFeature Approach in ImagesFeature based methods are the infrastructure laid in the base of many computer vision algorithms:Content-based image retrievalVideo trackingPanorama alignment3D reconstruction form stereo

3

3Problem formulation Find a semi-local feature detectorHigh repeatabilityInvariance to isometric deformationRobustness to noise, sampling, etc.Add discriminative descriptor

ResultsThe whatVisual Example6

Visual Example7

More Results

8(Taken from the TOSCA dataset)

Horse regions + Human regions- part human, part horse- partial matching- how to match?8Region Matching

Query1st, 2nd, 4th, 10th, and 15th matches9-now let's test it on real data

93D Human Scans 10Taken from the SCAPE dataset

Scanned Region MatchingQuery1st, 2nd, 4th, 10th, and 15th matches11Volume vs. Surface

12Volume & surface isometryBoundary isometryOriginalVolumetric ShapesUsually shapes are modeled as 2D boundary of a 3D shape.Volumetric shape model better captures "natural" behavior of non-rigid deformations.(Raviv et-al)Diffusion geometry terms can easily be applied to volumes2D Meshes can be voxelized13Volumetric Regions14Taken from the SCAPE dataset

Methodology(The how)Original MSER (Matas et-al)

Show some epipolar lines :)16MSERPopular image blob detectorNear-linear complexity:High repeatability [Mikolajczyk et al. 05]Robust to affine transformation and illumination changes

17MSER In a nutshellThreshold image at consecutive gray-levelsSearch regions whose area stay nearly the same through a wide range of thresholds

Efficient detection of maximally stable regions requires construction of a component tree18MSER In a nutshell

19

SKIP?19Algorithm overview Algorithm overview Represent as weighted graphComponent treeStable component detectionAlgorithm overview Represent as weighted graphImage as weighted graphAn undirected graph can be created from an image, where:Vertices are pixelsEdges by adjacency rule, e.g. 4-neiborhood23Weighting the graphIn imagesGray-scale as vertex-weightColor as edge-weight [Forssen]

In ShapesCurvature (not deformation invariant)Diffusion Geometry

Weighting OptionFor every point on the shape:Calculate the prob. of a random walk to return to the same point.Similar to Gaussian curvatureIntrinsic, i.e. deformation invariantWeight exampleColor-mappedLevel-set animation

26

Diffusion GeometryAnalysis of diffusion (random walk) processesGoverned by the heat equation

Solution is heat distributionat point at time

27

Heat-KernelGivenInitial condition Boundary condition, if theses a boundary Solve using:

i.e. - find the heat-kernel 28

The probability density for a transitionby random walk of length ,from to

Probabilistic Interpretation

29Spectral InterpretationHow to calculate ?Heat kernel can be calculated directly from eigen-decomposition of the Laplacain

By spectral decomposition theorem:

30

Laplace-Beltrami Eigenfunctions

Deformation Invariance

Computational aspectsShapes are discretized as triangular meshesCan be expressed as undirected graphHeat kernel & eigenfunctions are vectorsDiscrete Laplace-Beltrami operator

Several weight schemes for is usually discrete area elements

33Computational aspectsIn matrix notation

Solve eigendecomposition problem

34Scale SpaceThe time parameter of the heat kernel spans different scales of transition length is not invariant to shapes scaleCommute-time kernel - scale invariant

Probability of a transition by random walk of any length

35Auto-diffusivitySpecial case - The chance of returning to after time Related to Gaussian curvature by

Now we can attach scalar value to shapes!

36Weight exampleColor-mappedLevel-set animation

37

Algorithm overview Represent as weighted graphComponent treeStable component detectionThe Component TreeTree construction is a pre-process of stable region detection Contains level-set hierarchy,i.e. nesting relations. Constructed based on a weighted graph (vertex- or edge-weight)Trees nodes are level-sets(of the graphs cross-sections)39Tree Example

Preliminaries - 1/3We focus on the undirected graph with the vertex set and edge set , denotedVertices and are adjacent if The ordered sequence is a path if every consecutive pair is adjacent and are linkedA graph is connected if every vertex pair is linked

41Preliminaries - 2/3A graph is called a sub-graph of Given , we take only verticesbelonging to an edge

A graph is a (connected) component of if it is a maximal connected sub-graph

42Preliminaries - 3/3Vertex-weighted graph - Edge-weighted graph -

A cross-section is a sub-graph of a weighted graph, with all weights Level-set is a (connected) component of a cross-sectionAltitude of a level-set is the maximal weight it contains (can be smaller than )

43vertex weight are scalar only edge weight are more general43Graphic ExampleA graphEdge-weighted7 Cross-Section5 Cross-sectionTwo 5 level-sets(with altitude 4)Every level-set hasSize (area)Altitude (maximal weight)1478984144Tree ConstructionIterate over vertices by order of weightCreate a new component from vertexIf vertex is adjacent to existing component(s) add exiting components vertices to new oneStore components area & weight

4514789841Tree Construction4614789841

Algorithm overview Represent as weighted graphComponent treeStable component detectionDetection ProcessFor every leaf component in the tree:Climb the tree to its root, creating the sequence:Calculate component stability

Local maxima of the sequenceare Maximally stable components

48PerformanceThe DetilsBenchmarking The MethodMethod was tested on SHREC 2010 data-set:3 basic shapes (human, dog & horse)9 transformations, applied in 5 different strengths138 shapes in total

50

OriginalDeformationNoiseScaleHolesResults

51Quantitative ResultsVertex-wise correspondences were givenRegions were projected onto another shape, and overlap ratio was measuredOverlap ratio between a region and its projected counterpart is

Repeatability is the percent of regions with overlap above a threshold

52

Repeatability5365% at 0.75ConclusionStable region detector for deformable shapesGeneric detection framework:Vertex- and edge-weighted graph representationWorks on surface and/or volume dataPartial matching & retrieval potentialTested quantitatively (on SHREC10)

Thank YouAny Questions?