shape representation and similarity

25
Computer Vision November 2003 L1.1 © 2003 by Davi Geiger Shape Representation and Similarity Occlusion Articulation Shape similarities should be preserved under occlusions and articulation. Two equal stretching along contours can affect shape similarity differently. Shapes can be represented by contours, or by a set of interior points, or other ways. What is the shape representation that can be best used for shape similarity? Stretching Stretching

Upload: dinh

Post on 18-Feb-2016

36 views

Category:

Documents


4 download

DESCRIPTION

Stretching. Stretching. Shape Representation and Similarity. Occlusion. Articulation. Shape similarities should be preserved under occlusions and articulation. Two equal stretching along contours can affect shape similarity differently. . - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Shape Representation and Similarity

Computer Vision November 2003 L1.1© 2003 by Davi Geiger

Shape Representation and SimilarityOcclusion Articulation

Shape similarities should be preserved under occlusions and articulation.

Two equal stretching along contours can affect shape similarity differently.

Shapes can be represented by contours, or by a set of interior points, or other ways. What is the shape representation that can be best used for shape similarity?

Stretching Stretching

Page 2: Shape Representation and Similarity

Computer Vision November 2003 L1.2© 2003 by Davi Geiger

Shape Axis (SA)

SA-Tree

Shape

Symmetry Axis Representation

Page 3: Shape Representation and Similarity

Computer Vision November 2003 L1.3© 2003 by Davi Geiger

A variational shape representation model based on self-

similarity of shapes.

For each shape contour, first compute its shape axis then derive a unique shape-axis-tree (SA-tree) or shape-axis-forest (SA-forest) representation.

Shape Axis (SA) SA-TreeShape Contour

Shape Representation via Self-SimilarityBased on work of Liu, Kohn and Geiger

Page 4: Shape Representation and Similarity

Computer Vision November 2003 L1.4© 2003 by Davi Geiger

Use Use twotwo different parameterizations to compute the different parameterizations to compute the represention of a shape.represention of a shape.

Construct a cost functional to measure the Construct a cost functional to measure the goodnessgoodness of a of a match between the two parameterizations.match between the two parameterizations.

The cost functional is decided by the self-similarity criteria The cost functional is decided by the self-similarity criteria of of symmetrysymmetry..

The Insight

counterclockwise clockwise

Page 5: Shape Representation and Similarity

Computer Vision November 2003 L1.5© 2003 by Davi Geiger

Two different parameterizations:Two different parameterizations:

counterclockwisecounterclockwise

clockwiseclockwise

When the curve is closed we haveWhen the curve is closed we have

Matching the curves is described as the match of functionsMatching the curves is described as the match of functions

Parameterized Shapes

}10:)({1 ssx}10:)(~{2 ttx )1()(~: txtxnote

(1)x~(0)x~ and )1()0( xx

))((x~ ))(x(or[0,1];)()( tsts

Page 6: Shape Representation and Similarity

Computer Vision November 2003 L1.6© 2003 by Davi Geiger

A Global Optimization Approach We seek “good” matches over the possible We seek “good” matches over the possible

correspondencescorrespondences

[0,1]where))((x~ ))(x(or)()( tsts

)1()(~: txtxnote

Page 7: Shape Representation and Similarity

Computer Vision November 2003 L1.7© 2003 by Davi Geiger

Co-Circularity

Mirror Symmetry

Similarity criteria: Symmetry

)('))((~)()(~))((~

)())(()()())((where

)())((~))(())((~)((

)())((~))(())((~)((

of valuessmall give shoulderror y thisIntuitivelror.measure/ersymmetry

a define nowcan but we symmetry,perfect havelonger nomay we~ tangent extra the

Given y.circularit-osymmetry/cmirror defines

~point another and, tangent its ,point One

ttd

dtdt

txddtxd

ssd

dsds

sdxdsdx

bdtxd

dsdxtxsx

adtxd

dsdxtxsx

xx

)(~ t

)(~ tx)(~ t

)(~ t

)(~ tx)(~ t

)(sx)(s

)(~ tx

)(sx)(s

)(~ tx

Page 8: Shape Representation and Similarity

Computer Vision November 2003 L1.8© 2003 by Davi Geiger

nt vectorsunit tange oriented are))((~and))((where),()( escorresponc possible allover

)(),(,))((~)),((x~)),(()),(x( ))(t,)E(s(

Minimize1

0

tstσs

dtsttssF

A Global Optimization Approach

Page 9: Shape Representation and Similarity

Computer Vision November 2003 L1.9© 2003 by Davi Geiger

Constraints on the form of F

).( and)( maps of choice on thenot and)()(match on theonly dependscost that theassuring

~)~(),~(,~,x~,,x

~,~,~,x~,,x~~,,~,x~,,x

)(),(,~,x~,,x

)(~ :ationparametriz of changeany under invarianceobtain To)(),(,~,x~,,x)(),(,~,x~,,x Condition Scaling 2.

roles equivalentplay ~and i.e.,

)(),(,,x,~,x~)(),(,~,x~,,xmatch theofSymmetry 1.

1

0~

1

0

1

0

1

0

tsts

dtsF

ddd

ddt

dd

ddsFd

dd

ddt

ddsF

dtsF

tsFtsF

stFtsF

Page 10: Shape Representation and Similarity

Computer Vision November 2003 L1.10© 2003 by Davi Geiger

Constraints on the form of F (cont.)

. )(amount known andt independen shape aby cost thechange and matching thepreserve will

factorcommon by the ~ sand scaling Thus, ).(function somefor

)(),(,,~,x~,,x)()(),(,~,x~,,x Invariance Scaling 4.

cost affect thenot willations transformrigid that so )(),(,,~,x~,,x)(),(,~,x~,,x

InvarianceRotation 3.

g

g

tsFgtsFPartial

tsFtsRRRRF

Page 11: Shape Representation and Similarity

Computer Vision November 2003 L1.11© 2003 by Davi Geiger

1

02

2

1

0

1

0)(

)(')()('))((~)())(())((~))((

)(')()('))((~)())(())((~)((

)(),(,~,x~,,x

as defined is encecorrespondany for cost symmetry The).)( giveslly automatica(which )( mappings possible allover

cost theminize we)2constraint()(zation parameteriarbitrary any for where

)(),(,~,x~,,xmin)~,(

as measuresymmetry thewrite We

d

tsttsstxsx

tsttsstxsx

dtsF

stts

dtsFS t

Similarity criteria: Symmetry

Page 12: Shape Representation and Similarity

Computer Vision November 2003 L1.12© 2003 by Davi Geiger

Similarity criteria: Symmetry II

(a)

(b)

dFstE JumpCost )()similarity-self())(),((1

0

solution theof cost) (jump nsbifurcatiofor cost a add we(d), and (c) see parts,object for nsbifurcatio allow andty irregulari shape todues

nbifurcatio ofcreation theavoid To small. be should parameter The

)(')()(')(

))((~))((

)(')()('))((~)())(())((~))((

)(')()('))((~)())(())((~)((

)(),(,~,x~,,x

solutions (b)over (a) bias topoints, closedbetween encescorrespond bias to term thirda add We

1

02

2

2

2

1

0

c

d

tsts

txsxc

tsttsstxsx

tsttsstxsx

dtsF

(c)

(d)

Page 13: Shape Representation and Similarity

Computer Vision November 2003 L1.13© 2003 by Davi Geiger

Structural propertiesStructural propertieso SymmetricSymmetrico Parameterization IndependentParameterization Independent

Geometrical propertiesGeometrical propertieso Translation invariantTranslation invarianto Rotation invariantRotation invarianto Scale “almost” invarianceScale “almost” invariance

Self-Similarity propertiesSelf-Similarity properties

Summary: Cost Functional/Energy Density

Page 14: Shape Representation and Similarity

Computer Vision November 2003 L1.14© 2003 by Davi Geiger

A Dynamic Programming Solution

Page 15: Shape Representation and Similarity

Computer Vision November 2003 L1.15© 2003 by Davi Geiger

A Dynamic Programming

SolutionThe recurrence is on the “square boxes”. Each box is either

linked to one inside square box or linked to

a pair of inside and well aligned square

boxes (leading to bifurcations).

Assumption, that will not be needed later: The final state is a

match (0,0).

…?

!!

!!

!!

bifurcation

bifurcationt

s

1-s

1-t

tt

1-tts+1/N

t+1/N

!!

! !! !

Page 16: Shape Representation and Similarity

Computer Vision November 2003 L1.16© 2003 by Davi Geiger

)],(),1[(tt],1BestCost[smin

)],()1,[(]1ts,BestCost[smin

Jumpt]tt,-BestCost[1tt],BestCost[smin

min

t],BestCost[s

}11,...,1{tt

}11,...,1{ss

}1,11,...,1,{

tsttN

sFN

tsN

tssFN

Ns

Nt

Nt

Ns

sN

sN

tttt

t

1-s

tt

t+1/N …?

!!

!!

!!!

s 1-t1-tts+1/N

A Dynamic Programming Solution

s

t

Page 17: Shape Representation and Similarity

Computer Vision November 2003 L1.17© 2003 by Davi Geiger

A Dynamic Programming Solution

A few more steps …

iteration direction

s

t

Starting states

(matching candidates).

All corresponding to

“self-matches”

Page 18: Shape Representation and Similarity

Computer Vision November 2003 L1.18© 2003 by Davi Geiger

A General Dynamic Programming Solution

No need to force a match at (0,0) or any particular node/match.

s

t

Extended Graph

Starting states

(matching candidates).

All corresponding to

“self-matches”

Ending states

Extend the parameters s and t such that negative values x s,t=1+x.

x

x

Page 19: Shape Representation and Similarity

Computer Vision November 2003 L1.19© 2003 by Davi Geiger

A Dynamic Programming Solution

Page 20: Shape Representation and Similarity

Computer Vision November 2003 L1.20© 2003 by Davi Geiger

Experimental Results for Closed Shapes

Shape Axis

SA-Tree

Shape

Page 21: Shape Representation and Similarity

Computer Vision November 2003 L1.21© 2003 by Davi Geiger

Experimental Results for Open Shapes(the first and last points are assumed to be a match)

Shape Axis (SA)

SA-Tree

Shape

Page 22: Shape Representation and Similarity

Computer Vision November 2003 L1.22© 2003 by Davi Geiger

Shape Axis (SA)

SA-Forest

Experimental Results for Open Shapes

Page 23: Shape Representation and Similarity

Computer Vision November 2003 L1.23© 2003 by Davi Geiger

Matching Trees with deletions and merges for articulation and occlusions

Page 24: Shape Representation and Similarity

Computer Vision November 2003 L1.24© 2003 by Davi Geiger

Experimental Results for Open Shapes

Page 25: Shape Representation and Similarity

Computer Vision November 2003 L1.25© 2003 by Davi Geiger

Convexity v.s. Symmetry

White Convex Region Black Convex Region