An Optimization Approach to Improving Collections of Shape Maps

Andy Nguyen, Mirela Ben-Chen,Katarzyna Welnicka, Yinyu Ye, Leonidas Guibas

Computer Science Dept.Stanford University

Introduction

2

Introduction

3

IntroductionQuality of maps is a property of the collection

4

IntroductionCorollary: Individual maps cannot be evaluated in isolation

5

Problem Statement

InputA collection of related shapes

A collection of maps between all pairs of shapes

A distance measure on each shape

6

Problem StatementOutput: A collection of maps between pairs of shapes that is

Accurate

Consistent

7

Graph Representation

8

Approach

Cycle consistency tells us something about accuracy

Remove the inaccuracies we find

Repeat using the better collection

9

Related WorkLearning Shape Metrics based on Deformations and Transport, Charpiat, NORDIA09

Dynamic time warping + matching energy for good shortest paths

Disambiguating Visual Relations Using Loop Constraints, Zach et al, CVPR2010

Use cycle consistency to remove incorrect correspondences

Pairwise mapping methodsMöbius Voting for Surface Correspondence, Lipman et al, SIGGRAPH 2009One Point Isometric Matching with the Heat Kernel, Ovsjanikov et al, SGP 2010Blended Intrinsic Maps, Kim et al, SIGGRAPH 2011

10

Definitions

Accuracy error:

Consistency error:

11

Eacc(mA ;B ) =1

jAj

X

p2A

dB (mA ;B (p); ~mA ;B (p))

Econs(°) =1

jAj

X

p2A

dA (p;m° (p))

A B

C

B

A

Relating Cycles to Edges

Call low error “good,” high error “bad”

Good and bad edges cause good and bad cycles

If we can only evaluate the cycles, what can we say about the edges? 12

Relating Cycles to Edges

Accuracy error of a path ° = {i1, …, in} is bounded*:

13

Eacc(°) ·n¡ 1X

j =1

Eacc(mi j ;i j + 1 )

*If ground-truth maps preserve the distortion measure

Proposal – Linear ProgramFor each 3-cycle ° in the graph, compute the distortion C°

Solve the following linear program to find weights for the edges:

Minimize

Subject to

Where14

X

e2E

wece

X

e2°

ce ¸ C° 8°

ce ¸ 0 8e2 E

we = 1=(X

° :e2°

C° )

Are 3-Cycles Sufficient?

15

A B

C D

Proposal

LP gives us a weighted graph

Weights give us shortest-path map compositions

But these are just like our input

Run the LP again?16

Are 3-Cycles Sufficient?

17

A B

C D

A B

C D

ABD BAC

Proposal - CompleteRepeat the following:

Solve LP => obtain edge-weighted graph

Replace edges with shortest paths

Until one of the following is true:No edge replacements happen, or

No more 3-cycles are bad

18

Convergence - Experimental

19

Map Type LP Weights Final accuracy

Convergence - Theoretical

“Almost-accurate” collection: Each 3-cycle has at most 1 bad map

Every cycle’s distortion is either 0 or equal to the inaccuracy of the 1 bad map

LP weights are exactly the map accuracy errors

Guarantees consistency and accuracy after replacing maps with shortest paths

20

Results – 2D (DTW)

21

Max

con

sist

ency

err

or

Fraction of maps

Max

acc

urac

y er

ror

Fraction of cycles

Results – 2D (DTW)

22

Max

con

sist

ency

err

or

Fraction of maps

Max

acc

urac

y er

ror

Fraction of cycles

Results – 3D (Möbius Voting)

23

Results – 3D (Heat Kernel)

24Max

con

sist

ency

err

or

Fraction of maps

Max

acc

urac

y er

ror

Fraction of cycles

Results – 3D (Blended Maps)

25

Animals

Results – 3D (Blended Maps)

26

Hands

Results – 3D (Blended Maps)

27

Humans

Results – 3D (Blended Maps)

28

Teddies

Future Work

Prove convergence in more general casesAllow for multiple maps between a given pair of shapesDiscover the structure of the collection using consistency information

29

Conclusions

Collections contain information that allow us to better evaluate mapsCycle consistency can be used to identify and remove bad mapsUsing an LP with 3-cycle constraints lets us do this efficientlyRepeating the process lets us incorporate longer cycles

30

Thank You