a new rank constraint on multi -view fundamental matrices, ieee...
TRANSCRIPT
IEEE 2017 Conference on Computer Vision and Pattern
Recognition A New Rank Constraint on Multi-view Fundamental Matrices,
and its Application to Camera Location RecoveryS. Sengupta, T. Amir, M. Galun, A. Singer, T. Goldstein, D. Jacobs, R. Basri
Our Contribution
Ø A novel rank constraint on Fundamental matrices in multi-view settings
Ø Constrained low rank optimization to de-noise and recover missing fundamental matrices
Ø Improvement over state-of-the-art in structure from motion
Our Result
2
64
0 F̂12 F̂13 � �F̂21 0 F̂23 F̂24 �F̂31 F̂32 0 F̂34 F̂35
� F̂42 F̂43 0 F̂45
� � F̂53 F̂54 0
3
75
| {z }
⇡
2
40 �12 �13 �14 �15
�21 0 �23 �24 �25
�31 �32 0 �34 �35
�41 �42 �43 0 �45
�51 �52 �53 �54 0
3
5
| {z }
�
2
40 F12 F13 F14 F15
F21 0 F23 F24 F25
F31 F32 0 F34 F35
F41 F42 F43 0 F45
F51 F52 F53 F54 0
3
5
| {z }F̂ ⇤ F
with F = A+AT and rank(A) = 3.
F̂13
F̂23F̂24
F̂34
F̂35
F̂45
For n not all collinear cameras, the multi-view fundamental matrix F of size 3𝑛×3𝑛 satisfies :
Fii = 0,
F = A+AT ,
rank(A) = 3,
rank(F ) = 6.
Number of Images50 100 150
Rel
ativ
e Im
prov
emen
t (in
%)
0
5
10
15
20Median Essential Matrix Error
Our vs LUDOur vs ShapeKick
Number of Images50 100 150
Per
cent
of I
mpr
oved
Tria
ls
50
60
70
80
90
100Median Essential Matrix Error
Our vs LUDOur vs ShapeKick
Experimental ResultsØ 14 scenes from Photo-tourism datasetØ For each scene, 5 different sub-samples of 50, 100 and 150 imagesØ Measure error on Essential matrix and camera location recoveryØ Performance Measures:
o Relative Improvement (in %).o Number of Improved Trials.
Essential Matrix : Improves over LUD :
- by 18% on average- In 86% of all trials
Number of Images50 100 150
Rel
ativ
e Im
prov
emen
t (in
%)
0
10
20
30
40
50
60Median Translation Error
Our vs LUDOur vs ShapeKickOur vs 1DSfM
Number of Images50 100 150
Per
cent
of I
mpr
oved
Tria
ls
50
60
70
80
90
100Median Translation Error
Our vs LUDOur vs ShapeKickOur vs 1DSfM
Camera Location : Improves over LUD :
- by 20% on average- In 84% of all trials
References1.“Robust Global Translation with 1DSfM” K. Wilson et.al. ECCV 2014.(1DSfM)2.“ Robust camera location estimation by convex programming” O. Ozyesil et.al. CVPR 2015. (LUD)3. “ Shapefit and shapekick for robust, scalable structure from motion” T. Goldstein et.al. ECCV 2016. (ShapeKick)
Rank constrained Optimization
Robust L1 cost function
Solve using IRLS - ADMM
ConstructionFor simplicity, we show our construction for Essential matrices
𝑇& = 𝑡& ×: Camera location, 𝑅&: Camera orientation
Equality is proved in the paper
Result applies also to fundamental matrices since 𝐹&+ = 𝐾-.𝐸&+𝐾-0
median T error mean T error median E error mean E error
Impro
vem
ent over
LU
D (
%)
0
5
10
15
20
Our - fundamental recoveryOur - essential recovery
Our method can also optimize directly over Fundamental matrices by jointly optimizing camera rotation, location and calibration.
Code
www.umiacs.umd.edu/~sengupta/SfM_CVPR2017_code.zip
Our Rank Constraint