life-long place recognition by shared representative...
TRANSCRIPT
![Page 1: Life-Long Place Recognition by Shared Representative ...fhan/publication/slides/rssw16_sral...Life-Long Place Recognition by Shared Representative Appearance Learning Fei Han 1, Xue](https://reader033.vdocuments.net/reader033/viewer/2022051600/5aad34287f8b9ac55c8dfcf1/html5/thumbnails/1.jpg)
Life-Long Place Recognition by Shared Representative
Appearance Learning
Fei Han1, Xue Yang1, Yiming Deng2,
Mark Rentschler3, Dejun Yang1, and Hao Zhang1
1. Division of Computer Science, Colorado School of Mines
2. Department of Electrical Engineering, University of Colorado Denver
3. Department of Mechanical Engineering, University of Colorado Boulder
![Page 2: Life-Long Place Recognition by Shared Representative ...fhan/publication/slides/rssw16_sral...Life-Long Place Recognition by Shared Representative Appearance Learning Fei Han 1, Xue](https://reader033.vdocuments.net/reader033/viewer/2022051600/5aad34287f8b9ac55c8dfcf1/html5/thumbnails/2.jpg)
Introduction
This work addresses long-term place recognition with strong appearance variations
2
Spring Summer
Autumn Winter
Morning
Afternoon
October
December
![Page 3: Life-Long Place Recognition by Shared Representative ...fhan/publication/slides/rssw16_sral...Life-Long Place Recognition by Shared Representative Appearance Learning Fei Han 1, Xue](https://reader033.vdocuments.net/reader033/viewer/2022051600/5aad34287f8b9ac55c8dfcf1/html5/thumbnails/3.jpg)
Introduction
The Problem
3
• Which feature is more representative while insensitive to appearance changes?
• How to fuse these heterogeneous feature modalities?
![Page 4: Life-Long Place Recognition by Shared Representative ...fhan/publication/slides/rssw16_sral...Life-Long Place Recognition by Shared Representative Appearance Learning Fei Han 1, Xue](https://reader033.vdocuments.net/reader033/viewer/2022051600/5aad34287f8b9ac55c8dfcf1/html5/thumbnails/4.jpg)
Contributions
Shared Representative Appearance Learning (SRAL) model for long-term place recognition
Capabilities Autonomous discriminative feature modality
learning and fusion that is robust to strong appearance variations
Sparse optimization formulation regularized by structured sparsity-inducing norms
Efficient solver with theoretical convergence guarantee
4
![Page 5: Life-Long Place Recognition by Shared Representative ...fhan/publication/slides/rssw16_sral...Life-Long Place Recognition by Shared Representative Appearance Learning Fei Han 1, Xue](https://reader033.vdocuments.net/reader033/viewer/2022051600/5aad34287f8b9ac55c8dfcf1/html5/thumbnails/5.jpg)
ApproachScene Representation
5
• A set of scenes is represented by
• Scene scenarios are represented as
𝒙𝒊 =𝒙𝒊𝟏
⋮𝒙𝒊𝒎∈ ℝ𝑑
Feature modality 1
𝒀 =
𝒚𝟏⋮𝒚𝒏∈ ℝ𝑛×𝑐
𝑿 = 𝒙𝟏 ⋯ 𝒙𝒊 ⋯ 𝒙𝒏 ∈ ℝ𝑑×𝑛
⋮Feature modality m
![Page 6: Life-Long Place Recognition by Shared Representative ...fhan/publication/slides/rssw16_sral...Life-Long Place Recognition by Shared Representative Appearance Learning Fei Han 1, Xue](https://reader033.vdocuments.net/reader033/viewer/2022051600/5aad34287f8b9ac55c8dfcf1/html5/thumbnails/6.jpg)
ApproachShared Representative Appearance Learning
6
ℓ2,1-norm: enforces sparsity of each row of W
ℓ𝑀-norm: enforces sparsity of each feature modality
Formulation:
![Page 7: Life-Long Place Recognition by Shared Representative ...fhan/publication/slides/rssw16_sral...Life-Long Place Recognition by Shared Representative Appearance Learning Fei Han 1, Xue](https://reader033.vdocuments.net/reader033/viewer/2022051600/5aad34287f8b9ac55c8dfcf1/html5/thumbnails/7.jpg)
Approach Optimization Algorithm
Theorem:
The Algorithm
converges to the
global optimal solution
of the formulated
optimization problem
7
![Page 8: Life-Long Place Recognition by Shared Representative ...fhan/publication/slides/rssw16_sral...Life-Long Place Recognition by Shared Representative Appearance Learning Fei Han 1, Xue](https://reader033.vdocuments.net/reader033/viewer/2022051600/5aad34287f8b9ac55c8dfcf1/html5/thumbnails/8.jpg)
ApproachPlace Recognition
• Optimal weight 𝜔𝑖 for each feature modality 𝑖, 𝑖 = 1, … ,𝑚
𝜔𝑖 =∥ 𝑾𝒊∗ ∥𝐹
• Feature fusion for image matching
𝑠(𝑞, 𝑝) =
𝑖=1
𝑚
𝜔𝑖𝑠𝑖 (𝑞, 𝑝)
8
![Page 9: Life-Long Place Recognition by Shared Representative ...fhan/publication/slides/rssw16_sral...Life-Long Place Recognition by Shared Representative Appearance Learning Fei Han 1, Xue](https://reader033.vdocuments.net/reader033/viewer/2022051600/5aad34287f8b9ac55c8dfcf1/html5/thumbnails/9.jpg)
Experiments
• Three public datasets Different time of the day:
St Lucia Dataset Different months: CMU-
VL Dataset Different seasons:
Nordland Dataset
• Setups Hardware: Workstation
with i7 3.4GHz, 16GB Platform: Ubuntu 14.04 +
Matlab 2015 Feature modalities: Color,
GIST, HOG, and LBP features
9
![Page 10: Life-Long Place Recognition by Shared Representative ...fhan/publication/slides/rssw16_sral...Life-Long Place Recognition by Shared Representative Appearance Learning Fei Han 1, Xue](https://reader033.vdocuments.net/reader033/viewer/2022051600/5aad34287f8b9ac55c8dfcf1/html5/thumbnails/10.jpg)
ExperimentsSt Lucia Dataset (Various Times of the Day)
10
Qualitative Evaluation
![Page 11: Life-Long Place Recognition by Shared Representative ...fhan/publication/slides/rssw16_sral...Life-Long Place Recognition by Shared Representative Appearance Learning Fei Han 1, Xue](https://reader033.vdocuments.net/reader033/viewer/2022051600/5aad34287f8b9ac55c8dfcf1/html5/thumbnails/11.jpg)
ExperimentsSt Lucia Dataset (Various Times of the Day)
11
Quantitative Evaluation
![Page 12: Life-Long Place Recognition by Shared Representative ...fhan/publication/slides/rssw16_sral...Life-Long Place Recognition by Shared Representative Appearance Learning Fei Han 1, Xue](https://reader033.vdocuments.net/reader033/viewer/2022051600/5aad34287f8b9ac55c8dfcf1/html5/thumbnails/12.jpg)
Experiments CMU-VL Dataset (Different Months)
12
Qualitative Evaluation
![Page 13: Life-Long Place Recognition by Shared Representative ...fhan/publication/slides/rssw16_sral...Life-Long Place Recognition by Shared Representative Appearance Learning Fei Han 1, Xue](https://reader033.vdocuments.net/reader033/viewer/2022051600/5aad34287f8b9ac55c8dfcf1/html5/thumbnails/13.jpg)
ExperimentsCMU-VL Dataset (Different Months)
13
Quantitative Evaluation
![Page 14: Life-Long Place Recognition by Shared Representative ...fhan/publication/slides/rssw16_sral...Life-Long Place Recognition by Shared Representative Appearance Learning Fei Han 1, Xue](https://reader033.vdocuments.net/reader033/viewer/2022051600/5aad34287f8b9ac55c8dfcf1/html5/thumbnails/14.jpg)
ExperimentsNordland Dataset (Different Seasons)
14
Qualitative Evaluation
![Page 15: Life-Long Place Recognition by Shared Representative ...fhan/publication/slides/rssw16_sral...Life-Long Place Recognition by Shared Representative Appearance Learning Fei Han 1, Xue](https://reader033.vdocuments.net/reader033/viewer/2022051600/5aad34287f8b9ac55c8dfcf1/html5/thumbnails/15.jpg)
Experiments Nordland Dataset (Different Seasons)
15
Quantitative Evaluation
![Page 16: Life-Long Place Recognition by Shared Representative ...fhan/publication/slides/rssw16_sral...Life-Long Place Recognition by Shared Representative Appearance Learning Fei Han 1, Xue](https://reader033.vdocuments.net/reader033/viewer/2022051600/5aad34287f8b9ac55c8dfcf1/html5/thumbnails/16.jpg)
Summary
We introduced a long-term place recognition method, named Shared Representative Appearance Learning(SRAL), which
• Can autonomously learn and integrate shared discriminative features that are robust to strong appearance variations
• Formulates the task as a convex optimization problem regularized by structured sparsity
• Can be efficiently solved with theoretical convergence guarantee
16
Life-Long Place Recognition byShared Representative Appearance Learning