robust visual tracking – algorithms, evaluations and problems
DESCRIPTION
Robust Visual Tracking – Algorithms, Evaluations and Problems. Haibin Ling Department of Computer and Information Sciences Temple University Philadelphia, PA 19122. October 15, 2014. Visual Tracking. Continuously localization of a visual entity or visual entities. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Robust Visual Tracking – Algorithms, Evaluations and Problems](https://reader036.vdocuments.net/reader036/viewer/2022062721/5681364d550346895d9dcf02/html5/thumbnails/1.jpg)
Robust Visual Tracking – Algorithms, Evaluations and
Problems
Haibin LingDepartment of Computer and Information Sciences
Temple University
Philadelphia, PA 19122
October 15, 2014
![Page 2: Robust Visual Tracking – Algorithms, Evaluations and Problems](https://reader036.vdocuments.net/reader036/viewer/2022062721/5681364d550346895d9dcf02/html5/thumbnails/2.jpg)
Visual Tracking
Single target tracking (model-free) (PAMI’11,CVPR’11,ICCV’11,CVPR’12,ICCV’13,ECCV’14)
Pose tracking (Sigal et al 2004)
Contour tracking (CVPR’14b)
Continuously localization of a visual entity or visual entities.
Multi-target tracking (CVPR’13,CVPR’14a)
![Page 3: Robust Visual Tracking – Algorithms, Evaluations and Problems](https://reader036.vdocuments.net/reader036/viewer/2022062721/5681364d550346895d9dcf02/html5/thumbnails/3.jpg)
Visual TrackingContinuously localization of a visual entity or visual entities.
Related work- Tooooooo many to be listed- A survey by Yilmaz, Javed & Shah in 2006- There are many influential trackers after 2006
Single target tracking (model-free) (PAMI’11,CVPR’11,ICCV’11,CVPR’12,ICCV’13,ECCV’14)
![Page 4: Robust Visual Tracking – Algorithms, Evaluations and Problems](https://reader036.vdocuments.net/reader036/viewer/2022062721/5681364d550346895d9dcf02/html5/thumbnails/4.jpg)
Outline
• Problem formulation and particle filter tracking framework
• Visual tracking using sparse representation
• Reducing bias in tracking evaluation
• Recent and future work
![Page 5: Robust Visual Tracking – Algorithms, Evaluations and Problems](https://reader036.vdocuments.net/reader036/viewer/2022062721/5681364d550346895d9dcf02/html5/thumbnails/5.jpg)
Problem formulation
Input:
• A sequence of images: I0, I1, …, It, …
• Target of interest at the initial frame: x0
A target is represented by a state vector
x = (pos, scale, orientation)‘
Output:• Targets in each of the following frames
– x1, …, xt, …
![Page 6: Robust Visual Tracking – Algorithms, Evaluations and Problems](https://reader036.vdocuments.net/reader036/viewer/2022062721/5681364d550346895d9dcf02/html5/thumbnails/6.jpg)
Tracking by Bayesian Estimation
At frame t, find the best xt by Bayesian inference
Using observations (features) extracted from images I0, I1, …, It :
We have
Kalman filter– Gaussian everywhere closed form solution – But, probabilities in visual tracking is not usually Gaussian
Particle filter– Probability propagation: iterative prediction and updating – Sampling techniques
),,...,,|(maxarg 011 IIIIxpx tttx
tt
)|(maxarg :0 ttx
t yxpxt
},...,,{:;,...,, 10:010 ttt yyyyyyy
Bayesian estimation:
![Page 7: Robust Visual Tracking – Algorithms, Evaluations and Problems](https://reader036.vdocuments.net/reader036/viewer/2022062721/5681364d550346895d9dcf02/html5/thumbnails/7.jpg)
Particle Filter (Isard & Blake 98)
Prediction:
Update:
)|(maxarg :0 ttx
t yxpxt
Visual tracking
1
11:0111:0 )|()|()|(tx ttttttt dxyxpxxpyxp
likelihood nobservatio:)|(
etc) motion, (drift,y probabilit n transitiostate:)|( 1
tt
tt
xyp
xxp
)|()|()|( 1:0:0 tttttt yxpxypyxp
Probability propagation
Particle sampling (sequential Monte Carlo)
Approximate the posterior density by a set of weighted samples:
Niwx it
it ,...,2,1:),( )()(
).|( e.g.,, particlefor weight theiswhere it
it
it
it xypxw
Now we need to decide
![Page 8: Robust Visual Tracking – Algorithms, Evaluations and Problems](https://reader036.vdocuments.net/reader036/viewer/2022062721/5681364d550346895d9dcf02/html5/thumbnails/8.jpg)
Outline
• Problem formulation and particle filter tracking framework
• Visual tracking using sparse representation
• Reducing bias in tracking evaluation
• Recent and future work
![Page 9: Robust Visual Tracking – Algorithms, Evaluations and Problems](https://reader036.vdocuments.net/reader036/viewer/2022062721/5681364d550346895d9dcf02/html5/thumbnails/9.jpg)
MotivationIntuition• During tracking, there is a large redundancy in the observation of target
appearance• It is common to represent the target appearance using a linear
representation
Idea• Introduce sparse constraints in the linear target representation• Non-negativity constraints
Advantage• Models observation redundancy naturally.• Addresses discrete appearance corruption such as occlusion (Wright et al.
2009) • Benefits from recent advance in solutions for sparse coding/compressive
sensing (Candes et al. 2006, Donoho 2006)• A flexible framework (as illustrated in many extensions)
![Page 10: Robust Visual Tracking – Algorithms, Evaluations and Problems](https://reader036.vdocuments.net/reader036/viewer/2022062721/5681364d550346895d9dcf02/html5/thumbnails/10.jpg)
Sparse Representation for Tracking
• A candidate y approximately lies in a linear subspace, which is spanned by templates from past observation
e
a]I,T[ddnn eeeaaa iiittty 22112211
nnaaa ttty 2211 nnaaa ttty 2211
Task: find a sparse solution for a and e.
Rewrite as
![Page 11: Robust Visual Tracking – Algorithms, Evaluations and Problems](https://reader036.vdocuments.net/reader036/viewer/2022062721/5681364d550346895d9dcf02/html5/thumbnails/11.jpg)
Non-negativity Constraints
• In addition to the (positive) trivial templates I, we include negative trivial templates -I.
0cBc,ˆ
e
e
a
]II,,T[y
-
)i()i()i( d2211 deee
ddnn eeeaaa iiittty 22112211
where ai, ei, ei- >=0 .
The formula can be rewritten as
![Page 12: Robust Visual Tracking – Algorithms, Evaluations and Problems](https://reader036.vdocuments.net/reader036/viewer/2022062721/5681364d550346895d9dcf02/html5/thumbnails/12.jpg)
Example Templates
y
e
e
a
B c
![Page 13: Robust Visual Tracking – Algorithms, Evaluations and Problems](https://reader036.vdocuments.net/reader036/viewer/2022062721/5681364d550346895d9dcf02/html5/thumbnails/13.jpg)
Comparing Good and Bad Candidates
![Page 14: Robust Visual Tracking – Algorithms, Evaluations and Problems](https://reader036.vdocuments.net/reader036/viewer/2022062721/5681364d550346895d9dcf02/html5/thumbnails/14.jpg)
Achieving Sparse Solutions
0c,cyBcmin0
2
2
Our task is to find a sparse solution to the following linear system,
0cBc,y
It leads to an L0 minimization task, such as
This can be well approximated, under very flexible conditions, by an L1 minimization,
0c,cyBcmin1
2
2
![Page 15: Robust Visual Tracking – Algorithms, Evaluations and Problems](https://reader036.vdocuments.net/reader036/viewer/2022062721/5681364d550346895d9dcf02/html5/thumbnails/15.jpg)
Extension• Speed up
– Speed up: bounded particle resampling (CVPR’11)– Speed up: accelerated proximal gradient (CVPR’12)– Blurred target tracking (ICCV’11)
• Other sparse-representation trackers– Liu et al. ECCV'10, – Li, Shen & Shi CVPR'11, Liu et al CVPR'11, Kwak et al ICCV’11– Zhong, Lu & Yang CVPR'12; Jia, Lu & Yang CVPR'12; Zhang,
Zhang & Yang CVPR'12; ZhangT et al CVPR'12, – ZhangT et al IJCV’13, Hu et al PAMI’14– …
![Page 16: Robust Visual Tracking – Algorithms, Evaluations and Problems](https://reader036.vdocuments.net/reader036/viewer/2022062721/5681364d550346895d9dcf02/html5/thumbnails/16.jpg)
Outline
• Problem formulation and particle filter tracking framework
• Visual tracking using sparse representation
• Reducing bias in tracking evaluation
• Recent and future work
![Page 17: Robust Visual Tracking – Algorithms, Evaluations and Problems](https://reader036.vdocuments.net/reader036/viewer/2022062721/5681364d550346895d9dcf02/html5/thumbnails/17.jpg)
Reducing Subjective Bias
• Which are the best trackers among all?• Implementing and testing on a large benchmark (e.g.,
Wu et al 2013) is a huge project.
• Recent trend: compare the authors’ own tracker with many other trackers.
• Their own tracker typically performs the best.– It has advantages that the authors want to highlight.– Optimizing all trackers is non-trivial, if not possible.
• We aim to reduce such biases and provide a more practical comparison.
![Page 18: Robust Visual Tracking – Algorithms, Evaluations and Problems](https://reader036.vdocuments.net/reader036/viewer/2022062721/5681364d550346895d9dcf02/html5/thumbnails/18.jpg)
An example
A B C D E
Seq 1 17.5 56.7 11.3 10.5 5.0
Seq 2 7.0 39.2 8.5 39.2 6.1
… … … … … …
Seq N 30.7 66.2 20.4 120.4 24.9
• The best two results are shown in red and blue
Average Center Location Error
The proposed tracker
The authors’ previous tracker
![Page 19: Robust Visual Tracking – Algorithms, Evaluations and Problems](https://reader036.vdocuments.net/reader036/viewer/2022062721/5681364d550346895d9dcf02/html5/thumbnails/19.jpg)
Partial ranking representation
A B C D E
Seq 1 17.5 56.7 11.3 10.5 5.0
Seq 2 7.0 39.2 8.5 39.2 6.1
… … … … … …
Seq N 30.7 66.2 20.4 120.4 24.9
Average Center Location Error
Higher rank Lower rank
D < A < B
Higher rank Lower rank
D < A < B
A < B = D
… < … < …
A < B < D
D10.5
A17.5
B56.7
< <
![Page 20: Robust Visual Tracking – Algorithms, Evaluations and Problems](https://reader036.vdocuments.net/reader036/viewer/2022062721/5681364d550346895d9dcf02/html5/thumbnails/20.jpg)
A B C D E
Seq 1 17.5 56.7 11.3 10.5 5.0
Seq 2 7.0 39.2 8.5 39.2 6.1
… … … … … …
Seq N 30.7 66.2 20.4 120.4 24.9
Average Center Location Error
Pairwise representation
(A, B, 1)
(D, A, 1)
(D, B, 1)
(A, B, 1)
(A, D, 1)
(B, D, 0.5)
Seq 1 Seq 2
(D, B, 0.5)
…
(A, B, 1)
(A, D, 1)
(B, D, 1)
Seq N
A7.0
B39.2
<
D39.2
=
![Page 21: Robust Visual Tracking – Algorithms, Evaluations and Problems](https://reader036.vdocuments.net/reader036/viewer/2022062721/5681364d550346895d9dcf02/html5/thumbnails/21.jpg)
Data Statistics
• PAMI (2000 Vol.22– 2013 Vol.35),
IJCV (2000 Vol.36 – 2013 Vol.104)• ICCV, CVPR, ECCV (2005 – 2013)• 45 papers (tournament) contain useful table data• 48 trackers appear in the data at the first stage• 15 trackers are left after the cleaning• 664 partial rankings• 6280 pairs of records with 151 draw records
![Page 22: Robust Visual Tracking – Algorithms, Evaluations and Problems](https://reader036.vdocuments.net/reader036/viewer/2022062721/5681364d550346895d9dcf02/html5/thumbnails/22.jpg)
Paper selection and data cleaning
• More than 2 trackers left after remove unqualified trackers
• Independent assumption– Conference to journal extension– Duplicate experimental results
• Significant lack of data– Compared only in one tournament– #records ≤ 10
![Page 23: Robust Visual Tracking – Algorithms, Evaluations and Problems](https://reader036.vdocuments.net/reader036/viewer/2022062721/5681364d550346895d9dcf02/html5/thumbnails/23.jpg)
Rank aggregation• Rank aggregation (Ailon 2010)
– Find a full-ranking to minimize the total violation of pairwise comparison.
– NP-Hard, LpKwikSorth algorithm
• PageRank-like ranking (Page et al. 1999)– Graph-based solution
• Elo’s rating (Elo 1978)– Widely used in sport ranking (chess, football, …)– Sequentially update score based on each game
• Glicko’s rating (Glickman 1999)– Extension of Elo’s rating by introducing confidence
![Page 24: Robust Visual Tracking – Algorithms, Evaluations and Problems](https://reader036.vdocuments.net/reader036/viewer/2022062721/5681364d550346895d9dcf02/html5/thumbnails/24.jpg)
Ranking results
![Page 25: Robust Visual Tracking – Algorithms, Evaluations and Problems](https://reader036.vdocuments.net/reader036/viewer/2022062721/5681364d550346895d9dcf02/html5/thumbnails/25.jpg)
Outline
• Problem formulation and particle filter tracking framework
• Visual tracking using sparse representation
• Reducing bias in tracking evaluation
• Recent and future work
![Page 26: Robust Visual Tracking – Algorithms, Evaluations and Problems](https://reader036.vdocuments.net/reader036/viewer/2022062721/5681364d550346895d9dcf02/html5/thumbnails/26.jpg)
Tracking with GPR (TGPR)Transfer Learning Based Visual Tracking with Gaussian Processes Regression
Gao, Ling, Hu & Xing, ECCV 2014
Source code of TGPR available: http://www.dabi.temple.edu/~hbling/code/TGPR.htm or http://jingao.weebly.com/
![Page 27: Robust Visual Tracking – Algorithms, Evaluations and Problems](https://reader036.vdocuments.net/reader036/viewer/2022062721/5681364d550346895d9dcf02/html5/thumbnails/27.jpg)
Promising ResultsCVPR2013 Benchmark
(Wu et al 2013)
50 sequences
Princeton Benchmark (Song & Xiao 2013)
100 sequences
VOT2013 (Kristan et al
2013)
16 sequences
![Page 28: Robust Visual Tracking – Algorithms, Evaluations and Problems](https://reader036.vdocuments.net/reader036/viewer/2022062721/5681364d550346895d9dcf02/html5/thumbnails/28.jpg)
Acknowledgement
• CollaboratorsChenglong Bao, Erik Blasch, Jin Gao, Weiming Hu
Hui Ji, Xue Mei, Yu Pang, Yi Wu
• Funding• National Sciences Foundation
• Air Force Research Laboratory
![Page 29: Robust Visual Tracking – Algorithms, Evaluations and Problems](https://reader036.vdocuments.net/reader036/viewer/2022062721/5681364d550346895d9dcf02/html5/thumbnails/29.jpg)
Thank You!&
Questions?