sequential learned linear predictors for object tracking - center for machine...
TRANSCRIPT
Sequential Learned Linear Predictors for ObjectTracking
Tomas Svobodajoint work with Karel Zimmermann, Jirı Matas, and David Hurych
Czech Technical University, Faculty of Electrical EngineeringCenter for Machine Perception, Prague, Czech Republic
http://cmp.felk.cvut.cz/~svoboda
June 23, 2009
Object tracking
I Tracking - iterative estimation of an object pose in asequence (e.g., 2D position of Basil’s head).
I Trade-off among accuracy, speed, robustness is explicitlytaken into account.
video: Basil head
Object tracking
I Tracking - iterative estimation of an object pose in asequence (e.g., 2D position of Basil’s head).
I Trade-off among accuracy, speed, robustness is explicitlytaken into account.
video: Basil head
Object tracking
I Tracking - iterative estimation of an object pose in asequence (e.g., 2D position of Basil’s head).
I Trade-off among accuracy, speed, robustness is explicitlytaken into account.
video: Cartoon eye
Tracking
Current imageTemplate
Motion
Observation
Observation (image) I and template J
Tracking of a single point
X
Current position
Tracking of a single point
X
Old position
Tracking of a single point
X
Old position
[Lucas-Kanade1981] - Iterative minimization
argmint‖I(t)− J‖2argmint‖I(t)− J‖2image I alignment t
J← I(t)
optional template updateLucas–Kanade
[Lucas-Kanade1981] - Iterative minimization
argmint‖I(t)− J‖2argmint‖I(t)− J‖2image I alignment t
J← I(t)
optional template updateLucas–Kanade
video: KLT iterations
Regression - learn the mapping in advanceo
H(I− J)H(I− J)image I alignment t
J← I(t)
optional template updateJurie–Dhome
I [Jurie-BMVC-2002] - learned motion and optional hardtemplate update
I [Cootes-PAMI-2001] - learned regression during AAM iterations
I . . .
Learning alignment for one predictor
I ϕ( )= (0, 0)>
I ϕ( )= (−25, 0)>
I ϕ( )= (25,−15)>
I ϕ( )= (0, 0)>
I ϕ( )= (−25, 0)>
I ϕ( )= (25,−15)>
Learning alignment for one predictor
I ϕ( )= (0, 0)>
I ϕ( )= (−25, 0)>
I ϕ( )= (25,−15)>
I ϕ( )= (0, 0)>
I ϕ( )= (−25, 0)>
I ϕ( )= (25,−15)>
Learning alignment for one predictor
I ϕ( )= (0, 0)>
I ϕ( )= (−25, 0)>
I ϕ( )= (25,−15)>
I ϕ( )= (0, 0)>
I ϕ( )= (−25, 0)>
I ϕ( )= (25,−15)>
Learning alignment for one predictor
I ϕ( )= (0, 0)>
I ϕ( )= (−25, 0)>
I ϕ( )= (25,−15)>
I ϕ( )= (0, 0)>
I ϕ( )= (−25, 0)>
I ϕ( )= (25,−15)>
Learning alignment for one predictor
I ϕ( )= (0, 0)>
I ϕ( )= (−25, 0)>
I ϕ( )= (25,−15)>
I ϕ( )= (0, 0)>
I ϕ( )= (−25, 0)>
I ϕ( )= (25,−15)>
Learning alignment for one predictor
I ϕ( )= (0, 0)>
I ϕ( )= (−25, 0)>
I ϕ( )= (25,−15)>
I ϕ( )= (0, 0)>
I ϕ( )= (−25, 0)>
I ϕ( )= (25,−15)>
Generating training examples
[ ]
t − motioni I = I(x ,...x ) − observation1 ci
examples
displacementestimation
generating
+2
+7+9
+4
−8
−4
training set: +2 −4 +4+9 +7 −8
T= I=
Training set: (I, T)I = [I1 − J, I2 − J, . . . Id − J] and T = [t1, t2, . . . td ].
LS learning
ϕ∗ = argminϕ∑
t
‖ϕ(I(t ◦ X
))− t‖2.
Minimizes sum of square errors over all training set. Leads tomatrix pseudoinverse computation.
Example for linear mapping:
H∗ = argminH
d∑i=1
‖H(Ii − J)− ti‖22 = argminH‖HI− T‖2F
after some derivation
H∗ = T I>(II>)−1︸ ︷︷ ︸I+
= TI+.
LS learning
ϕ∗ = argminϕ∑
t
‖ϕ(I(t ◦ X
))− t‖2.
Minimizes sum of square errors over all training set. Leads tomatrix pseudoinverse computation.
Example for linear mapping:
H∗ = argminH
d∑i=1
‖H(Ii − J)− ti‖22 = argminH‖HI− T‖2F
after some derivation
H∗ = T I>(II>)−1︸ ︷︷ ︸I+
= TI+.
LS learning
ϕ∗ = argminϕ∑
t
‖ϕ(I(t ◦ X
))− t‖2.
Minimizes sum of square errors over all training set. Leads tomatrix pseudoinverse computation.
Example for linear mapping:
H∗ = argminH
d∑i=1
‖H(Ii − J)− ti‖22 = argminH‖HI− T‖2F
after some derivation
H∗ = T I>(II>)−1︸ ︷︷ ︸I+
= TI+.
Min-max learning
ϕ∗ = argminϕ maxt‖ϕ(I(t ◦ X
))− t‖∞.
Minimizes the worst case (the biggest estimation error) in thetraining set. Leads to linear programming.
Tracking of a single point by a sequence of predictors
X
t =1 ϕ1
Old position
Tracking of a single point by a sequence of predictors
X
o
1
t =1 ϕ1
Old position
Tracking of a single point by a sequence of predictors
X
o
1
t =1 ϕ1
Old position
Tracking of a single point by a sequence of predictors
X
o
1
t =1 ϕ1
Old position
Tracking of a single point by a sequence of predictors
X
o
1
t =1 ϕ1
t =2 ϕ2
Old position
Tracking of a single point by a sequence of predictors
X
o
o2
1
t =1 ϕ1
t =2 ϕ2
Old position
Tracking of a single point by a sequence of predictors
X
o
o2
1
t =1 ϕ1
t =2 ϕ2
Old position
Tracking of a single point by a sequence of predictors
X
o
o2
1
t =1 ϕ1
t =2 ϕ2
Old position
Tracking of a single point by a sequence of predictors
X
o
o2
1
t =1 ϕ1
t =3 ϕ3
t =2 ϕ2
Old position
Tracking of a single point by a sequence of predictors
X
o
o
o
2
1
3 t =1 ϕ1
t =3 ϕ3
t =2 ϕ2
Old position
Tracking of a single point by a sequence of predictors
X
o
o
o
2
1
3
New position
t =1 ϕ1
t =3 ϕ3
t =2 ϕ2
Old position
Tracking of a single point by a sequence of predictors
1 2Φ=(ϕ,ϕ,ϕ)
3
X
o
o
o
2
1
3
New position
t =1 ϕ1
Motion
t =3 ϕ3
t =2 ϕ2
Old position
Tracking of a single point by a sequence of predictors
1 2Φ=(ϕ,ϕ,ϕ)
3
Ranges
X
o
o
o
2
1
3
New position
t =1 ϕ1
Motion
t =3 ϕ3
t =2 ϕ2
Old position
Learning of sequential predictor
I Learning - searching for the sequence with predefined range,accuracy and minimal computational cost.
I [Zimmermann-PAMI-2009] - Dynamic programming estimatesthe optimal sequence of linear predictors.
I [Zimmermann-IVC-2009] - Branch & bound search allows fortime constrained learning (demo in MATLAB).
[Zimmermann-PAMI-2009] K.Zimmermann, J.Matas, T.Svoboda. Tracking by anOptimal Sequence of Linear Predictors, in IEEE Transactions on Pattern Analysis andMachine Intelligence, IEEE computer society, 2009, vol. 31, No 4, pp 677–692.
[Zimmermann-IVC-2009] K.Zimmermann, T.Svoboda, J.Matas. Anytime learning for
the NoSLLiP tracker. Image and Vision Computing, Elsevier, accepted, available
on-line.
Learning of sequential predictor
I Learning - searching for the sequence with predefined range,accuracy and minimal computational cost.
I [Zimmermann-PAMI-2009] - Dynamic programming estimatesthe optimal sequence of linear predictors.
I [Zimmermann-IVC-2009] - Branch & bound search allows fortime constrained learning (demo in MATLAB).
[Zimmermann-PAMI-2009] K.Zimmermann, J.Matas, T.Svoboda. Tracking by anOptimal Sequence of Linear Predictors, in IEEE Transactions on Pattern Analysis andMachine Intelligence, IEEE computer society, 2009, vol. 31, No 4, pp 677–692.
[Zimmermann-IVC-2009] K.Zimmermann, T.Svoboda, J.Matas. Anytime learning for
the NoSLLiP tracker. Image and Vision Computing, Elsevier, accepted, available
on-line.
Learning of sequential predictor
I Learning - searching for the sequence with predefined range,accuracy and minimal computational cost.
I [Zimmermann-PAMI-2009] - Dynamic programming estimatesthe optimal sequence of linear predictors.
I [Zimmermann-IVC-2009] - Branch & bound search allows fortime constrained learning (demo in MATLAB).
[Zimmermann-PAMI-2009] K.Zimmermann, J.Matas, T.Svoboda. Tracking by anOptimal Sequence of Linear Predictors, in IEEE Transactions on Pattern Analysis andMachine Intelligence, IEEE computer society, 2009, vol. 31, No 4, pp 677–692.
[Zimmermann-IVC-2009] K.Zimmermann, T.Svoboda, J.Matas. Anytime learning for
the NoSLLiP tracker. Image and Vision Computing, Elsevier, accepted, available
on-line.
Learning of the optimal sequence of linear predictors
Range
Complexity
Uncertaintyregion
I Range: the set of admissible motions, r .I Complexity: cardinality of support set, c .I Uncertainty region: the region within which all predictions
lie, λ. Small red circles show acceptable uncertainty.
Learning of the optimal sequence of linear predictors
0
20
40
60
80
100
0
50
100
150
200
0
10
20
30
40
50
r − range
c − complexity
λ −
unc
erta
inty
are
a
Achievablepredictors ω
Inachievablepredictors
λ−minimal predictors
I Range: the set of admissible motions, r .I Complexity: cardinality of support set, c .I Uncertainty region: the region within which all predictions
lie, λ. Small red circles show acceptable uncertainty.
Learning of the optimal sequence of linear predictors
I Range: the set of admissible motions, r .I Complexity: cardinality of support set, c .I Uncertainty region: the region within which all predictions
lie, λ. Small red circles show acceptable uncertainty.
Learning of the optimal sequence of linear predictors
I Range: the set of admissible motions, r .I Complexity: cardinality of support set, c .I Uncertainty region: the region within which all predictions
lie, λ. Small red circles show acceptable uncertainty.
Branch and Bound
0 1 2 3 4 50
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
Graph depth
Pre
dic
tion e
rror
c=600c=320 c=360
c=80
0 1 2 3 4 50
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
Graph depth
Pre
dic
tion e
rror
c=100
c=600c=320
c=380
Don’t forget to show the live demo!
Branch and Bound
0 1 2 3 4 50
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
Graph depth
Pre
dic
tion e
rror
c=600c=320 c=360
c=80
0 1 2 3 4 50
0.05
0.1
0.15
0.2
0.25
0.3
0.35
0.4
Graph depth
Pre
dic
tion e
rror
c=100
c=600c=320
c=380
Don’t forget to show the live demo!
Tracking with one linear predictor.
Modeling motion by number of linear predictors.
Motion blur, fast motion, views from acute angles andother image distortions.
Support set selection
0.065 0.07 0.075 0.08 0.085 0.09 0.095 0.1 0.105 0.110
50
100
150
200
250
300
350
400
450
500
MSE
I Greedy LS selection (red) of an efficient support set.
I Much better than 1%-quantile (green) achieavable byrandomized sampling
Tracking of objects with variable appearance
I Variable appearance - the way how the object looks like inthe camera changes due to illumination, non-rigiddeformation, out-of-plane rotation, ...
Tracking of objects with variable appearance
I Variable appearance - the way how the object looks like inthe camera changes due to illumination, non-rigiddeformation, out-of-plane rotation, ...
Simultaneous learning of motion and appearance
ϕ(I)ϕ(I)image I alignment t
I Introduce feedback which encodes appearance in a lowdimensional space and adjust the predictor.
I Appearance parameters learned in unsupervised way.
I Simultaneous learning of ϕ and γ ⇒ appearance encoded inthe low dimensional space, which is the most suitable for themotion estimation.
Simultaneous learning of motion and appearance
ϕ(I;θ)ϕ(I;θ)
γ(J)γ(J)
image I alignment t
J← I(t) imageθ appearance parameters
I Introduce feedback which encodes appearance in a lowdimensional space and adjust the predictor.
I Appearance parameters learned in unsupervised way.
I Simultaneous learning of ϕ and γ ⇒ appearance encoded inthe low dimensional space, which is the most suitable for themotion estimation.
Simultaneous learning of motion and appearance
ϕ(I;θ)ϕ(I;θ)
γ(J)γ(J)
image I alignment t
J← I(t) imageθ appearance parameters
I Introduce feedback which encodes appearance in a lowdimensional space and adjust the predictor.
I Appearance parameters learned in unsupervised way.
I Simultaneous learning of ϕ and γ ⇒ appearance encoded inthe low dimensional space, which is the most suitable for themotion estimation.
Simultaneous learning of motion and appearance
ϕ(I;θ)ϕ(I;θ)
γ(J)γ(J)
image I alignment t
J← I(t) imageθ appearance parameters
I Introduce feedback which encodes appearance in a lowdimensional space and adjust the predictor.
I Appearance parameters learned in unsupervised way.
I Simultaneous learning of ϕ and γ ⇒ appearance encoded inthe low dimensional space, which is the most suitable for themotion estimation.
Learning appearance
H(I− J)H(I− J)image I alignment t
J← I(t)
optional template updateJurie–Dhome
Learning appearance
H(I− J(θ))H(I− J(θ))H(I− J(θ))H(I− J(θ))
PCAJPCAJ
timage I
I(t)θ
Cootes–Edwards
Learning appearance – our approach
H(I− J(θ))H(I− J(θ))H(I− J(θ))H(I− J(θ))
PCAJPCAJ
timage I
I(t)θ
Cootes–Edwards
ϕ(I;θ)ϕ(I;θ)
γ(I(t))γ(I(t))
image I alignment t
I(t) imageθ appearance parameters
Our algorithm
Learning appearance – our approach
H(I− J(θ))H(I− J(θ))H(I− J(θ))H(I− J(θ))
PCAJPCAJ
timage I
I(t)θ
Cootes–Edwards
ϕ(I;θ)ϕ(I;θ)
γ(I(t))γ(I(t))
image I t
I(t) imageθ appearance parameters
Our algorithm
motion estimator
appearance encoder
Learning the appearance encoder γ
I Current appearance encoded in low-dim parameters.
I γ( ) = θ1 I γ( ) = θ2
Learning the appearance encoder γ
I Current appearance encoded in low-dim parameters.
I γ( ) = θ1 I γ( ) = θ2
Learning the appearance encoder γ
I Current appearance encoded in low-dim parameters.
I γ( ) = θ1 I γ( ) = θ2
Learning the tracker ϕ(I; θ)
I ϕ( ; θ1)= (0, 0)>
I ϕ( ; θ1)= (−25, 0)>
I ϕ( ; θ1)= (25,−15)>
I ϕ( ; θ2)= (0, 0)>
I ϕ( ; θ2)= (−25, 0)>
I ϕ( ; θ2)= (25,−15)>
Learning the tracker ϕ(I; θ)
I ϕ( ; θ1)= (0, 0)>
I ϕ( ; θ1)= (−25, 0)>
I ϕ( ; θ1)= (25,−15)>
I ϕ( ; θ2)= (0, 0)>
I ϕ( ; θ2)= (−25, 0)>
I ϕ( ; θ2)= (25,−15)>
Learning the tracker ϕ(I; θ)
I ϕ( ; θ1)= (0, 0)>
I ϕ( ; θ1)= (−25, 0)>
I ϕ( ; θ1)= (25,−15)>
I ϕ( ; θ2)= (0, 0)>
I ϕ( ; θ2)= (−25, 0)>
I ϕ( ; θ2)= (25,−15)>
Learning the tracker ϕ(I; θ)
I ϕ( ; θ1)= (0, 0)>
I ϕ( ; θ1)= (−25, 0)>
I ϕ( ; θ1)= (25,−15)>
I ϕ( ; θ2)= (0, 0)>
I ϕ( ; θ2)= (−25, 0)>
I ϕ( ; θ2)= (25,−15)>
Learning the tracker ϕ(I; θ)
I ϕ( ; θ1)= (0, 0)>
I ϕ( ; θ1)= (−25, 0)>
I ϕ( ; θ1)= (25,−15)>
I ϕ( ; θ2)= (0, 0)>
I ϕ( ; θ2)= (−25, 0)>
I ϕ( ; θ2)= (25,−15)>
Learning the tracker ϕ(I; θ)
I ϕ( ; θ1)= (0, 0)>
I ϕ( ; θ1)= (−25, 0)>
I ϕ( ; θ1)= (25,−15)>
I ϕ( ; θ2)= (0, 0)>
I ϕ( ; θ2)= (−25, 0)>
I ϕ( ; θ2)= (25,−15)>
Simultaneous learning of ϕ and γ
I Learning = minimization of the least-squares error
(ϕ∗, γ∗) = arg minϕ,γ[ϕ(
; γ( ))−(0, 0)>
]2+[
ϕ(
; γ( ))−(−25, 0)>
]2+[
ϕ(
; γ( ))−(25,−15)>
]2+[
ϕ(
; γ( ))−(0, 0)>
]2+[
ϕ(
; γ( ))−(−25, 0)>
]2+[
ϕ(
; γ( ))−(25,−15)>
]2
Simultaneous learning of ϕ and γ
I Learning = minimization of the least-squares error
(ϕ∗, γ∗) = arg minϕ,γ[ϕ(
; γ( ))−(0, 0)>
]2+[
ϕ(
; γ( ))−(−25, 0)>
]2+[
ϕ(
; γ( ))−(25,−15)>
]2+[
ϕ(
; γ( ))−(0, 0)>
]2+[
ϕ(
; γ( ))−(−25, 0)>
]2+[
ϕ(
; γ( ))−(25,−15)>
]2
Linear mapping
I γ(J) : θ = GJ
I ϕ(I,θ) : t = (H0 + θ1H1 + · · ·+ θnHn)I
I Criterion is sum of squares of bilinear functions.
Linear mapping
I γ(J) : θ = GJ
I ϕ(I,θ) : t = (H0 + θ1H1 + · · ·+ θnHn)I
I Criterion is sum of squares of bilinear functions.
Linear mapping
I γ(J) : θ = GJ
I ϕ(I,θ) : t = (H0 + θ1H1 + · · ·+ θnHn)I
I Criterion is sum of squares of bilinear functions.
Algorithm: iterative minimization of criterion e(ϕ, γ)
ϕ – motion (geometry mapping), γ – appearance mapping
I Iterative minimization:I initialization γ0 = randI ϕ1 = arg minϕ e(ϕ, γ0)I γ1 = arg minγ e(ϕ1, γ)I ϕ2 = arg minϕ e(ϕ, γ1)I γ2 = arg minγ e(ϕ2, γ)I ϕ3 = arg minϕ e(ϕ, γ2)I γ3 = arg minγ e(ϕ3, γ)I until convergence reached
color encodes criterion value e(ϕ, γ)
γϕ
0 2 4 6 8
0.8
1
1.2
1.4
1.6
1.8
2
2.2
2.4
I Global optimality for linear ϕ, γ experimentally shown.
Algorithm: iterative minimization of criterion e(ϕ, γ)
ϕ – motion (geometry mapping), γ – appearance mapping
I Iterative minimization:I initialization γ0 = randI ϕ1 = arg minϕ e(ϕ, γ0)I γ1 = arg minγ e(ϕ1, γ)I ϕ2 = arg minϕ e(ϕ, γ1)I γ2 = arg minγ e(ϕ2, γ)I ϕ3 = arg minϕ e(ϕ, γ2)I γ3 = arg minγ e(ϕ3, γ)I until convergence reached
color encodes criterion value e(ϕ, γ)
γϕ
0 2 4 6 8
0.8
1
1.2
1.4
1.6
1.8
2
2.2
2.4
I Global optimality for linear ϕ, γ experimentally shown.
Algorithm: iterative minimization of criterion e(ϕ, γ)
ϕ – motion (geometry mapping), γ – appearance mapping
I Iterative minimization:I initialization γ0 = randI ϕ1 = arg minϕ e(ϕ, γ0)I γ1 = arg minγ e(ϕ1, γ)I ϕ2 = arg minϕ e(ϕ, γ1)I γ2 = arg minγ e(ϕ2, γ)I ϕ3 = arg minϕ e(ϕ, γ2)I γ3 = arg minγ e(ϕ3, γ)I until convergence reached
color encodes criterion value e(ϕ, γ)
γϕ
[ϕ0, γ0]
0 2 4 6 8
0.8
1
1.2
1.4
1.6
1.8
2
2.2
2.4
I Global optimality for linear ϕ, γ experimentally shown.
Algorithm: iterative minimization of criterion e(ϕ, γ)
ϕ – motion (geometry mapping), γ – appearance mapping
I Iterative minimization:I initialization γ0 = randI ϕ1 = arg minϕ e(ϕ, γ0)I γ1 = arg minγ e(ϕ1, γ)I ϕ2 = arg minϕ e(ϕ, γ1)I γ2 = arg minγ e(ϕ2, γ)I ϕ3 = arg minϕ e(ϕ, γ2)I γ3 = arg minγ e(ϕ3, γ)I until convergence reached
color encodes criterion value e(ϕ, γ)
γϕ [ϕ1, γ0]
0 2 4 6 8
0.8
1
1.2
1.4
1.6
1.8
2
2.2
2.4
I Global optimality for linear ϕ, γ experimentally shown.
Algorithm: iterative minimization of criterion e(ϕ, γ)
ϕ – motion (geometry mapping), γ – appearance mapping
I Iterative minimization:I initialization γ0 = randI ϕ1 = arg minϕ e(ϕ, γ0)I γ1 = arg minγ e(ϕ1, γ)I ϕ2 = arg minϕ e(ϕ, γ1)I γ2 = arg minγ e(ϕ2, γ)I ϕ3 = arg minϕ e(ϕ, γ2)I γ3 = arg minγ e(ϕ3, γ)I until convergence reached
color encodes criterion value e(ϕ, γ)
γϕ [ϕ1, γ1]
0 2 4 6 8
0.8
1
1.2
1.4
1.6
1.8
2
2.2
2.4
I Global optimality for linear ϕ, γ experimentally shown.
Algorithm: iterative minimization of criterion e(ϕ, γ)
ϕ – motion (geometry mapping), γ – appearance mapping
I Iterative minimization:I initialization γ0 = randI ϕ1 = arg minϕ e(ϕ, γ0)I γ1 = arg minγ e(ϕ1, γ)I ϕ2 = arg minϕ e(ϕ, γ1)I γ2 = arg minγ e(ϕ2, γ)I ϕ3 = arg minϕ e(ϕ, γ2)I γ3 = arg minγ e(ϕ3, γ)I until convergence reached
color encodes criterion value e(ϕ, γ)
γϕ [ϕ2, γ1]
0 2 4 6 8
0.8
1
1.2
1.4
1.6
1.8
2
2.2
2.4
I Global optimality for linear ϕ, γ experimentally shown.
Algorithm: iterative minimization of criterion e(ϕ, γ)
ϕ – motion (geometry mapping), γ – appearance mapping
I Iterative minimization:I initialization γ0 = randI ϕ1 = arg minϕ e(ϕ, γ0)I γ1 = arg minγ e(ϕ1, γ)I ϕ2 = arg minϕ e(ϕ, γ1)I γ2 = arg minγ e(ϕ2, γ)I ϕ3 = arg minϕ e(ϕ, γ2)I γ3 = arg minγ e(ϕ3, γ)I until convergence reached
color encodes criterion value e(ϕ, γ)
γϕ [ϕ2, γ2]
0 2 4 6 8
0.8
1
1.2
1.4
1.6
1.8
2
2.2
2.4
I Global optimality for linear ϕ, γ experimentally shown.
Algorithm: iterative minimization of criterion e(ϕ, γ)
ϕ – motion (geometry mapping), γ – appearance mapping
I Iterative minimization:I initialization γ0 = randI ϕ1 = arg minϕ e(ϕ, γ0)I γ1 = arg minγ e(ϕ1, γ)I ϕ2 = arg minϕ e(ϕ, γ1)I γ2 = arg minγ e(ϕ2, γ)I ϕ3 = arg minϕ e(ϕ, γ2)I γ3 = arg minγ e(ϕ3, γ)I until convergence reached
color encodes criterion value e(ϕ, γ)
γϕ
[ϕ3, γ2]
0 2 4 6 8
0.8
1
1.2
1.4
1.6
1.8
2
2.2
2.4
I Global optimality for linear ϕ, γ experimentally shown.
Algorithm: iterative minimization of criterion e(ϕ, γ)
ϕ – motion (geometry mapping), γ – appearance mapping
I Iterative minimization:I initialization γ0 = randI ϕ1 = arg minϕ e(ϕ, γ0)I γ1 = arg minγ e(ϕ1, γ)I ϕ2 = arg minϕ e(ϕ, γ1)I γ2 = arg minγ e(ϕ2, γ)I ϕ3 = arg minϕ e(ϕ, γ2)I γ3 = arg minγ e(ϕ3, γ)I until convergence reached
color encodes criterion value e(ϕ, γ)
γϕ
[ϕ3, γ3]
0 2 4 6 8
0.8
1
1.2
1.4
1.6
1.8
2
2.2
2.4
I Global optimality for linear ϕ, γ experimentally shown.
Algorithm: iterative minimization of criterion e(ϕ, γ)
ϕ – motion (geometry mapping), γ – appearance mapping
I Iterative minimization:I initialization γ0 = randI ϕ1 = arg minϕ e(ϕ, γ0)I γ1 = arg minγ e(ϕ1, γ)I ϕ2 = arg minϕ e(ϕ, γ1)I γ2 = arg minγ e(ϕ2, γ)I ϕ3 = arg minϕ e(ϕ, γ2)I γ3 = arg minγ e(ϕ3, γ)I until convergence reached
color encodes criterion value e(ϕ, γ)
γϕ
[ϕ9, γ9]
0 2 4 6 8
0.8
1
1.2
1.4
1.6
1.8
2
2.2
2.4
I Global optimality for linear ϕ, γ experimentally shown.
Algorithm: iterative minimization of criterion e(ϕ, γ)
ϕ – motion (geometry mapping), γ – appearance mapping
I Iterative minimization:I initialization γ0 = randI ϕ1 = arg minϕ e(ϕ, γ0)I γ1 = arg minγ e(ϕ1, γ)I ϕ2 = arg minϕ e(ϕ, γ1)I γ2 = arg minγ e(ϕ2, γ)I ϕ3 = arg minϕ e(ϕ, γ2)I γ3 = arg minγ e(ϕ3, γ)I until convergence reached
color encodes criterion value e(ϕ, γ)
0 50 100 15020
25
30
35
40
45
50
iterationsM
SE
I Global optimality for linear ϕ, γ experimentally shown.
Algorithm: iterative minimization of criterion e(ϕ, γ)
ϕ – motion (geometry mapping), γ – appearance mapping
I Iterative minimization:I initialization γ0 = randI ϕ1 = arg minϕ e(ϕ, γ0)I γ1 = arg minγ e(ϕ1, γ)I ϕ2 = arg minϕ e(ϕ, γ1)I γ2 = arg minγ e(ϕ2, γ)I ϕ3 = arg minϕ e(ϕ, γ2)I γ3 = arg minγ e(ϕ3, γ)I until convergence reached
color encodes criterion value e(ϕ, γ)
0 5 10 15 20
22
23
24
25
26
27
28
29
30
31
32
iterationsM
SE
I Global optimality for linear ϕ, γ experimentally shown.
Simultaneous learning of motion and appearance
Experiments - videos II
Conclusions
I Learnable and very efficient tracking of objects with variableappearance.
I Accuracy, speed, robustness explicitly taken into account.
I Simultaneous learning motion and appearance.
Limitations
I Small, thin objects intractable.
Data, papers, various implemenations freely available athttp://cmp.felk.cvut.cz/demos/Tracking/linTrack/
Conclusions
I Learnable and very efficient tracking of objects with variableappearance.
I Accuracy, speed, robustness explicitly taken into account.
I Simultaneous learning motion and appearance.
Limitations
I Small, thin objects intractable.
Data, papers, various implemenations freely available athttp://cmp.felk.cvut.cz/demos/Tracking/linTrack/