background vs. foreground segmentation of video sequences = +
Post on 15-Jan-2016
226 views
TRANSCRIPT
Background vs foreground segmentation of video sequences
= +
The Problem
bull Separate video into two layersndash stationary backgroundndash moving foreground
bull Sequence is very noisy reference image (background) is not given
Simple approach (1)
temporal mean
background
temporal median
Simple approach (2)
threshold
Simple approach noise can spoil everything
Variational approach
Find the background and foregroundsimultaneously by minimizing energy functional
Bonus remove noise
Notations
[0t
max ]
N(xt) original noisy sequence
B(x) background image
C(xt) background mask(1 on background 0 on foreground)
give
n
need
to fi
nd
Energy functional data term
B N
B - N C
Energy functional data term
Degeneracy can be trivially minimized bybull C 0 (everything is foreground)
bull B N (take original image as background)
Energy functional data term
C 1
Energy functional data term
there should be enough of background
original images should be close to the restored background image
in the background areas
Energy functional smoothness
For background image B
For background mask C
Energy functional
Edge-preserving smoothnessRegularization term
Quadratic regularization [Tikhonov Arsenin 1977]
ELE
Known to produce very strong isotropic smoothing
Edge-preserving smoothnessRegularization term
Change regularization
ELE
Edge-preserving smoothnessRegularization term
ELE
Edge-preserving smoothnessRegularization term
ELE
n
Change the coordinate system
across the edgealong the edge
Compare
Edge-preserving smoothnessRegularization term
Weak edge (s 0)
Conditions on
Isotropic smoothings) is quadratic at zero
(s)
s
Edge-preserving smoothnessRegularization term
Strong edge (s )
Conditions on
bull no smoothing across the edge
bull more smoothing along the edge
Anisotropic smoothings) does not grow too fast at infinity
(s)
s
Edge-preserving smoothnessRegularization term
ConclusionUsing regularization term of the form
we can achieve both
isotropic smoothness in uniform regions
and anisotropic smoothness on edges
with one function
0 1 2 3 4 50
05
1
15
2
25
3
35
4
Edge-preserving smoothnessRegularization term
Example of an edge-preserving function
0 005 01 015 020
0005
001
0015
002
Edge-preserving smoothnessSpace of Bounded Variations
Even if we have an edge-preserving functional
if the space of solutions u contains only smooth functions we may not achieve the desired minimum
-1 -05 0 05 1-1
-05
0
05
1
Edge-preserving smoothnessSpace of Bounded Variations
which one is ldquobetterrdquo
Bounded Variation ndash ND caseBounded Variation ndash ND case
sup |)(| dxdivffD
)( x
N
1i i
i xdiv
1 || )( ) ( )(
101
L
NN C
bounded open subset function NR )( 1 Lf
Variation of over f
where
φ
Edge-preserving smoothnessSpace of Bounded Variations
integrable (absolute value) and with bounded variation
Functions are not required to have an integrable derivative hellip
What is the meaning of u in the regularization term
Intuitively norm of gradient |u| is
replaced with variation |Du|
Total variation
Theorem (informally) if u BV() then
Hausdorff measure
area gt 0area = 0
How can we measure zero-measure sets
Hausdorff measure
1) cover with balls of diameter
2) sum up diameters for optimal cover (do not waste balls)
3) refine 0
Hausdorff measureFormally
For A RN k-dimensional Hausdorff measure of A
up to normalization factor covers are countable
bull HN is just the Lebesgue measure
bull curve in image
its length = H1 in R2
Total variation
Theorem (more formally) if u BV() then
u+
u-
u(x)
xx0
u+ u- - approximate upper and lower limits
Su = x u+gtu-the jump set
Energy functional
data term
regularization for background image
regularization for background masks
Total variation example
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
= perimeter = 4
Divide each side into n parts
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation(= sum of perimeters)
Large total variation(= sum of perimeters)
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation Large total variation
Edge-preserving smoothnessSpace of Bounded Variations
BV informally functions with discontinuities on curves
Edges are preserved texture is not preserved
original sequencetemporal median energy minimization
in BV
Energy functional
Time-discretized problem
Find minimum of E subject to
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
The Problem
bull Separate video into two layersndash stationary backgroundndash moving foreground
bull Sequence is very noisy reference image (background) is not given
Simple approach (1)
temporal mean
background
temporal median
Simple approach (2)
threshold
Simple approach noise can spoil everything
Variational approach
Find the background and foregroundsimultaneously by minimizing energy functional
Bonus remove noise
Notations
[0t
max ]
N(xt) original noisy sequence
B(x) background image
C(xt) background mask(1 on background 0 on foreground)
give
n
need
to fi
nd
Energy functional data term
B N
B - N C
Energy functional data term
Degeneracy can be trivially minimized bybull C 0 (everything is foreground)
bull B N (take original image as background)
Energy functional data term
C 1
Energy functional data term
there should be enough of background
original images should be close to the restored background image
in the background areas
Energy functional smoothness
For background image B
For background mask C
Energy functional
Edge-preserving smoothnessRegularization term
Quadratic regularization [Tikhonov Arsenin 1977]
ELE
Known to produce very strong isotropic smoothing
Edge-preserving smoothnessRegularization term
Change regularization
ELE
Edge-preserving smoothnessRegularization term
ELE
Edge-preserving smoothnessRegularization term
ELE
n
Change the coordinate system
across the edgealong the edge
Compare
Edge-preserving smoothnessRegularization term
Weak edge (s 0)
Conditions on
Isotropic smoothings) is quadratic at zero
(s)
s
Edge-preserving smoothnessRegularization term
Strong edge (s )
Conditions on
bull no smoothing across the edge
bull more smoothing along the edge
Anisotropic smoothings) does not grow too fast at infinity
(s)
s
Edge-preserving smoothnessRegularization term
ConclusionUsing regularization term of the form
we can achieve both
isotropic smoothness in uniform regions
and anisotropic smoothness on edges
with one function
0 1 2 3 4 50
05
1
15
2
25
3
35
4
Edge-preserving smoothnessRegularization term
Example of an edge-preserving function
0 005 01 015 020
0005
001
0015
002
Edge-preserving smoothnessSpace of Bounded Variations
Even if we have an edge-preserving functional
if the space of solutions u contains only smooth functions we may not achieve the desired minimum
-1 -05 0 05 1-1
-05
0
05
1
Edge-preserving smoothnessSpace of Bounded Variations
which one is ldquobetterrdquo
Bounded Variation ndash ND caseBounded Variation ndash ND case
sup |)(| dxdivffD
)( x
N
1i i
i xdiv
1 || )( ) ( )(
101
L
NN C
bounded open subset function NR )( 1 Lf
Variation of over f
where
φ
Edge-preserving smoothnessSpace of Bounded Variations
integrable (absolute value) and with bounded variation
Functions are not required to have an integrable derivative hellip
What is the meaning of u in the regularization term
Intuitively norm of gradient |u| is
replaced with variation |Du|
Total variation
Theorem (informally) if u BV() then
Hausdorff measure
area gt 0area = 0
How can we measure zero-measure sets
Hausdorff measure
1) cover with balls of diameter
2) sum up diameters for optimal cover (do not waste balls)
3) refine 0
Hausdorff measureFormally
For A RN k-dimensional Hausdorff measure of A
up to normalization factor covers are countable
bull HN is just the Lebesgue measure
bull curve in image
its length = H1 in R2
Total variation
Theorem (more formally) if u BV() then
u+
u-
u(x)
xx0
u+ u- - approximate upper and lower limits
Su = x u+gtu-the jump set
Energy functional
data term
regularization for background image
regularization for background masks
Total variation example
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
= perimeter = 4
Divide each side into n parts
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation(= sum of perimeters)
Large total variation(= sum of perimeters)
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation Large total variation
Edge-preserving smoothnessSpace of Bounded Variations
BV informally functions with discontinuities on curves
Edges are preserved texture is not preserved
original sequencetemporal median energy minimization
in BV
Energy functional
Time-discretized problem
Find minimum of E subject to
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Simple approach (1)
temporal mean
background
temporal median
Simple approach (2)
threshold
Simple approach noise can spoil everything
Variational approach
Find the background and foregroundsimultaneously by minimizing energy functional
Bonus remove noise
Notations
[0t
max ]
N(xt) original noisy sequence
B(x) background image
C(xt) background mask(1 on background 0 on foreground)
give
n
need
to fi
nd
Energy functional data term
B N
B - N C
Energy functional data term
Degeneracy can be trivially minimized bybull C 0 (everything is foreground)
bull B N (take original image as background)
Energy functional data term
C 1
Energy functional data term
there should be enough of background
original images should be close to the restored background image
in the background areas
Energy functional smoothness
For background image B
For background mask C
Energy functional
Edge-preserving smoothnessRegularization term
Quadratic regularization [Tikhonov Arsenin 1977]
ELE
Known to produce very strong isotropic smoothing
Edge-preserving smoothnessRegularization term
Change regularization
ELE
Edge-preserving smoothnessRegularization term
ELE
Edge-preserving smoothnessRegularization term
ELE
n
Change the coordinate system
across the edgealong the edge
Compare
Edge-preserving smoothnessRegularization term
Weak edge (s 0)
Conditions on
Isotropic smoothings) is quadratic at zero
(s)
s
Edge-preserving smoothnessRegularization term
Strong edge (s )
Conditions on
bull no smoothing across the edge
bull more smoothing along the edge
Anisotropic smoothings) does not grow too fast at infinity
(s)
s
Edge-preserving smoothnessRegularization term
ConclusionUsing regularization term of the form
we can achieve both
isotropic smoothness in uniform regions
and anisotropic smoothness on edges
with one function
0 1 2 3 4 50
05
1
15
2
25
3
35
4
Edge-preserving smoothnessRegularization term
Example of an edge-preserving function
0 005 01 015 020
0005
001
0015
002
Edge-preserving smoothnessSpace of Bounded Variations
Even if we have an edge-preserving functional
if the space of solutions u contains only smooth functions we may not achieve the desired minimum
-1 -05 0 05 1-1
-05
0
05
1
Edge-preserving smoothnessSpace of Bounded Variations
which one is ldquobetterrdquo
Bounded Variation ndash ND caseBounded Variation ndash ND case
sup |)(| dxdivffD
)( x
N
1i i
i xdiv
1 || )( ) ( )(
101
L
NN C
bounded open subset function NR )( 1 Lf
Variation of over f
where
φ
Edge-preserving smoothnessSpace of Bounded Variations
integrable (absolute value) and with bounded variation
Functions are not required to have an integrable derivative hellip
What is the meaning of u in the regularization term
Intuitively norm of gradient |u| is
replaced with variation |Du|
Total variation
Theorem (informally) if u BV() then
Hausdorff measure
area gt 0area = 0
How can we measure zero-measure sets
Hausdorff measure
1) cover with balls of diameter
2) sum up diameters for optimal cover (do not waste balls)
3) refine 0
Hausdorff measureFormally
For A RN k-dimensional Hausdorff measure of A
up to normalization factor covers are countable
bull HN is just the Lebesgue measure
bull curve in image
its length = H1 in R2
Total variation
Theorem (more formally) if u BV() then
u+
u-
u(x)
xx0
u+ u- - approximate upper and lower limits
Su = x u+gtu-the jump set
Energy functional
data term
regularization for background image
regularization for background masks
Total variation example
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
= perimeter = 4
Divide each side into n parts
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation(= sum of perimeters)
Large total variation(= sum of perimeters)
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation Large total variation
Edge-preserving smoothnessSpace of Bounded Variations
BV informally functions with discontinuities on curves
Edges are preserved texture is not preserved
original sequencetemporal median energy minimization
in BV
Energy functional
Time-discretized problem
Find minimum of E subject to
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Simple approach (2)
threshold
Simple approach noise can spoil everything
Variational approach
Find the background and foregroundsimultaneously by minimizing energy functional
Bonus remove noise
Notations
[0t
max ]
N(xt) original noisy sequence
B(x) background image
C(xt) background mask(1 on background 0 on foreground)
give
n
need
to fi
nd
Energy functional data term
B N
B - N C
Energy functional data term
Degeneracy can be trivially minimized bybull C 0 (everything is foreground)
bull B N (take original image as background)
Energy functional data term
C 1
Energy functional data term
there should be enough of background
original images should be close to the restored background image
in the background areas
Energy functional smoothness
For background image B
For background mask C
Energy functional
Edge-preserving smoothnessRegularization term
Quadratic regularization [Tikhonov Arsenin 1977]
ELE
Known to produce very strong isotropic smoothing
Edge-preserving smoothnessRegularization term
Change regularization
ELE
Edge-preserving smoothnessRegularization term
ELE
Edge-preserving smoothnessRegularization term
ELE
n
Change the coordinate system
across the edgealong the edge
Compare
Edge-preserving smoothnessRegularization term
Weak edge (s 0)
Conditions on
Isotropic smoothings) is quadratic at zero
(s)
s
Edge-preserving smoothnessRegularization term
Strong edge (s )
Conditions on
bull no smoothing across the edge
bull more smoothing along the edge
Anisotropic smoothings) does not grow too fast at infinity
(s)
s
Edge-preserving smoothnessRegularization term
ConclusionUsing regularization term of the form
we can achieve both
isotropic smoothness in uniform regions
and anisotropic smoothness on edges
with one function
0 1 2 3 4 50
05
1
15
2
25
3
35
4
Edge-preserving smoothnessRegularization term
Example of an edge-preserving function
0 005 01 015 020
0005
001
0015
002
Edge-preserving smoothnessSpace of Bounded Variations
Even if we have an edge-preserving functional
if the space of solutions u contains only smooth functions we may not achieve the desired minimum
-1 -05 0 05 1-1
-05
0
05
1
Edge-preserving smoothnessSpace of Bounded Variations
which one is ldquobetterrdquo
Bounded Variation ndash ND caseBounded Variation ndash ND case
sup |)(| dxdivffD
)( x
N
1i i
i xdiv
1 || )( ) ( )(
101
L
NN C
bounded open subset function NR )( 1 Lf
Variation of over f
where
φ
Edge-preserving smoothnessSpace of Bounded Variations
integrable (absolute value) and with bounded variation
Functions are not required to have an integrable derivative hellip
What is the meaning of u in the regularization term
Intuitively norm of gradient |u| is
replaced with variation |Du|
Total variation
Theorem (informally) if u BV() then
Hausdorff measure
area gt 0area = 0
How can we measure zero-measure sets
Hausdorff measure
1) cover with balls of diameter
2) sum up diameters for optimal cover (do not waste balls)
3) refine 0
Hausdorff measureFormally
For A RN k-dimensional Hausdorff measure of A
up to normalization factor covers are countable
bull HN is just the Lebesgue measure
bull curve in image
its length = H1 in R2
Total variation
Theorem (more formally) if u BV() then
u+
u-
u(x)
xx0
u+ u- - approximate upper and lower limits
Su = x u+gtu-the jump set
Energy functional
data term
regularization for background image
regularization for background masks
Total variation example
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
= perimeter = 4
Divide each side into n parts
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation(= sum of perimeters)
Large total variation(= sum of perimeters)
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation Large total variation
Edge-preserving smoothnessSpace of Bounded Variations
BV informally functions with discontinuities on curves
Edges are preserved texture is not preserved
original sequencetemporal median energy minimization
in BV
Energy functional
Time-discretized problem
Find minimum of E subject to
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Simple approach noise can spoil everything
Variational approach
Find the background and foregroundsimultaneously by minimizing energy functional
Bonus remove noise
Notations
[0t
max ]
N(xt) original noisy sequence
B(x) background image
C(xt) background mask(1 on background 0 on foreground)
give
n
need
to fi
nd
Energy functional data term
B N
B - N C
Energy functional data term
Degeneracy can be trivially minimized bybull C 0 (everything is foreground)
bull B N (take original image as background)
Energy functional data term
C 1
Energy functional data term
there should be enough of background
original images should be close to the restored background image
in the background areas
Energy functional smoothness
For background image B
For background mask C
Energy functional
Edge-preserving smoothnessRegularization term
Quadratic regularization [Tikhonov Arsenin 1977]
ELE
Known to produce very strong isotropic smoothing
Edge-preserving smoothnessRegularization term
Change regularization
ELE
Edge-preserving smoothnessRegularization term
ELE
Edge-preserving smoothnessRegularization term
ELE
n
Change the coordinate system
across the edgealong the edge
Compare
Edge-preserving smoothnessRegularization term
Weak edge (s 0)
Conditions on
Isotropic smoothings) is quadratic at zero
(s)
s
Edge-preserving smoothnessRegularization term
Strong edge (s )
Conditions on
bull no smoothing across the edge
bull more smoothing along the edge
Anisotropic smoothings) does not grow too fast at infinity
(s)
s
Edge-preserving smoothnessRegularization term
ConclusionUsing regularization term of the form
we can achieve both
isotropic smoothness in uniform regions
and anisotropic smoothness on edges
with one function
0 1 2 3 4 50
05
1
15
2
25
3
35
4
Edge-preserving smoothnessRegularization term
Example of an edge-preserving function
0 005 01 015 020
0005
001
0015
002
Edge-preserving smoothnessSpace of Bounded Variations
Even if we have an edge-preserving functional
if the space of solutions u contains only smooth functions we may not achieve the desired minimum
-1 -05 0 05 1-1
-05
0
05
1
Edge-preserving smoothnessSpace of Bounded Variations
which one is ldquobetterrdquo
Bounded Variation ndash ND caseBounded Variation ndash ND case
sup |)(| dxdivffD
)( x
N
1i i
i xdiv
1 || )( ) ( )(
101
L
NN C
bounded open subset function NR )( 1 Lf
Variation of over f
where
φ
Edge-preserving smoothnessSpace of Bounded Variations
integrable (absolute value) and with bounded variation
Functions are not required to have an integrable derivative hellip
What is the meaning of u in the regularization term
Intuitively norm of gradient |u| is
replaced with variation |Du|
Total variation
Theorem (informally) if u BV() then
Hausdorff measure
area gt 0area = 0
How can we measure zero-measure sets
Hausdorff measure
1) cover with balls of diameter
2) sum up diameters for optimal cover (do not waste balls)
3) refine 0
Hausdorff measureFormally
For A RN k-dimensional Hausdorff measure of A
up to normalization factor covers are countable
bull HN is just the Lebesgue measure
bull curve in image
its length = H1 in R2
Total variation
Theorem (more formally) if u BV() then
u+
u-
u(x)
xx0
u+ u- - approximate upper and lower limits
Su = x u+gtu-the jump set
Energy functional
data term
regularization for background image
regularization for background masks
Total variation example
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
= perimeter = 4
Divide each side into n parts
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation(= sum of perimeters)
Large total variation(= sum of perimeters)
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation Large total variation
Edge-preserving smoothnessSpace of Bounded Variations
BV informally functions with discontinuities on curves
Edges are preserved texture is not preserved
original sequencetemporal median energy minimization
in BV
Energy functional
Time-discretized problem
Find minimum of E subject to
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Variational approach
Find the background and foregroundsimultaneously by minimizing energy functional
Bonus remove noise
Notations
[0t
max ]
N(xt) original noisy sequence
B(x) background image
C(xt) background mask(1 on background 0 on foreground)
give
n
need
to fi
nd
Energy functional data term
B N
B - N C
Energy functional data term
Degeneracy can be trivially minimized bybull C 0 (everything is foreground)
bull B N (take original image as background)
Energy functional data term
C 1
Energy functional data term
there should be enough of background
original images should be close to the restored background image
in the background areas
Energy functional smoothness
For background image B
For background mask C
Energy functional
Edge-preserving smoothnessRegularization term
Quadratic regularization [Tikhonov Arsenin 1977]
ELE
Known to produce very strong isotropic smoothing
Edge-preserving smoothnessRegularization term
Change regularization
ELE
Edge-preserving smoothnessRegularization term
ELE
Edge-preserving smoothnessRegularization term
ELE
n
Change the coordinate system
across the edgealong the edge
Compare
Edge-preserving smoothnessRegularization term
Weak edge (s 0)
Conditions on
Isotropic smoothings) is quadratic at zero
(s)
s
Edge-preserving smoothnessRegularization term
Strong edge (s )
Conditions on
bull no smoothing across the edge
bull more smoothing along the edge
Anisotropic smoothings) does not grow too fast at infinity
(s)
s
Edge-preserving smoothnessRegularization term
ConclusionUsing regularization term of the form
we can achieve both
isotropic smoothness in uniform regions
and anisotropic smoothness on edges
with one function
0 1 2 3 4 50
05
1
15
2
25
3
35
4
Edge-preserving smoothnessRegularization term
Example of an edge-preserving function
0 005 01 015 020
0005
001
0015
002
Edge-preserving smoothnessSpace of Bounded Variations
Even if we have an edge-preserving functional
if the space of solutions u contains only smooth functions we may not achieve the desired minimum
-1 -05 0 05 1-1
-05
0
05
1
Edge-preserving smoothnessSpace of Bounded Variations
which one is ldquobetterrdquo
Bounded Variation ndash ND caseBounded Variation ndash ND case
sup |)(| dxdivffD
)( x
N
1i i
i xdiv
1 || )( ) ( )(
101
L
NN C
bounded open subset function NR )( 1 Lf
Variation of over f
where
φ
Edge-preserving smoothnessSpace of Bounded Variations
integrable (absolute value) and with bounded variation
Functions are not required to have an integrable derivative hellip
What is the meaning of u in the regularization term
Intuitively norm of gradient |u| is
replaced with variation |Du|
Total variation
Theorem (informally) if u BV() then
Hausdorff measure
area gt 0area = 0
How can we measure zero-measure sets
Hausdorff measure
1) cover with balls of diameter
2) sum up diameters for optimal cover (do not waste balls)
3) refine 0
Hausdorff measureFormally
For A RN k-dimensional Hausdorff measure of A
up to normalization factor covers are countable
bull HN is just the Lebesgue measure
bull curve in image
its length = H1 in R2
Total variation
Theorem (more formally) if u BV() then
u+
u-
u(x)
xx0
u+ u- - approximate upper and lower limits
Su = x u+gtu-the jump set
Energy functional
data term
regularization for background image
regularization for background masks
Total variation example
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
= perimeter = 4
Divide each side into n parts
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation(= sum of perimeters)
Large total variation(= sum of perimeters)
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation Large total variation
Edge-preserving smoothnessSpace of Bounded Variations
BV informally functions with discontinuities on curves
Edges are preserved texture is not preserved
original sequencetemporal median energy minimization
in BV
Energy functional
Time-discretized problem
Find minimum of E subject to
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Notations
[0t
max ]
N(xt) original noisy sequence
B(x) background image
C(xt) background mask(1 on background 0 on foreground)
give
n
need
to fi
nd
Energy functional data term
B N
B - N C
Energy functional data term
Degeneracy can be trivially minimized bybull C 0 (everything is foreground)
bull B N (take original image as background)
Energy functional data term
C 1
Energy functional data term
there should be enough of background
original images should be close to the restored background image
in the background areas
Energy functional smoothness
For background image B
For background mask C
Energy functional
Edge-preserving smoothnessRegularization term
Quadratic regularization [Tikhonov Arsenin 1977]
ELE
Known to produce very strong isotropic smoothing
Edge-preserving smoothnessRegularization term
Change regularization
ELE
Edge-preserving smoothnessRegularization term
ELE
Edge-preserving smoothnessRegularization term
ELE
n
Change the coordinate system
across the edgealong the edge
Compare
Edge-preserving smoothnessRegularization term
Weak edge (s 0)
Conditions on
Isotropic smoothings) is quadratic at zero
(s)
s
Edge-preserving smoothnessRegularization term
Strong edge (s )
Conditions on
bull no smoothing across the edge
bull more smoothing along the edge
Anisotropic smoothings) does not grow too fast at infinity
(s)
s
Edge-preserving smoothnessRegularization term
ConclusionUsing regularization term of the form
we can achieve both
isotropic smoothness in uniform regions
and anisotropic smoothness on edges
with one function
0 1 2 3 4 50
05
1
15
2
25
3
35
4
Edge-preserving smoothnessRegularization term
Example of an edge-preserving function
0 005 01 015 020
0005
001
0015
002
Edge-preserving smoothnessSpace of Bounded Variations
Even if we have an edge-preserving functional
if the space of solutions u contains only smooth functions we may not achieve the desired minimum
-1 -05 0 05 1-1
-05
0
05
1
Edge-preserving smoothnessSpace of Bounded Variations
which one is ldquobetterrdquo
Bounded Variation ndash ND caseBounded Variation ndash ND case
sup |)(| dxdivffD
)( x
N
1i i
i xdiv
1 || )( ) ( )(
101
L
NN C
bounded open subset function NR )( 1 Lf
Variation of over f
where
φ
Edge-preserving smoothnessSpace of Bounded Variations
integrable (absolute value) and with bounded variation
Functions are not required to have an integrable derivative hellip
What is the meaning of u in the regularization term
Intuitively norm of gradient |u| is
replaced with variation |Du|
Total variation
Theorem (informally) if u BV() then
Hausdorff measure
area gt 0area = 0
How can we measure zero-measure sets
Hausdorff measure
1) cover with balls of diameter
2) sum up diameters for optimal cover (do not waste balls)
3) refine 0
Hausdorff measureFormally
For A RN k-dimensional Hausdorff measure of A
up to normalization factor covers are countable
bull HN is just the Lebesgue measure
bull curve in image
its length = H1 in R2
Total variation
Theorem (more formally) if u BV() then
u+
u-
u(x)
xx0
u+ u- - approximate upper and lower limits
Su = x u+gtu-the jump set
Energy functional
data term
regularization for background image
regularization for background masks
Total variation example
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
= perimeter = 4
Divide each side into n parts
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation(= sum of perimeters)
Large total variation(= sum of perimeters)
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation Large total variation
Edge-preserving smoothnessSpace of Bounded Variations
BV informally functions with discontinuities on curves
Edges are preserved texture is not preserved
original sequencetemporal median energy minimization
in BV
Energy functional
Time-discretized problem
Find minimum of E subject to
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Energy functional data term
B N
B - N C
Energy functional data term
Degeneracy can be trivially minimized bybull C 0 (everything is foreground)
bull B N (take original image as background)
Energy functional data term
C 1
Energy functional data term
there should be enough of background
original images should be close to the restored background image
in the background areas
Energy functional smoothness
For background image B
For background mask C
Energy functional
Edge-preserving smoothnessRegularization term
Quadratic regularization [Tikhonov Arsenin 1977]
ELE
Known to produce very strong isotropic smoothing
Edge-preserving smoothnessRegularization term
Change regularization
ELE
Edge-preserving smoothnessRegularization term
ELE
Edge-preserving smoothnessRegularization term
ELE
n
Change the coordinate system
across the edgealong the edge
Compare
Edge-preserving smoothnessRegularization term
Weak edge (s 0)
Conditions on
Isotropic smoothings) is quadratic at zero
(s)
s
Edge-preserving smoothnessRegularization term
Strong edge (s )
Conditions on
bull no smoothing across the edge
bull more smoothing along the edge
Anisotropic smoothings) does not grow too fast at infinity
(s)
s
Edge-preserving smoothnessRegularization term
ConclusionUsing regularization term of the form
we can achieve both
isotropic smoothness in uniform regions
and anisotropic smoothness on edges
with one function
0 1 2 3 4 50
05
1
15
2
25
3
35
4
Edge-preserving smoothnessRegularization term
Example of an edge-preserving function
0 005 01 015 020
0005
001
0015
002
Edge-preserving smoothnessSpace of Bounded Variations
Even if we have an edge-preserving functional
if the space of solutions u contains only smooth functions we may not achieve the desired minimum
-1 -05 0 05 1-1
-05
0
05
1
Edge-preserving smoothnessSpace of Bounded Variations
which one is ldquobetterrdquo
Bounded Variation ndash ND caseBounded Variation ndash ND case
sup |)(| dxdivffD
)( x
N
1i i
i xdiv
1 || )( ) ( )(
101
L
NN C
bounded open subset function NR )( 1 Lf
Variation of over f
where
φ
Edge-preserving smoothnessSpace of Bounded Variations
integrable (absolute value) and with bounded variation
Functions are not required to have an integrable derivative hellip
What is the meaning of u in the regularization term
Intuitively norm of gradient |u| is
replaced with variation |Du|
Total variation
Theorem (informally) if u BV() then
Hausdorff measure
area gt 0area = 0
How can we measure zero-measure sets
Hausdorff measure
1) cover with balls of diameter
2) sum up diameters for optimal cover (do not waste balls)
3) refine 0
Hausdorff measureFormally
For A RN k-dimensional Hausdorff measure of A
up to normalization factor covers are countable
bull HN is just the Lebesgue measure
bull curve in image
its length = H1 in R2
Total variation
Theorem (more formally) if u BV() then
u+
u-
u(x)
xx0
u+ u- - approximate upper and lower limits
Su = x u+gtu-the jump set
Energy functional
data term
regularization for background image
regularization for background masks
Total variation example
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
= perimeter = 4
Divide each side into n parts
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation(= sum of perimeters)
Large total variation(= sum of perimeters)
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation Large total variation
Edge-preserving smoothnessSpace of Bounded Variations
BV informally functions with discontinuities on curves
Edges are preserved texture is not preserved
original sequencetemporal median energy minimization
in BV
Energy functional
Time-discretized problem
Find minimum of E subject to
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Energy functional data term
Degeneracy can be trivially minimized bybull C 0 (everything is foreground)
bull B N (take original image as background)
Energy functional data term
C 1
Energy functional data term
there should be enough of background
original images should be close to the restored background image
in the background areas
Energy functional smoothness
For background image B
For background mask C
Energy functional
Edge-preserving smoothnessRegularization term
Quadratic regularization [Tikhonov Arsenin 1977]
ELE
Known to produce very strong isotropic smoothing
Edge-preserving smoothnessRegularization term
Change regularization
ELE
Edge-preserving smoothnessRegularization term
ELE
Edge-preserving smoothnessRegularization term
ELE
n
Change the coordinate system
across the edgealong the edge
Compare
Edge-preserving smoothnessRegularization term
Weak edge (s 0)
Conditions on
Isotropic smoothings) is quadratic at zero
(s)
s
Edge-preserving smoothnessRegularization term
Strong edge (s )
Conditions on
bull no smoothing across the edge
bull more smoothing along the edge
Anisotropic smoothings) does not grow too fast at infinity
(s)
s
Edge-preserving smoothnessRegularization term
ConclusionUsing regularization term of the form
we can achieve both
isotropic smoothness in uniform regions
and anisotropic smoothness on edges
with one function
0 1 2 3 4 50
05
1
15
2
25
3
35
4
Edge-preserving smoothnessRegularization term
Example of an edge-preserving function
0 005 01 015 020
0005
001
0015
002
Edge-preserving smoothnessSpace of Bounded Variations
Even if we have an edge-preserving functional
if the space of solutions u contains only smooth functions we may not achieve the desired minimum
-1 -05 0 05 1-1
-05
0
05
1
Edge-preserving smoothnessSpace of Bounded Variations
which one is ldquobetterrdquo
Bounded Variation ndash ND caseBounded Variation ndash ND case
sup |)(| dxdivffD
)( x
N
1i i
i xdiv
1 || )( ) ( )(
101
L
NN C
bounded open subset function NR )( 1 Lf
Variation of over f
where
φ
Edge-preserving smoothnessSpace of Bounded Variations
integrable (absolute value) and with bounded variation
Functions are not required to have an integrable derivative hellip
What is the meaning of u in the regularization term
Intuitively norm of gradient |u| is
replaced with variation |Du|
Total variation
Theorem (informally) if u BV() then
Hausdorff measure
area gt 0area = 0
How can we measure zero-measure sets
Hausdorff measure
1) cover with balls of diameter
2) sum up diameters for optimal cover (do not waste balls)
3) refine 0
Hausdorff measureFormally
For A RN k-dimensional Hausdorff measure of A
up to normalization factor covers are countable
bull HN is just the Lebesgue measure
bull curve in image
its length = H1 in R2
Total variation
Theorem (more formally) if u BV() then
u+
u-
u(x)
xx0
u+ u- - approximate upper and lower limits
Su = x u+gtu-the jump set
Energy functional
data term
regularization for background image
regularization for background masks
Total variation example
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
= perimeter = 4
Divide each side into n parts
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation(= sum of perimeters)
Large total variation(= sum of perimeters)
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation Large total variation
Edge-preserving smoothnessSpace of Bounded Variations
BV informally functions with discontinuities on curves
Edges are preserved texture is not preserved
original sequencetemporal median energy minimization
in BV
Energy functional
Time-discretized problem
Find minimum of E subject to
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Energy functional data term
C 1
Energy functional data term
there should be enough of background
original images should be close to the restored background image
in the background areas
Energy functional smoothness
For background image B
For background mask C
Energy functional
Edge-preserving smoothnessRegularization term
Quadratic regularization [Tikhonov Arsenin 1977]
ELE
Known to produce very strong isotropic smoothing
Edge-preserving smoothnessRegularization term
Change regularization
ELE
Edge-preserving smoothnessRegularization term
ELE
Edge-preserving smoothnessRegularization term
ELE
n
Change the coordinate system
across the edgealong the edge
Compare
Edge-preserving smoothnessRegularization term
Weak edge (s 0)
Conditions on
Isotropic smoothings) is quadratic at zero
(s)
s
Edge-preserving smoothnessRegularization term
Strong edge (s )
Conditions on
bull no smoothing across the edge
bull more smoothing along the edge
Anisotropic smoothings) does not grow too fast at infinity
(s)
s
Edge-preserving smoothnessRegularization term
ConclusionUsing regularization term of the form
we can achieve both
isotropic smoothness in uniform regions
and anisotropic smoothness on edges
with one function
0 1 2 3 4 50
05
1
15
2
25
3
35
4
Edge-preserving smoothnessRegularization term
Example of an edge-preserving function
0 005 01 015 020
0005
001
0015
002
Edge-preserving smoothnessSpace of Bounded Variations
Even if we have an edge-preserving functional
if the space of solutions u contains only smooth functions we may not achieve the desired minimum
-1 -05 0 05 1-1
-05
0
05
1
Edge-preserving smoothnessSpace of Bounded Variations
which one is ldquobetterrdquo
Bounded Variation ndash ND caseBounded Variation ndash ND case
sup |)(| dxdivffD
)( x
N
1i i
i xdiv
1 || )( ) ( )(
101
L
NN C
bounded open subset function NR )( 1 Lf
Variation of over f
where
φ
Edge-preserving smoothnessSpace of Bounded Variations
integrable (absolute value) and with bounded variation
Functions are not required to have an integrable derivative hellip
What is the meaning of u in the regularization term
Intuitively norm of gradient |u| is
replaced with variation |Du|
Total variation
Theorem (informally) if u BV() then
Hausdorff measure
area gt 0area = 0
How can we measure zero-measure sets
Hausdorff measure
1) cover with balls of diameter
2) sum up diameters for optimal cover (do not waste balls)
3) refine 0
Hausdorff measureFormally
For A RN k-dimensional Hausdorff measure of A
up to normalization factor covers are countable
bull HN is just the Lebesgue measure
bull curve in image
its length = H1 in R2
Total variation
Theorem (more formally) if u BV() then
u+
u-
u(x)
xx0
u+ u- - approximate upper and lower limits
Su = x u+gtu-the jump set
Energy functional
data term
regularization for background image
regularization for background masks
Total variation example
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
= perimeter = 4
Divide each side into n parts
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation(= sum of perimeters)
Large total variation(= sum of perimeters)
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation Large total variation
Edge-preserving smoothnessSpace of Bounded Variations
BV informally functions with discontinuities on curves
Edges are preserved texture is not preserved
original sequencetemporal median energy minimization
in BV
Energy functional
Time-discretized problem
Find minimum of E subject to
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Energy functional data term
there should be enough of background
original images should be close to the restored background image
in the background areas
Energy functional smoothness
For background image B
For background mask C
Energy functional
Edge-preserving smoothnessRegularization term
Quadratic regularization [Tikhonov Arsenin 1977]
ELE
Known to produce very strong isotropic smoothing
Edge-preserving smoothnessRegularization term
Change regularization
ELE
Edge-preserving smoothnessRegularization term
ELE
Edge-preserving smoothnessRegularization term
ELE
n
Change the coordinate system
across the edgealong the edge
Compare
Edge-preserving smoothnessRegularization term
Weak edge (s 0)
Conditions on
Isotropic smoothings) is quadratic at zero
(s)
s
Edge-preserving smoothnessRegularization term
Strong edge (s )
Conditions on
bull no smoothing across the edge
bull more smoothing along the edge
Anisotropic smoothings) does not grow too fast at infinity
(s)
s
Edge-preserving smoothnessRegularization term
ConclusionUsing regularization term of the form
we can achieve both
isotropic smoothness in uniform regions
and anisotropic smoothness on edges
with one function
0 1 2 3 4 50
05
1
15
2
25
3
35
4
Edge-preserving smoothnessRegularization term
Example of an edge-preserving function
0 005 01 015 020
0005
001
0015
002
Edge-preserving smoothnessSpace of Bounded Variations
Even if we have an edge-preserving functional
if the space of solutions u contains only smooth functions we may not achieve the desired minimum
-1 -05 0 05 1-1
-05
0
05
1
Edge-preserving smoothnessSpace of Bounded Variations
which one is ldquobetterrdquo
Bounded Variation ndash ND caseBounded Variation ndash ND case
sup |)(| dxdivffD
)( x
N
1i i
i xdiv
1 || )( ) ( )(
101
L
NN C
bounded open subset function NR )( 1 Lf
Variation of over f
where
φ
Edge-preserving smoothnessSpace of Bounded Variations
integrable (absolute value) and with bounded variation
Functions are not required to have an integrable derivative hellip
What is the meaning of u in the regularization term
Intuitively norm of gradient |u| is
replaced with variation |Du|
Total variation
Theorem (informally) if u BV() then
Hausdorff measure
area gt 0area = 0
How can we measure zero-measure sets
Hausdorff measure
1) cover with balls of diameter
2) sum up diameters for optimal cover (do not waste balls)
3) refine 0
Hausdorff measureFormally
For A RN k-dimensional Hausdorff measure of A
up to normalization factor covers are countable
bull HN is just the Lebesgue measure
bull curve in image
its length = H1 in R2
Total variation
Theorem (more formally) if u BV() then
u+
u-
u(x)
xx0
u+ u- - approximate upper and lower limits
Su = x u+gtu-the jump set
Energy functional
data term
regularization for background image
regularization for background masks
Total variation example
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
= perimeter = 4
Divide each side into n parts
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation(= sum of perimeters)
Large total variation(= sum of perimeters)
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation Large total variation
Edge-preserving smoothnessSpace of Bounded Variations
BV informally functions with discontinuities on curves
Edges are preserved texture is not preserved
original sequencetemporal median energy minimization
in BV
Energy functional
Time-discretized problem
Find minimum of E subject to
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Energy functional smoothness
For background image B
For background mask C
Energy functional
Edge-preserving smoothnessRegularization term
Quadratic regularization [Tikhonov Arsenin 1977]
ELE
Known to produce very strong isotropic smoothing
Edge-preserving smoothnessRegularization term
Change regularization
ELE
Edge-preserving smoothnessRegularization term
ELE
Edge-preserving smoothnessRegularization term
ELE
n
Change the coordinate system
across the edgealong the edge
Compare
Edge-preserving smoothnessRegularization term
Weak edge (s 0)
Conditions on
Isotropic smoothings) is quadratic at zero
(s)
s
Edge-preserving smoothnessRegularization term
Strong edge (s )
Conditions on
bull no smoothing across the edge
bull more smoothing along the edge
Anisotropic smoothings) does not grow too fast at infinity
(s)
s
Edge-preserving smoothnessRegularization term
ConclusionUsing regularization term of the form
we can achieve both
isotropic smoothness in uniform regions
and anisotropic smoothness on edges
with one function
0 1 2 3 4 50
05
1
15
2
25
3
35
4
Edge-preserving smoothnessRegularization term
Example of an edge-preserving function
0 005 01 015 020
0005
001
0015
002
Edge-preserving smoothnessSpace of Bounded Variations
Even if we have an edge-preserving functional
if the space of solutions u contains only smooth functions we may not achieve the desired minimum
-1 -05 0 05 1-1
-05
0
05
1
Edge-preserving smoothnessSpace of Bounded Variations
which one is ldquobetterrdquo
Bounded Variation ndash ND caseBounded Variation ndash ND case
sup |)(| dxdivffD
)( x
N
1i i
i xdiv
1 || )( ) ( )(
101
L
NN C
bounded open subset function NR )( 1 Lf
Variation of over f
where
φ
Edge-preserving smoothnessSpace of Bounded Variations
integrable (absolute value) and with bounded variation
Functions are not required to have an integrable derivative hellip
What is the meaning of u in the regularization term
Intuitively norm of gradient |u| is
replaced with variation |Du|
Total variation
Theorem (informally) if u BV() then
Hausdorff measure
area gt 0area = 0
How can we measure zero-measure sets
Hausdorff measure
1) cover with balls of diameter
2) sum up diameters for optimal cover (do not waste balls)
3) refine 0
Hausdorff measureFormally
For A RN k-dimensional Hausdorff measure of A
up to normalization factor covers are countable
bull HN is just the Lebesgue measure
bull curve in image
its length = H1 in R2
Total variation
Theorem (more formally) if u BV() then
u+
u-
u(x)
xx0
u+ u- - approximate upper and lower limits
Su = x u+gtu-the jump set
Energy functional
data term
regularization for background image
regularization for background masks
Total variation example
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
= perimeter = 4
Divide each side into n parts
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation(= sum of perimeters)
Large total variation(= sum of perimeters)
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation Large total variation
Edge-preserving smoothnessSpace of Bounded Variations
BV informally functions with discontinuities on curves
Edges are preserved texture is not preserved
original sequencetemporal median energy minimization
in BV
Energy functional
Time-discretized problem
Find minimum of E subject to
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Energy functional
Edge-preserving smoothnessRegularization term
Quadratic regularization [Tikhonov Arsenin 1977]
ELE
Known to produce very strong isotropic smoothing
Edge-preserving smoothnessRegularization term
Change regularization
ELE
Edge-preserving smoothnessRegularization term
ELE
Edge-preserving smoothnessRegularization term
ELE
n
Change the coordinate system
across the edgealong the edge
Compare
Edge-preserving smoothnessRegularization term
Weak edge (s 0)
Conditions on
Isotropic smoothings) is quadratic at zero
(s)
s
Edge-preserving smoothnessRegularization term
Strong edge (s )
Conditions on
bull no smoothing across the edge
bull more smoothing along the edge
Anisotropic smoothings) does not grow too fast at infinity
(s)
s
Edge-preserving smoothnessRegularization term
ConclusionUsing regularization term of the form
we can achieve both
isotropic smoothness in uniform regions
and anisotropic smoothness on edges
with one function
0 1 2 3 4 50
05
1
15
2
25
3
35
4
Edge-preserving smoothnessRegularization term
Example of an edge-preserving function
0 005 01 015 020
0005
001
0015
002
Edge-preserving smoothnessSpace of Bounded Variations
Even if we have an edge-preserving functional
if the space of solutions u contains only smooth functions we may not achieve the desired minimum
-1 -05 0 05 1-1
-05
0
05
1
Edge-preserving smoothnessSpace of Bounded Variations
which one is ldquobetterrdquo
Bounded Variation ndash ND caseBounded Variation ndash ND case
sup |)(| dxdivffD
)( x
N
1i i
i xdiv
1 || )( ) ( )(
101
L
NN C
bounded open subset function NR )( 1 Lf
Variation of over f
where
φ
Edge-preserving smoothnessSpace of Bounded Variations
integrable (absolute value) and with bounded variation
Functions are not required to have an integrable derivative hellip
What is the meaning of u in the regularization term
Intuitively norm of gradient |u| is
replaced with variation |Du|
Total variation
Theorem (informally) if u BV() then
Hausdorff measure
area gt 0area = 0
How can we measure zero-measure sets
Hausdorff measure
1) cover with balls of diameter
2) sum up diameters for optimal cover (do not waste balls)
3) refine 0
Hausdorff measureFormally
For A RN k-dimensional Hausdorff measure of A
up to normalization factor covers are countable
bull HN is just the Lebesgue measure
bull curve in image
its length = H1 in R2
Total variation
Theorem (more formally) if u BV() then
u+
u-
u(x)
xx0
u+ u- - approximate upper and lower limits
Su = x u+gtu-the jump set
Energy functional
data term
regularization for background image
regularization for background masks
Total variation example
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
= perimeter = 4
Divide each side into n parts
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation(= sum of perimeters)
Large total variation(= sum of perimeters)
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation Large total variation
Edge-preserving smoothnessSpace of Bounded Variations
BV informally functions with discontinuities on curves
Edges are preserved texture is not preserved
original sequencetemporal median energy minimization
in BV
Energy functional
Time-discretized problem
Find minimum of E subject to
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Edge-preserving smoothnessRegularization term
Quadratic regularization [Tikhonov Arsenin 1977]
ELE
Known to produce very strong isotropic smoothing
Edge-preserving smoothnessRegularization term
Change regularization
ELE
Edge-preserving smoothnessRegularization term
ELE
Edge-preserving smoothnessRegularization term
ELE
n
Change the coordinate system
across the edgealong the edge
Compare
Edge-preserving smoothnessRegularization term
Weak edge (s 0)
Conditions on
Isotropic smoothings) is quadratic at zero
(s)
s
Edge-preserving smoothnessRegularization term
Strong edge (s )
Conditions on
bull no smoothing across the edge
bull more smoothing along the edge
Anisotropic smoothings) does not grow too fast at infinity
(s)
s
Edge-preserving smoothnessRegularization term
ConclusionUsing regularization term of the form
we can achieve both
isotropic smoothness in uniform regions
and anisotropic smoothness on edges
with one function
0 1 2 3 4 50
05
1
15
2
25
3
35
4
Edge-preserving smoothnessRegularization term
Example of an edge-preserving function
0 005 01 015 020
0005
001
0015
002
Edge-preserving smoothnessSpace of Bounded Variations
Even if we have an edge-preserving functional
if the space of solutions u contains only smooth functions we may not achieve the desired minimum
-1 -05 0 05 1-1
-05
0
05
1
Edge-preserving smoothnessSpace of Bounded Variations
which one is ldquobetterrdquo
Bounded Variation ndash ND caseBounded Variation ndash ND case
sup |)(| dxdivffD
)( x
N
1i i
i xdiv
1 || )( ) ( )(
101
L
NN C
bounded open subset function NR )( 1 Lf
Variation of over f
where
φ
Edge-preserving smoothnessSpace of Bounded Variations
integrable (absolute value) and with bounded variation
Functions are not required to have an integrable derivative hellip
What is the meaning of u in the regularization term
Intuitively norm of gradient |u| is
replaced with variation |Du|
Total variation
Theorem (informally) if u BV() then
Hausdorff measure
area gt 0area = 0
How can we measure zero-measure sets
Hausdorff measure
1) cover with balls of diameter
2) sum up diameters for optimal cover (do not waste balls)
3) refine 0
Hausdorff measureFormally
For A RN k-dimensional Hausdorff measure of A
up to normalization factor covers are countable
bull HN is just the Lebesgue measure
bull curve in image
its length = H1 in R2
Total variation
Theorem (more formally) if u BV() then
u+
u-
u(x)
xx0
u+ u- - approximate upper and lower limits
Su = x u+gtu-the jump set
Energy functional
data term
regularization for background image
regularization for background masks
Total variation example
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
= perimeter = 4
Divide each side into n parts
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation(= sum of perimeters)
Large total variation(= sum of perimeters)
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation Large total variation
Edge-preserving smoothnessSpace of Bounded Variations
BV informally functions with discontinuities on curves
Edges are preserved texture is not preserved
original sequencetemporal median energy minimization
in BV
Energy functional
Time-discretized problem
Find minimum of E subject to
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Edge-preserving smoothnessRegularization term
Change regularization
ELE
Edge-preserving smoothnessRegularization term
ELE
Edge-preserving smoothnessRegularization term
ELE
n
Change the coordinate system
across the edgealong the edge
Compare
Edge-preserving smoothnessRegularization term
Weak edge (s 0)
Conditions on
Isotropic smoothings) is quadratic at zero
(s)
s
Edge-preserving smoothnessRegularization term
Strong edge (s )
Conditions on
bull no smoothing across the edge
bull more smoothing along the edge
Anisotropic smoothings) does not grow too fast at infinity
(s)
s
Edge-preserving smoothnessRegularization term
ConclusionUsing regularization term of the form
we can achieve both
isotropic smoothness in uniform regions
and anisotropic smoothness on edges
with one function
0 1 2 3 4 50
05
1
15
2
25
3
35
4
Edge-preserving smoothnessRegularization term
Example of an edge-preserving function
0 005 01 015 020
0005
001
0015
002
Edge-preserving smoothnessSpace of Bounded Variations
Even if we have an edge-preserving functional
if the space of solutions u contains only smooth functions we may not achieve the desired minimum
-1 -05 0 05 1-1
-05
0
05
1
Edge-preserving smoothnessSpace of Bounded Variations
which one is ldquobetterrdquo
Bounded Variation ndash ND caseBounded Variation ndash ND case
sup |)(| dxdivffD
)( x
N
1i i
i xdiv
1 || )( ) ( )(
101
L
NN C
bounded open subset function NR )( 1 Lf
Variation of over f
where
φ
Edge-preserving smoothnessSpace of Bounded Variations
integrable (absolute value) and with bounded variation
Functions are not required to have an integrable derivative hellip
What is the meaning of u in the regularization term
Intuitively norm of gradient |u| is
replaced with variation |Du|
Total variation
Theorem (informally) if u BV() then
Hausdorff measure
area gt 0area = 0
How can we measure zero-measure sets
Hausdorff measure
1) cover with balls of diameter
2) sum up diameters for optimal cover (do not waste balls)
3) refine 0
Hausdorff measureFormally
For A RN k-dimensional Hausdorff measure of A
up to normalization factor covers are countable
bull HN is just the Lebesgue measure
bull curve in image
its length = H1 in R2
Total variation
Theorem (more formally) if u BV() then
u+
u-
u(x)
xx0
u+ u- - approximate upper and lower limits
Su = x u+gtu-the jump set
Energy functional
data term
regularization for background image
regularization for background masks
Total variation example
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
= perimeter = 4
Divide each side into n parts
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation(= sum of perimeters)
Large total variation(= sum of perimeters)
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation Large total variation
Edge-preserving smoothnessSpace of Bounded Variations
BV informally functions with discontinuities on curves
Edges are preserved texture is not preserved
original sequencetemporal median energy minimization
in BV
Energy functional
Time-discretized problem
Find minimum of E subject to
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Edge-preserving smoothnessRegularization term
ELE
Edge-preserving smoothnessRegularization term
ELE
n
Change the coordinate system
across the edgealong the edge
Compare
Edge-preserving smoothnessRegularization term
Weak edge (s 0)
Conditions on
Isotropic smoothings) is quadratic at zero
(s)
s
Edge-preserving smoothnessRegularization term
Strong edge (s )
Conditions on
bull no smoothing across the edge
bull more smoothing along the edge
Anisotropic smoothings) does not grow too fast at infinity
(s)
s
Edge-preserving smoothnessRegularization term
ConclusionUsing regularization term of the form
we can achieve both
isotropic smoothness in uniform regions
and anisotropic smoothness on edges
with one function
0 1 2 3 4 50
05
1
15
2
25
3
35
4
Edge-preserving smoothnessRegularization term
Example of an edge-preserving function
0 005 01 015 020
0005
001
0015
002
Edge-preserving smoothnessSpace of Bounded Variations
Even if we have an edge-preserving functional
if the space of solutions u contains only smooth functions we may not achieve the desired minimum
-1 -05 0 05 1-1
-05
0
05
1
Edge-preserving smoothnessSpace of Bounded Variations
which one is ldquobetterrdquo
Bounded Variation ndash ND caseBounded Variation ndash ND case
sup |)(| dxdivffD
)( x
N
1i i
i xdiv
1 || )( ) ( )(
101
L
NN C
bounded open subset function NR )( 1 Lf
Variation of over f
where
φ
Edge-preserving smoothnessSpace of Bounded Variations
integrable (absolute value) and with bounded variation
Functions are not required to have an integrable derivative hellip
What is the meaning of u in the regularization term
Intuitively norm of gradient |u| is
replaced with variation |Du|
Total variation
Theorem (informally) if u BV() then
Hausdorff measure
area gt 0area = 0
How can we measure zero-measure sets
Hausdorff measure
1) cover with balls of diameter
2) sum up diameters for optimal cover (do not waste balls)
3) refine 0
Hausdorff measureFormally
For A RN k-dimensional Hausdorff measure of A
up to normalization factor covers are countable
bull HN is just the Lebesgue measure
bull curve in image
its length = H1 in R2
Total variation
Theorem (more formally) if u BV() then
u+
u-
u(x)
xx0
u+ u- - approximate upper and lower limits
Su = x u+gtu-the jump set
Energy functional
data term
regularization for background image
regularization for background masks
Total variation example
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
= perimeter = 4
Divide each side into n parts
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation(= sum of perimeters)
Large total variation(= sum of perimeters)
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation Large total variation
Edge-preserving smoothnessSpace of Bounded Variations
BV informally functions with discontinuities on curves
Edges are preserved texture is not preserved
original sequencetemporal median energy minimization
in BV
Energy functional
Time-discretized problem
Find minimum of E subject to
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Edge-preserving smoothnessRegularization term
ELE
n
Change the coordinate system
across the edgealong the edge
Compare
Edge-preserving smoothnessRegularization term
Weak edge (s 0)
Conditions on
Isotropic smoothings) is quadratic at zero
(s)
s
Edge-preserving smoothnessRegularization term
Strong edge (s )
Conditions on
bull no smoothing across the edge
bull more smoothing along the edge
Anisotropic smoothings) does not grow too fast at infinity
(s)
s
Edge-preserving smoothnessRegularization term
ConclusionUsing regularization term of the form
we can achieve both
isotropic smoothness in uniform regions
and anisotropic smoothness on edges
with one function
0 1 2 3 4 50
05
1
15
2
25
3
35
4
Edge-preserving smoothnessRegularization term
Example of an edge-preserving function
0 005 01 015 020
0005
001
0015
002
Edge-preserving smoothnessSpace of Bounded Variations
Even if we have an edge-preserving functional
if the space of solutions u contains only smooth functions we may not achieve the desired minimum
-1 -05 0 05 1-1
-05
0
05
1
Edge-preserving smoothnessSpace of Bounded Variations
which one is ldquobetterrdquo
Bounded Variation ndash ND caseBounded Variation ndash ND case
sup |)(| dxdivffD
)( x
N
1i i
i xdiv
1 || )( ) ( )(
101
L
NN C
bounded open subset function NR )( 1 Lf
Variation of over f
where
φ
Edge-preserving smoothnessSpace of Bounded Variations
integrable (absolute value) and with bounded variation
Functions are not required to have an integrable derivative hellip
What is the meaning of u in the regularization term
Intuitively norm of gradient |u| is
replaced with variation |Du|
Total variation
Theorem (informally) if u BV() then
Hausdorff measure
area gt 0area = 0
How can we measure zero-measure sets
Hausdorff measure
1) cover with balls of diameter
2) sum up diameters for optimal cover (do not waste balls)
3) refine 0
Hausdorff measureFormally
For A RN k-dimensional Hausdorff measure of A
up to normalization factor covers are countable
bull HN is just the Lebesgue measure
bull curve in image
its length = H1 in R2
Total variation
Theorem (more formally) if u BV() then
u+
u-
u(x)
xx0
u+ u- - approximate upper and lower limits
Su = x u+gtu-the jump set
Energy functional
data term
regularization for background image
regularization for background masks
Total variation example
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
= perimeter = 4
Divide each side into n parts
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation(= sum of perimeters)
Large total variation(= sum of perimeters)
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation Large total variation
Edge-preserving smoothnessSpace of Bounded Variations
BV informally functions with discontinuities on curves
Edges are preserved texture is not preserved
original sequencetemporal median energy minimization
in BV
Energy functional
Time-discretized problem
Find minimum of E subject to
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Edge-preserving smoothnessRegularization term
Weak edge (s 0)
Conditions on
Isotropic smoothings) is quadratic at zero
(s)
s
Edge-preserving smoothnessRegularization term
Strong edge (s )
Conditions on
bull no smoothing across the edge
bull more smoothing along the edge
Anisotropic smoothings) does not grow too fast at infinity
(s)
s
Edge-preserving smoothnessRegularization term
ConclusionUsing regularization term of the form
we can achieve both
isotropic smoothness in uniform regions
and anisotropic smoothness on edges
with one function
0 1 2 3 4 50
05
1
15
2
25
3
35
4
Edge-preserving smoothnessRegularization term
Example of an edge-preserving function
0 005 01 015 020
0005
001
0015
002
Edge-preserving smoothnessSpace of Bounded Variations
Even if we have an edge-preserving functional
if the space of solutions u contains only smooth functions we may not achieve the desired minimum
-1 -05 0 05 1-1
-05
0
05
1
Edge-preserving smoothnessSpace of Bounded Variations
which one is ldquobetterrdquo
Bounded Variation ndash ND caseBounded Variation ndash ND case
sup |)(| dxdivffD
)( x
N
1i i
i xdiv
1 || )( ) ( )(
101
L
NN C
bounded open subset function NR )( 1 Lf
Variation of over f
where
φ
Edge-preserving smoothnessSpace of Bounded Variations
integrable (absolute value) and with bounded variation
Functions are not required to have an integrable derivative hellip
What is the meaning of u in the regularization term
Intuitively norm of gradient |u| is
replaced with variation |Du|
Total variation
Theorem (informally) if u BV() then
Hausdorff measure
area gt 0area = 0
How can we measure zero-measure sets
Hausdorff measure
1) cover with balls of diameter
2) sum up diameters for optimal cover (do not waste balls)
3) refine 0
Hausdorff measureFormally
For A RN k-dimensional Hausdorff measure of A
up to normalization factor covers are countable
bull HN is just the Lebesgue measure
bull curve in image
its length = H1 in R2
Total variation
Theorem (more formally) if u BV() then
u+
u-
u(x)
xx0
u+ u- - approximate upper and lower limits
Su = x u+gtu-the jump set
Energy functional
data term
regularization for background image
regularization for background masks
Total variation example
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
= perimeter = 4
Divide each side into n parts
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation(= sum of perimeters)
Large total variation(= sum of perimeters)
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation Large total variation
Edge-preserving smoothnessSpace of Bounded Variations
BV informally functions with discontinuities on curves
Edges are preserved texture is not preserved
original sequencetemporal median energy minimization
in BV
Energy functional
Time-discretized problem
Find minimum of E subject to
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Edge-preserving smoothnessRegularization term
Strong edge (s )
Conditions on
bull no smoothing across the edge
bull more smoothing along the edge
Anisotropic smoothings) does not grow too fast at infinity
(s)
s
Edge-preserving smoothnessRegularization term
ConclusionUsing regularization term of the form
we can achieve both
isotropic smoothness in uniform regions
and anisotropic smoothness on edges
with one function
0 1 2 3 4 50
05
1
15
2
25
3
35
4
Edge-preserving smoothnessRegularization term
Example of an edge-preserving function
0 005 01 015 020
0005
001
0015
002
Edge-preserving smoothnessSpace of Bounded Variations
Even if we have an edge-preserving functional
if the space of solutions u contains only smooth functions we may not achieve the desired minimum
-1 -05 0 05 1-1
-05
0
05
1
Edge-preserving smoothnessSpace of Bounded Variations
which one is ldquobetterrdquo
Bounded Variation ndash ND caseBounded Variation ndash ND case
sup |)(| dxdivffD
)( x
N
1i i
i xdiv
1 || )( ) ( )(
101
L
NN C
bounded open subset function NR )( 1 Lf
Variation of over f
where
φ
Edge-preserving smoothnessSpace of Bounded Variations
integrable (absolute value) and with bounded variation
Functions are not required to have an integrable derivative hellip
What is the meaning of u in the regularization term
Intuitively norm of gradient |u| is
replaced with variation |Du|
Total variation
Theorem (informally) if u BV() then
Hausdorff measure
area gt 0area = 0
How can we measure zero-measure sets
Hausdorff measure
1) cover with balls of diameter
2) sum up diameters for optimal cover (do not waste balls)
3) refine 0
Hausdorff measureFormally
For A RN k-dimensional Hausdorff measure of A
up to normalization factor covers are countable
bull HN is just the Lebesgue measure
bull curve in image
its length = H1 in R2
Total variation
Theorem (more formally) if u BV() then
u+
u-
u(x)
xx0
u+ u- - approximate upper and lower limits
Su = x u+gtu-the jump set
Energy functional
data term
regularization for background image
regularization for background masks
Total variation example
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
= perimeter = 4
Divide each side into n parts
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation(= sum of perimeters)
Large total variation(= sum of perimeters)
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation Large total variation
Edge-preserving smoothnessSpace of Bounded Variations
BV informally functions with discontinuities on curves
Edges are preserved texture is not preserved
original sequencetemporal median energy minimization
in BV
Energy functional
Time-discretized problem
Find minimum of E subject to
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Edge-preserving smoothnessRegularization term
ConclusionUsing regularization term of the form
we can achieve both
isotropic smoothness in uniform regions
and anisotropic smoothness on edges
with one function
0 1 2 3 4 50
05
1
15
2
25
3
35
4
Edge-preserving smoothnessRegularization term
Example of an edge-preserving function
0 005 01 015 020
0005
001
0015
002
Edge-preserving smoothnessSpace of Bounded Variations
Even if we have an edge-preserving functional
if the space of solutions u contains only smooth functions we may not achieve the desired minimum
-1 -05 0 05 1-1
-05
0
05
1
Edge-preserving smoothnessSpace of Bounded Variations
which one is ldquobetterrdquo
Bounded Variation ndash ND caseBounded Variation ndash ND case
sup |)(| dxdivffD
)( x
N
1i i
i xdiv
1 || )( ) ( )(
101
L
NN C
bounded open subset function NR )( 1 Lf
Variation of over f
where
φ
Edge-preserving smoothnessSpace of Bounded Variations
integrable (absolute value) and with bounded variation
Functions are not required to have an integrable derivative hellip
What is the meaning of u in the regularization term
Intuitively norm of gradient |u| is
replaced with variation |Du|
Total variation
Theorem (informally) if u BV() then
Hausdorff measure
area gt 0area = 0
How can we measure zero-measure sets
Hausdorff measure
1) cover with balls of diameter
2) sum up diameters for optimal cover (do not waste balls)
3) refine 0
Hausdorff measureFormally
For A RN k-dimensional Hausdorff measure of A
up to normalization factor covers are countable
bull HN is just the Lebesgue measure
bull curve in image
its length = H1 in R2
Total variation
Theorem (more formally) if u BV() then
u+
u-
u(x)
xx0
u+ u- - approximate upper and lower limits
Su = x u+gtu-the jump set
Energy functional
data term
regularization for background image
regularization for background masks
Total variation example
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
= perimeter = 4
Divide each side into n parts
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation(= sum of perimeters)
Large total variation(= sum of perimeters)
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation Large total variation
Edge-preserving smoothnessSpace of Bounded Variations
BV informally functions with discontinuities on curves
Edges are preserved texture is not preserved
original sequencetemporal median energy minimization
in BV
Energy functional
Time-discretized problem
Find minimum of E subject to
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
0 1 2 3 4 50
05
1
15
2
25
3
35
4
Edge-preserving smoothnessRegularization term
Example of an edge-preserving function
0 005 01 015 020
0005
001
0015
002
Edge-preserving smoothnessSpace of Bounded Variations
Even if we have an edge-preserving functional
if the space of solutions u contains only smooth functions we may not achieve the desired minimum
-1 -05 0 05 1-1
-05
0
05
1
Edge-preserving smoothnessSpace of Bounded Variations
which one is ldquobetterrdquo
Bounded Variation ndash ND caseBounded Variation ndash ND case
sup |)(| dxdivffD
)( x
N
1i i
i xdiv
1 || )( ) ( )(
101
L
NN C
bounded open subset function NR )( 1 Lf
Variation of over f
where
φ
Edge-preserving smoothnessSpace of Bounded Variations
integrable (absolute value) and with bounded variation
Functions are not required to have an integrable derivative hellip
What is the meaning of u in the regularization term
Intuitively norm of gradient |u| is
replaced with variation |Du|
Total variation
Theorem (informally) if u BV() then
Hausdorff measure
area gt 0area = 0
How can we measure zero-measure sets
Hausdorff measure
1) cover with balls of diameter
2) sum up diameters for optimal cover (do not waste balls)
3) refine 0
Hausdorff measureFormally
For A RN k-dimensional Hausdorff measure of A
up to normalization factor covers are countable
bull HN is just the Lebesgue measure
bull curve in image
its length = H1 in R2
Total variation
Theorem (more formally) if u BV() then
u+
u-
u(x)
xx0
u+ u- - approximate upper and lower limits
Su = x u+gtu-the jump set
Energy functional
data term
regularization for background image
regularization for background masks
Total variation example
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
= perimeter = 4
Divide each side into n parts
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation(= sum of perimeters)
Large total variation(= sum of perimeters)
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation Large total variation
Edge-preserving smoothnessSpace of Bounded Variations
BV informally functions with discontinuities on curves
Edges are preserved texture is not preserved
original sequencetemporal median energy minimization
in BV
Energy functional
Time-discretized problem
Find minimum of E subject to
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Edge-preserving smoothnessSpace of Bounded Variations
Even if we have an edge-preserving functional
if the space of solutions u contains only smooth functions we may not achieve the desired minimum
-1 -05 0 05 1-1
-05
0
05
1
Edge-preserving smoothnessSpace of Bounded Variations
which one is ldquobetterrdquo
Bounded Variation ndash ND caseBounded Variation ndash ND case
sup |)(| dxdivffD
)( x
N
1i i
i xdiv
1 || )( ) ( )(
101
L
NN C
bounded open subset function NR )( 1 Lf
Variation of over f
where
φ
Edge-preserving smoothnessSpace of Bounded Variations
integrable (absolute value) and with bounded variation
Functions are not required to have an integrable derivative hellip
What is the meaning of u in the regularization term
Intuitively norm of gradient |u| is
replaced with variation |Du|
Total variation
Theorem (informally) if u BV() then
Hausdorff measure
area gt 0area = 0
How can we measure zero-measure sets
Hausdorff measure
1) cover with balls of diameter
2) sum up diameters for optimal cover (do not waste balls)
3) refine 0
Hausdorff measureFormally
For A RN k-dimensional Hausdorff measure of A
up to normalization factor covers are countable
bull HN is just the Lebesgue measure
bull curve in image
its length = H1 in R2
Total variation
Theorem (more formally) if u BV() then
u+
u-
u(x)
xx0
u+ u- - approximate upper and lower limits
Su = x u+gtu-the jump set
Energy functional
data term
regularization for background image
regularization for background masks
Total variation example
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
= perimeter = 4
Divide each side into n parts
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation(= sum of perimeters)
Large total variation(= sum of perimeters)
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation Large total variation
Edge-preserving smoothnessSpace of Bounded Variations
BV informally functions with discontinuities on curves
Edges are preserved texture is not preserved
original sequencetemporal median energy minimization
in BV
Energy functional
Time-discretized problem
Find minimum of E subject to
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Edge-preserving smoothnessSpace of Bounded Variations
which one is ldquobetterrdquo
Bounded Variation ndash ND caseBounded Variation ndash ND case
sup |)(| dxdivffD
)( x
N
1i i
i xdiv
1 || )( ) ( )(
101
L
NN C
bounded open subset function NR )( 1 Lf
Variation of over f
where
φ
Edge-preserving smoothnessSpace of Bounded Variations
integrable (absolute value) and with bounded variation
Functions are not required to have an integrable derivative hellip
What is the meaning of u in the regularization term
Intuitively norm of gradient |u| is
replaced with variation |Du|
Total variation
Theorem (informally) if u BV() then
Hausdorff measure
area gt 0area = 0
How can we measure zero-measure sets
Hausdorff measure
1) cover with balls of diameter
2) sum up diameters for optimal cover (do not waste balls)
3) refine 0
Hausdorff measureFormally
For A RN k-dimensional Hausdorff measure of A
up to normalization factor covers are countable
bull HN is just the Lebesgue measure
bull curve in image
its length = H1 in R2
Total variation
Theorem (more formally) if u BV() then
u+
u-
u(x)
xx0
u+ u- - approximate upper and lower limits
Su = x u+gtu-the jump set
Energy functional
data term
regularization for background image
regularization for background masks
Total variation example
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
= perimeter = 4
Divide each side into n parts
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation(= sum of perimeters)
Large total variation(= sum of perimeters)
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation Large total variation
Edge-preserving smoothnessSpace of Bounded Variations
BV informally functions with discontinuities on curves
Edges are preserved texture is not preserved
original sequencetemporal median energy minimization
in BV
Energy functional
Time-discretized problem
Find minimum of E subject to
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Bounded Variation ndash ND caseBounded Variation ndash ND case
sup |)(| dxdivffD
)( x
N
1i i
i xdiv
1 || )( ) ( )(
101
L
NN C
bounded open subset function NR )( 1 Lf
Variation of over f
where
φ
Edge-preserving smoothnessSpace of Bounded Variations
integrable (absolute value) and with bounded variation
Functions are not required to have an integrable derivative hellip
What is the meaning of u in the regularization term
Intuitively norm of gradient |u| is
replaced with variation |Du|
Total variation
Theorem (informally) if u BV() then
Hausdorff measure
area gt 0area = 0
How can we measure zero-measure sets
Hausdorff measure
1) cover with balls of diameter
2) sum up diameters for optimal cover (do not waste balls)
3) refine 0
Hausdorff measureFormally
For A RN k-dimensional Hausdorff measure of A
up to normalization factor covers are countable
bull HN is just the Lebesgue measure
bull curve in image
its length = H1 in R2
Total variation
Theorem (more formally) if u BV() then
u+
u-
u(x)
xx0
u+ u- - approximate upper and lower limits
Su = x u+gtu-the jump set
Energy functional
data term
regularization for background image
regularization for background masks
Total variation example
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
= perimeter = 4
Divide each side into n parts
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation(= sum of perimeters)
Large total variation(= sum of perimeters)
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation Large total variation
Edge-preserving smoothnessSpace of Bounded Variations
BV informally functions with discontinuities on curves
Edges are preserved texture is not preserved
original sequencetemporal median energy minimization
in BV
Energy functional
Time-discretized problem
Find minimum of E subject to
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Edge-preserving smoothnessSpace of Bounded Variations
integrable (absolute value) and with bounded variation
Functions are not required to have an integrable derivative hellip
What is the meaning of u in the regularization term
Intuitively norm of gradient |u| is
replaced with variation |Du|
Total variation
Theorem (informally) if u BV() then
Hausdorff measure
area gt 0area = 0
How can we measure zero-measure sets
Hausdorff measure
1) cover with balls of diameter
2) sum up diameters for optimal cover (do not waste balls)
3) refine 0
Hausdorff measureFormally
For A RN k-dimensional Hausdorff measure of A
up to normalization factor covers are countable
bull HN is just the Lebesgue measure
bull curve in image
its length = H1 in R2
Total variation
Theorem (more formally) if u BV() then
u+
u-
u(x)
xx0
u+ u- - approximate upper and lower limits
Su = x u+gtu-the jump set
Energy functional
data term
regularization for background image
regularization for background masks
Total variation example
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
= perimeter = 4
Divide each side into n parts
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation(= sum of perimeters)
Large total variation(= sum of perimeters)
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation Large total variation
Edge-preserving smoothnessSpace of Bounded Variations
BV informally functions with discontinuities on curves
Edges are preserved texture is not preserved
original sequencetemporal median energy minimization
in BV
Energy functional
Time-discretized problem
Find minimum of E subject to
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Total variation
Theorem (informally) if u BV() then
Hausdorff measure
area gt 0area = 0
How can we measure zero-measure sets
Hausdorff measure
1) cover with balls of diameter
2) sum up diameters for optimal cover (do not waste balls)
3) refine 0
Hausdorff measureFormally
For A RN k-dimensional Hausdorff measure of A
up to normalization factor covers are countable
bull HN is just the Lebesgue measure
bull curve in image
its length = H1 in R2
Total variation
Theorem (more formally) if u BV() then
u+
u-
u(x)
xx0
u+ u- - approximate upper and lower limits
Su = x u+gtu-the jump set
Energy functional
data term
regularization for background image
regularization for background masks
Total variation example
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
= perimeter = 4
Divide each side into n parts
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation(= sum of perimeters)
Large total variation(= sum of perimeters)
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation Large total variation
Edge-preserving smoothnessSpace of Bounded Variations
BV informally functions with discontinuities on curves
Edges are preserved texture is not preserved
original sequencetemporal median energy minimization
in BV
Energy functional
Time-discretized problem
Find minimum of E subject to
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Hausdorff measure
area gt 0area = 0
How can we measure zero-measure sets
Hausdorff measure
1) cover with balls of diameter
2) sum up diameters for optimal cover (do not waste balls)
3) refine 0
Hausdorff measureFormally
For A RN k-dimensional Hausdorff measure of A
up to normalization factor covers are countable
bull HN is just the Lebesgue measure
bull curve in image
its length = H1 in R2
Total variation
Theorem (more formally) if u BV() then
u+
u-
u(x)
xx0
u+ u- - approximate upper and lower limits
Su = x u+gtu-the jump set
Energy functional
data term
regularization for background image
regularization for background masks
Total variation example
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
= perimeter = 4
Divide each side into n parts
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation(= sum of perimeters)
Large total variation(= sum of perimeters)
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation Large total variation
Edge-preserving smoothnessSpace of Bounded Variations
BV informally functions with discontinuities on curves
Edges are preserved texture is not preserved
original sequencetemporal median energy minimization
in BV
Energy functional
Time-discretized problem
Find minimum of E subject to
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Hausdorff measure
1) cover with balls of diameter
2) sum up diameters for optimal cover (do not waste balls)
3) refine 0
Hausdorff measureFormally
For A RN k-dimensional Hausdorff measure of A
up to normalization factor covers are countable
bull HN is just the Lebesgue measure
bull curve in image
its length = H1 in R2
Total variation
Theorem (more formally) if u BV() then
u+
u-
u(x)
xx0
u+ u- - approximate upper and lower limits
Su = x u+gtu-the jump set
Energy functional
data term
regularization for background image
regularization for background masks
Total variation example
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
= perimeter = 4
Divide each side into n parts
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation(= sum of perimeters)
Large total variation(= sum of perimeters)
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation Large total variation
Edge-preserving smoothnessSpace of Bounded Variations
BV informally functions with discontinuities on curves
Edges are preserved texture is not preserved
original sequencetemporal median energy minimization
in BV
Energy functional
Time-discretized problem
Find minimum of E subject to
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Hausdorff measureFormally
For A RN k-dimensional Hausdorff measure of A
up to normalization factor covers are countable
bull HN is just the Lebesgue measure
bull curve in image
its length = H1 in R2
Total variation
Theorem (more formally) if u BV() then
u+
u-
u(x)
xx0
u+ u- - approximate upper and lower limits
Su = x u+gtu-the jump set
Energy functional
data term
regularization for background image
regularization for background masks
Total variation example
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
= perimeter = 4
Divide each side into n parts
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation(= sum of perimeters)
Large total variation(= sum of perimeters)
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation Large total variation
Edge-preserving smoothnessSpace of Bounded Variations
BV informally functions with discontinuities on curves
Edges are preserved texture is not preserved
original sequencetemporal median energy minimization
in BV
Energy functional
Time-discretized problem
Find minimum of E subject to
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Total variation
Theorem (more formally) if u BV() then
u+
u-
u(x)
xx0
u+ u- - approximate upper and lower limits
Su = x u+gtu-the jump set
Energy functional
data term
regularization for background image
regularization for background masks
Total variation example
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
= perimeter = 4
Divide each side into n parts
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation(= sum of perimeters)
Large total variation(= sum of perimeters)
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation Large total variation
Edge-preserving smoothnessSpace of Bounded Variations
BV informally functions with discontinuities on curves
Edges are preserved texture is not preserved
original sequencetemporal median energy minimization
in BV
Energy functional
Time-discretized problem
Find minimum of E subject to
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Energy functional
data term
regularization for background image
regularization for background masks
Total variation example
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
= perimeter = 4
Divide each side into n parts
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation(= sum of perimeters)
Large total variation(= sum of perimeters)
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation Large total variation
Edge-preserving smoothnessSpace of Bounded Variations
BV informally functions with discontinuities on curves
Edges are preserved texture is not preserved
original sequencetemporal median energy minimization
in BV
Energy functional
Time-discretized problem
Find minimum of E subject to
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Total variation example
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
-1 -05 0 05 1 15 2
-1
0
1
2
0
05
1
= perimeter = 4
Divide each side into n parts
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation(= sum of perimeters)
Large total variation(= sum of perimeters)
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation Large total variation
Edge-preserving smoothnessSpace of Bounded Variations
BV informally functions with discontinuities on curves
Edges are preserved texture is not preserved
original sequencetemporal median energy minimization
in BV
Energy functional
Time-discretized problem
Find minimum of E subject to
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation(= sum of perimeters)
Large total variation(= sum of perimeters)
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation Large total variation
Edge-preserving smoothnessSpace of Bounded Variations
BV informally functions with discontinuities on curves
Edges are preserved texture is not preserved
original sequencetemporal median energy minimization
in BV
Energy functional
Time-discretized problem
Find minimum of E subject to
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Edge-preserving smoothnessSpace of Bounded Variations
Small total variation Large total variation
Edge-preserving smoothnessSpace of Bounded Variations
BV informally functions with discontinuities on curves
Edges are preserved texture is not preserved
original sequencetemporal median energy minimization
in BV
Energy functional
Time-discretized problem
Find minimum of E subject to
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Edge-preserving smoothnessSpace of Bounded Variations
BV informally functions with discontinuities on curves
Edges are preserved texture is not preserved
original sequencetemporal median energy minimization
in BV
Energy functional
Time-discretized problem
Find minimum of E subject to
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Energy functional
Time-discretized problem
Find minimum of E subject to
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Existence of solution
Under usual assumptions
12 R+ R+ strictly convex nondecreasing
with linear growth at infinity
minimum of E exists in BV(BC1hellipCT)
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
(non-)Uniqueness
is not convex wrt (BC1hellipCT) Solution may not be unique
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Uniqueness
But if c 3range2(Nt t=1hellipT x ) then the functional is strictly convex and solution is unique
Interpretation if we are allowed to say that everything is foreground background image is not well-defined
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Finding solution
BV is a difficult space you cannot write Euler-Lagrange equations cannot work numerically with function in BV
Strategy bull construct approximating functionals admitting solution in a more regular spacebull solve minimization problem for these functionalsbull find solution as limit of the approximate solutions
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Approximating functionals
Recall 12(s) = s2 gives smooth solutions
Idea replace i with iwhich are quadratic
at s 0 and s
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Approximating functionals
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
Approximating problems
has unique solution in the space
ndash convergence of functionals if E -converge to Ethen approximate solutions of min E
converge to min E
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
More results Sweden
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
More results Highway
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
More results INRIA_1
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
More results INRIA_1Sequence restoration
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
More results INRIA_2Sequence restoration
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-
- Background vs foreground segmentation of video sequences
- The Problem
- Simple approach (1)
- Simple approach (2)
- Simple approach noise can spoil everything
- Variational approach
- Notations
- Energy functional data term
- Slide 9
- Slide 10
- Slide 11
- Energy functional smoothness
- Energy functional
- Edge-preserving smoothness Regularization term
- Slide 15
- Slide 16
- Slide 17
- Slide 18
- Slide 19
- Slide 20
- Slide 21
- Edge-preserving smoothness Space of Bounded Variations
- Slide 23
- Bounded Variation ndash ND case
- Slide 25
- Total variation
- Hausdorff measure
- Slide 28
- Slide 29
- Slide 30
- Slide 31
- Total variation example
- Slide 33
- Slide 34
- Slide 35
- Slide 36
- Existence of solution
- (non-)Uniqueness
- Uniqueness
- Finding solution
- Approximating functionals
- Slide 42
- Approximating problems
- More results Sweden
- More results Highway
- More results INRIA_1
- More results INRIA_1 Sequence restoration
- More results INRIA_2 Sequence restoration
- Slide 49
-