variational methods in image processing - caszoi.utia.cas.cz/files/var_calculus_2.pdfvariational...

109
Introduction Variational Methods in Image Processing ÚTIA AV ˇ CR Variational Methods

Upload: others

Post on 01-Jun-2020

30 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction

Variational Methods in Image Processing

ÚTIA AV CR

Variational Methods

Page 2: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction

Outline

1 IntroductionMotivationDerivation of Euler-Lagrange EquationVariational Problem and P.D.E.

Variational Methods

Page 3: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Outline

1 IntroductionMotivationDerivation of Euler-Lagrange EquationVariational Problem and P.D.E.

Variational Methods

Page 4: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

History

The Brachistochrone Problem:

“Given two points A and B in a verticalplane, what is the curve traced out bya point acted on only by gravity, whichstarts at A and reaches B in theshortest time.”Johann Bernoulli in 1696

In one year Newton, Johann andJacob Bernoulli, Leibniz, and deL’Hôpital came with the solution.

Variational Methods

Page 5: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

History

The Brachistochrone Problem:

“Given two points A and B in a verticalplane, what is the curve traced out bya point acted on only by gravity, whichstarts at A and reaches B in theshortest time.”Johann Bernoulli in 1696

In one year Newton, Johann andJacob Bernoulli, Leibniz, and deL’Hôpital came with the solution.

Variational Methods

Page 6: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

History

The Brachistochrone Problem:

“Given two points A and B in a verticalplane, what is the curve traced out bya point acted on only by gravity, whichstarts at A and reaches B in theshortest time.”Johann Bernoulli in 1696

In one year Newton, Johann andJacob Bernoulli, Leibniz, and deL’Hôpital came with the solution.

Variational Methods

Page 7: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

History

The problem was generalized and an analytic method wasgiven by Euler (1744) and Lagrange (1760).

Variational Methods

Page 8: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Calculus of Variations

Most of the image processing tasks can be formulated asoptimization problems, i.e., minimization of functionals

Calculus of Variations solves

minu

F (u(x)) ,

where u ∈ X ,F : X → R,X . . . Banach spacesolution by means of Euler-Lagrange (E-L) equation

Variational Methods

Page 9: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Calculus of Variations

Most of the image processing tasks can be formulated asoptimization problems, i.e., minimization of functionalsCalculus of Variations solves

minu

F (u(x)) ,

where u ∈ X ,F : X → R,X . . . Banach space

solution by means of Euler-Lagrange (E-L) equation

Variational Methods

Page 10: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Calculus of Variations

Most of the image processing tasks can be formulated asoptimization problems, i.e., minimization of functionalsCalculus of Variations solves

minu

F (u(x)) ,

where u ∈ X ,F : X → R,X . . . Banach spacesolution by means of Euler-Lagrange (E-L) equation

Variational Methods

Page 11: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Calculus of Variations

Integral functionals

F (u) =

∫Ω

f (x ,u(x),∇u(x))dx

Example

x ∈ R2 . . . space of coordinates [x1, x2]

Ω . . . image supportu(x) : R2 → R . . . grayscale image∇u(x) . . . image gradient [ux1 ,ux2 ]

Variational Methods

Page 12: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Examples

Image Registrationgiven a set of CP pairs [xi , yi ]↔ [xi , yi ]find x = f (x , y), y = g(x , y)

F (f ) =∑

i

(xi − f (xi , yi))2 + λ

∫ ∫f 2xx + 2f 2

xy + f 2yydxdy

and a similar equation for g(x , y)

Image Reconstruction

Variational Methods

Page 13: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Examples

Image Registration

Image Reconstruction

Variational Methods

Page 14: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Examples

Image Registration

Image Reconstructiongiven an image acquisition model H(·) and measurement gfind the original image u

F (u) =

∫(H(u)− g)2dx + λ

∫|∇u|2

Variational Methods

Page 15: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Examples

Image Registration

Image Reconstruction

Variational Methods

Page 16: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Examples

Image Segmentationfind a piece-wise constant representation u of an image g

F (u,K ) =

∫Ω−K

(u − g)2dx + α

∫Ω−K|∇u|2dx + β

∫K

ds

Motion Estimation

Variational Methods

Page 17: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Examples

Image Segmentation

Motion Estimation

Variational Methods

Page 18: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Examples

Image Segmentation

Motion Estimationfind velocity field v(x) ≡ [v1(x), v2(x)] in an image sequenceu(x , t)

F (v) =

∫|v · ∇u + ut |dx + α

∑j

∫|∇vj |dx + β

∫c(∇u)|v |2dx

Variational Methods

Page 19: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Examples

Image Segmentation

Motion Estimation

Variational Methods

Page 20: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Examples

Image classification

and many more

Variational Methods

Page 21: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Examples

Image classificationand many more

Variational Methods

Page 22: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Outline

1 IntroductionMotivationDerivation of Euler-Lagrange EquationVariational Problem and P.D.E.

Variational Methods

Page 23: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Extrema points

From the differential calculus follows that

if x is an extremum of g(x) : RN → R then ∀ν ∈ RN

ddε

g(x + εν)∣∣∣ε=0

= 0

= 〈∇g(x), ν〉 ⇔ ∇g(x) = 0

in 1-D (g : R → R) we get the classical condition

g′(x) = 0

Variational Methods

Page 24: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Extrema points

From the differential calculus follows thatif x is an extremum of g(x) : RN → R then ∀ν ∈ RN

ddε

g(x + εν)∣∣∣ε=0

= 0

= 〈∇g(x), ν〉 ⇔ ∇g(x) = 0

in 1-D (g : R → R) we get the classical condition

g′(x) = 0

Variational Methods

Page 25: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Extrema points

From the differential calculus follows thatif x is an extremum of g(x) : RN → R then ∀ν ∈ RN

ddε

g(x + εν)∣∣∣ε=0

= 0

= 〈∇g(x), ν〉 ⇔ ∇g(x) = 0

in 1-D (g : R → R) we get the classical condition

g′(x) = 0

Variational Methods

Page 26: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Extrema points

From the differential calculus follows thatif x is an extremum of g(x) : RN → R then ∀ν ∈ RN

ddε

g(x + εν)∣∣∣ε=0

= 0 = 〈∇g(x), ν〉

⇔ ∇g(x) = 0

in 1-D (g : R → R) we get the classical condition

g′(x) = 0

Variational Methods

Page 27: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Extrema points

From the differential calculus follows thatif x is an extremum of g(x) : RN → R then ∀ν ∈ RN

ddε

g(x + εν)∣∣∣ε=0

= 0 = 〈∇g(x), ν〉 ⇔ ∇g(x) = 0

in 1-D (g : R → R) we get the classical condition

g′(x) = 0

Variational Methods

Page 28: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Extrema points

From the differential calculus follows thatif x is an extremum of g(x) : RN → R then ∀ν ∈ RN

ddε

g(x + εν)∣∣∣ε=0

= 0 = 〈∇g(x), ν〉 ⇔ ∇g(x) = 0

in 1-D (g : R → R) we get the classical condition

g′(x) = 0

Variational Methods

Page 29: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Variation of Functional

F (u) =

∫ b

af (x ,u,u′)dx

if u is extremum of F then fromdifferential calculus follows

ddε

F (u + εv)∣∣∣ε=0

= 0 ∀v

F (u +εv) =

∫ b

af (x ,u +εv ,u′+εv ′)dx

Variational Methods

Page 30: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Variation of Functional

F (u) =

∫ b

af (x ,u,u′)dx

if u is extremum of F then fromdifferential calculus follows

ddε

F (u + εv)∣∣∣ε=0

= 0 ∀v

F (u +εv) =

∫ b

af (x ,u +εv ,u′+εv ′)dx

Variational Methods

Page 31: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Chain Rule

ddx

f (u(x), v(x)) =( ∂∂u

f (u, v))du

dx+( ∂∂v

f (u, v))dv

dx

Example

u = x , v = sin x , f = uv = x sin x

ddx

f (u, v) = v(x)1 + u(x) cos x = sin x + x cos x

Variational Methods

Page 32: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Chain Rule

ddx

f (u(x), v(x)) =( ∂∂u

f (u, v))du

dx+( ∂∂v

f (u, v))dv

dx

Example

u = x , v = sin x , f = uv = x sin x

ddx

f (u, v) = v(x)1 + u(x) cos x = sin x + x cos x

Variational Methods

Page 33: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Chain Rule

ddx

f (u(x), v(x)) =( ∂∂u

f (u, v))du

dx+( ∂∂v

f (u, v))dv

dx

Example

u = x , v = sin x , f = uv = x sin x

ddx

f (u, v) = v(x)1 + u(x) cos x = sin x + x cos x

Variational Methods

Page 34: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Chain Rule

ddx

f (u(x), v(x)) =( ∂∂u

f (u, v))du

dx+( ∂∂v

f (u, v))dv

dx

Example

u = x , v = sin x , f = uv = x sin x

ddx

f (u, v) = v(x)1 + u(x) cos x = sin x + x cos x

Variational Methods

Page 35: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Partial derivatives

Example

f (x ,u) = x sin x

dfdx

= chain rule = sin x + x cos x

but∂f∂x

= sin x

Variational Methods

Page 36: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Partial derivatives

Example

f (x ,u) = x sin x

dfdx

= chain rule = sin x + x cos x

but∂f∂x

= sin x

Variational Methods

Page 37: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

per partes

∫ b

auv ′ = uv

∣∣∣ba−∫ b

au′v

Variational Methods

Page 38: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Derivation of E-L equation

ddε

F (u + εv) =ddε

∫ b

af (x ,u + εv ,u′ + εv ′)

=

∫ b

a

∂f∂u

v +∂f∂u′

v ′ chain rule

=

∫ b

a

∂f∂u

v −∫ b

a

ddx

∂f∂u′

v +∂f∂u′

v∣∣∣ba

per partes

=

∫ b

a

[∂f∂u− d

dx∂f∂u′

]v +

∂f∂u′

v∣∣∣ba

= 0

Variational Methods

Page 39: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Derivation of E-L equation

ddε

F (u + εv) =ddε

∫ b

af (x ,u + εv ,u′ + εv ′)

=

∫ b

a

∂f∂u

v +∂f∂u′

v ′ chain rule

=

∫ b

a

∂f∂u

v −∫ b

a

ddx

∂f∂u′

v +∂f∂u′

v∣∣∣ba

per partes

=

∫ b

a

[∂f∂u− d

dx∂f∂u′

]v +

∂f∂u′

v∣∣∣ba

= 0

Variational Methods

Page 40: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Derivation of E-L equation

ddε

F (u + εv) =ddε

∫ b

af (x ,u + εv ,u′ + εv ′)

=

∫ b

a

∂f∂u

v +∂f∂u′

v ′ chain rule

=

∫ b

a

∂f∂u

v −∫ b

a

ddx

∂f∂u′

v +∂f∂u′

v∣∣∣ba

per partes

=

∫ b

a

[∂f∂u− d

dx∂f∂u′

]v +

∂f∂u′

v∣∣∣ba

= 0

Variational Methods

Page 41: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Derivation of E-L equation

ddε

F (u + εv) =ddε

∫ b

af (x ,u + εv ,u′ + εv ′)

=

∫ b

a

∂f∂u

v +∂f∂u′

v ′ chain rule

=

∫ b

a

∂f∂u

v −∫ b

a

ddx

∂f∂u′

v +∂f∂u′

v∣∣∣ba

per partes

=

∫ b

a

[∂f∂u− d

dx∂f∂u′

]v +

∂f∂u′

v∣∣∣ba

= 0

Variational Methods

Page 42: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Derivation of E-L equation

ddε

F (u + εv) =ddε

∫ b

af (x ,u + εv ,u′ + εv ′)

=

∫ b

a

∂f∂u

v +∂f∂u′

v ′ chain rule

=

∫ b

a

∂f∂u

v −∫ b

a

ddx

∂f∂u′

v +∂f∂u′

v∣∣∣ba

per partes

=

∫ b

a

[∂f∂u− d

dx∂f∂u′

]v +

∂f∂u′

v∣∣∣ba

= 0

to be equal to 0 for any v ,[∂f∂u −

ddx

∂f∂u′

]= 0→ E-L equation

Variational Methods

Page 43: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Derivation of E-L equation

ddε

F (u + εv) =ddε

∫ b

af (x ,u + εv ,u′ + εv ′)

=

∫ b

a

∂f∂u

v +∂f∂u′

v ′ chain rule

=

∫ b

a

∂f∂u

v −∫ b

a

ddx

∂f∂u′

v +∂f∂u′

v∣∣∣ba

per partes

=

∫ b

a

[∂f∂u− d

dx∂f∂u′

]v +

∂f∂u′

v∣∣∣ba

= 0

to be equal to 0, we need boundary conditions,e.g., fixed u(a),u(b)→ v(a) = v(b) = 0.

Variational Methods

Page 44: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Toy caseShortest path

Find the shortest path betweenpoints A and B, assuming thatone can write y = y(x).

We want to minimize F (y(x)) =∫ b

a

√1 + y ′(x)2dx

with b.c. y(a) = α, y(b) = β.

E-L eq.: − ddx

y ′(x)√1+y ′(x)2

= 0

⇒ y ′ = C√

1 + y ′2 ⇒

y ′ = constant

y(x) is a straight line between A and B.

Variational Methods

Page 45: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Toy caseShortest path

Find the shortest path betweenpoints A and B, assuming thatone can write y = y(x).

We want to minimize F (y(x)) =∫ b

a

√1 + y ′(x)2dx

with b.c. y(a) = α, y(b) = β.

E-L eq.: − ddx

y ′(x)√1+y ′(x)2

= 0

⇒ y ′ = C√

1 + y ′2 ⇒

y ′ = constant

y(x) is a straight line between A and B.

Variational Methods

Page 46: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Toy caseShortest path

Find the shortest path betweenpoints A and B, assuming thatone can write y = y(x).

We want to minimize F (y(x)) =∫ b

a

√1 + y ′(x)2dx

with b.c. y(a) = α, y(b) = β.

E-L eq.: − ddx

y ′(x)√1+y ′(x)2

= 0

⇒ y ′ = C√

1 + y ′2 ⇒

y ′ = constanty(x) is a straight line between A and B.

Variational Methods

Page 47: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Toy caseShortest path

Find the shortest path betweenpoints A and B, assuming thatone can write y = y(x).

We want to minimize F (y(x)) =∫ b

a

√1 + y ′(x)2dx

with b.c. y(a) = α, y(b) = β.

E-L eq.: − ddx

y ′(x)√1+y ′(x)2

= 0⇒ y ′ = C√

1 + y ′2

y ′ = constanty(x) is a straight line between A and B.

Variational Methods

Page 48: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Toy caseShortest path

Find the shortest path betweenpoints A and B, assuming thatone can write y = y(x).

We want to minimize F (y(x)) =∫ b

a

√1 + y ′(x)2dx

with b.c. y(a) = α, y(b) = β.

E-L eq.: − ddx

y ′(x)√1+y ′(x)2

= 0⇒ y ′ = C√

1 + y ′2 ⇒

y ′ = constant

y(x) is a straight line between A and B.

Variational Methods

Page 49: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Toy caseShortest path

Find the shortest path betweenpoints A and B, assuming thatone can write y = y(x).

We want to minimize F (y(x)) =∫ b

a

√1 + y ′(x)2dx

with b.c. y(a) = α, y(b) = β.

E-L eq.: − ddx

y ′(x)√1+y ′(x)2

= 0⇒ y ′ = C√

1 + y ′2 ⇒

y ′ = constanty(x) is a straight line between A and B.

Variational Methods

Page 50: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

E-L equation

If u(x) : RN → R is extremum of F (u) =∫

Ω f (x ,u,∇u)dx ,where ∇u ≡ [ux1 , . . . ,uxN ]then

F ′(u) =∂f∂u

(x ,u,∇u)−N∑

i=1

ddxi

(∂f∂uxi

(x ,u,∇u)

)= 0 ,

which is the E-L equation.

Variational Methods

Page 51: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

E-L equation

If u(x) : RN → R is extremum of F (u) =∫

Ω f (x ,u,∇u)dx ,where ∇u ≡ [ux1 , . . . ,uxN ]then

F ′(u) =∂f∂u

(x ,u,∇u)−N∑

i=1

ddxi

(∂f∂uxi

(x ,u,∇u)

)= 0 ,

which is the E-L equation.

Variational Methods

Page 52: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Beltrami Identity

f (x ,u,u′)∂f∂u− d

dx

( ∂f∂u′)

= 0

dfdx

=∂f∂u

u′ +∂f∂u′

u′′ +∂f∂x

∂f∂u

u′ =dfdx− ∂f∂u′

u′′ − ∂f∂x

u′∂f∂u− u′

ddx

( ∂f∂u′)

= 0

dfdx− ∂f∂u′

u′′ − ∂f∂x− u′

ddx

( ∂f∂u′)

= 0

ddx

(f − u′

∂f∂u′)− ∂f∂x

= 0

Variational Methods

Page 53: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Beltrami Identity

f (x ,u,u′)∂f∂u− d

dx

( ∂f∂u′)

= 0

dfdx

=∂f∂u

u′ +∂f∂u′

u′′ +∂f∂x

∂f∂u

u′ =dfdx− ∂f∂u′

u′′ − ∂f∂x

u′∂f∂u− u′

ddx

( ∂f∂u′)

= 0

dfdx− ∂f∂u′

u′′ − ∂f∂x− u′

ddx

( ∂f∂u′)

= 0

ddx

(f − u′

∂f∂u′)− ∂f∂x

= 0

Variational Methods

Page 54: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Beltrami Identity

f (x ,u,u′)∂f∂u− d

dx

( ∂f∂u′)

= 0

dfdx

=∂f∂u

u′ +∂f∂u′

u′′ +∂f∂x

∂f∂u

u′ =dfdx− ∂f∂u′

u′′ − ∂f∂x

u′∂f∂u− u′

ddx

( ∂f∂u′)

= 0

dfdx− ∂f∂u′

u′′ − ∂f∂x− u′

ddx

( ∂f∂u′)

= 0

ddx

(f − u′

∂f∂u′)− ∂f∂x

= 0

Variational Methods

Page 55: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Beltrami Identity

f (x ,u,u′)∂f∂u− d

dx

( ∂f∂u′)

= 0

dfdx

=∂f∂u

u′ +∂f∂u′

u′′ +∂f∂x

∂f∂u

u′ =dfdx− ∂f∂u′

u′′ − ∂f∂x

u′∂f∂u− u′

ddx

( ∂f∂u′)

= 0

dfdx− ∂f∂u′

u′′ − ∂f∂x− u′

ddx

( ∂f∂u′)

= 0

ddx

(f − u′

∂f∂u′)− ∂f∂x

= 0

Variational Methods

Page 56: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Beltrami Identity

f (x ,u,u′)∂f∂u− d

dx

( ∂f∂u′)

= 0

dfdx

=∂f∂u

u′ +∂f∂u′

u′′ +∂f∂x

∂f∂u

u′ =dfdx− ∂f∂u′

u′′ − ∂f∂x

u′∂f∂u− u′

ddx

( ∂f∂u′)

= 0

dfdx− ∂f∂u′

u′′ − ∂f∂x− u′

ddx

( ∂f∂u′)

= 0

ddx

(f − u′

∂f∂u′)− ∂f∂x

= 0

Variational Methods

Page 57: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Beltrami Identity

ddx

(f − u′

∂f∂u′)− ∂f∂x

= 0

if ∂f∂x = 0 then

ddx

(f − u′

∂f∂u′)

= 0⇐⇒ f − u′∂f∂u′

= C

Variational Methods

Page 58: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Beltrami Identity

ddx

(f − u′

∂f∂u′)− ∂f∂x

= 0

if ∂f∂x = 0 then

ddx

(f − u′

∂f∂u′)

= 0⇐⇒ f − u′∂f∂u′

= C

Variational Methods

Page 59: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Brachistochrone

F =∫

dt , minF . . . curve of theshortest time.

F =∫ ds

v =∫ b

0

√1+(y ′(x))2

v dx12mv2 = mgy(x)⇒ v =

√2gy(x)

F =

∫ b

0

√1 + (y ′)2√

2gydx

Variational Methods

Page 60: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Brachistochrone

F =∫

dt , minF . . . curve of theshortest time.

F =∫ ds

v =∫ b

0

√1+(y ′(x))2

v dx12mv2 = mgy(x)⇒ v =

√2gy(x)

F =

∫ b

0

√1 + (y ′)2√

2gydx

Variational Methods

Page 61: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Brachistochrone

f (y , y ′) =

√1 + (y ′)2√

2gy

f − y ′∂f∂y ′

= C Beltrami identity

......

y(1 + (y ′)2) =1

2gC2 = k

The solution is a cycloid

x(θ) =12

k(θ − sin θ), y(θ) =12

k(1− cosθ)

Variational Methods

Page 62: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Brachistochrone

f (y , y ′) =

√1 + (y ′)2√

2gy

f − y ′∂f∂y ′

= C Beltrami identity

......

y(1 + (y ′)2) =1

2gC2 = k

The solution is a cycloid

x(θ) =12

k(θ − sin θ), y(θ) =12

k(1− cosθ)

Variational Methods

Page 63: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Brachistochrone

f (y , y ′) =

√1 + (y ′)2√

2gy

f − y ′∂f∂y ′

= C Beltrami identity

......

y(1 + (y ′)2) =1

2gC2 = k

The solution is a cycloid

x(θ) =12

k(θ − sin θ), y(θ) =12

k(1− cosθ)

Variational Methods

Page 64: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Brachistochrone

f (y , y ′) =

√1 + (y ′)2√

2gy

f − y ′∂f∂y ′

= C Beltrami identity

......

y(1 + (y ′)2) =1

2gC2 = k

The solution is a cycloid

x(θ) =12

k(θ − sin θ), y(θ) =12

k(1− cosθ)

Variational Methods

Page 65: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Brachistochrone

f (y , y ′) =

√1 + (y ′)2√

2gy

f − y ′∂f∂y ′

= C Beltrami identity

......

y(1 + (y ′)2) =1

2gC2 = k

The solution is a cycloid

x(θ) =12

k(θ − sin θ), y(θ) =12

k(1− cosθ)

Variational Methods

Page 66: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Cycloid

Play avi

Play avi

Variational Methods

Page 67: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Boundary conditions

using “per partes” on u(x , y), n(x , y) ≡ [n1(x , y),n2(x , y)]normal vector at the boundary ∂Ω

∂εF (u + εv) =

∫(·)dxdy +

∫∂Ω

[∂f∂ux

n1 +∂f∂uy

n2

]v ds

Dirichlet b.c.u is predefined at the boundary ∂Ω→ v(∂Ω) = 0Neumann b.c.derivative in the direction of normal ∂u

∂n = 0

Example

Consider F (u) =∫

Ω12 |∇u|2 =

∫Ω

12(u2

x + u2y )

∂f∂ux

= ux , ∂f∂uy

= uy∂f∂ux

n1 + ∂f∂uy

n2 = uxn1 + uyn2 = ∂u∂n = 0

Variational Methods

Page 68: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Boundary conditions

using “per partes” on u(x , y), n(x , y) ≡ [n1(x , y),n2(x , y)]normal vector at the boundary ∂Ω

∂εF (u + εv) =

∫(·)dxdy +

∫∂Ω

[∂f∂ux

n1 +∂f∂uy

n2

]v ds

Dirichlet b.c.u is predefined at the boundary ∂Ω→ v(∂Ω) = 0

Neumann b.c.derivative in the direction of normal ∂u

∂n = 0

Example

Consider F (u) =∫

Ω12 |∇u|2 =

∫Ω

12(u2

x + u2y )

∂f∂ux

= ux , ∂f∂uy

= uy∂f∂ux

n1 + ∂f∂uy

n2 = uxn1 + uyn2 = ∂u∂n = 0

Variational Methods

Page 69: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Boundary conditions

using “per partes” on u(x , y), n(x , y) ≡ [n1(x , y),n2(x , y)]normal vector at the boundary ∂Ω

∂εF (u + εv) =

∫(·)dxdy +

∫∂Ω

[∂f∂ux

n1 +∂f∂uy

n2

]v ds

Dirichlet b.c.u is predefined at the boundary ∂Ω→ v(∂Ω) = 0Neumann b.c.derivative in the direction of normal ∂u

∂n = 0

Example

Consider F (u) =∫

Ω12 |∇u|2 =

∫Ω

12(u2

x + u2y )

∂f∂ux

= ux , ∂f∂uy

= uy∂f∂ux

n1 + ∂f∂uy

n2 = uxn1 + uyn2 = ∂u∂n = 0

Variational Methods

Page 70: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Boundary conditions

using “per partes” on u(x , y), n(x , y) ≡ [n1(x , y),n2(x , y)]normal vector at the boundary ∂Ω

∂εF (u + εv) =

∫(·)dxdy +

∫∂Ω

[∂f∂ux

n1 +∂f∂uy

n2

]v ds

Dirichlet b.c.u is predefined at the boundary ∂Ω→ v(∂Ω) = 0Neumann b.c.derivative in the direction of normal ∂u

∂n = 0

Example

Consider F (u) =∫

Ω12 |∇u|2 =

∫Ω

12(u2

x + u2y )

∂f∂ux

= ux , ∂f∂uy

= uy∂f∂ux

n1 + ∂f∂uy

n2 = uxn1 + uyn2 = ∂u∂n = 0

Variational Methods

Page 71: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Boundary conditions

using “per partes” on u(x , y), n(x , y) ≡ [n1(x , y),n2(x , y)]normal vector at the boundary ∂Ω

∂εF (u + εv) =

∫(·)dxdy +

∫∂Ω

[∂f∂ux

n1 +∂f∂uy

n2

]v ds

Dirichlet b.c.u is predefined at the boundary ∂Ω→ v(∂Ω) = 0Neumann b.c.derivative in the direction of normal ∂u

∂n = 0

Example

Consider F (u) =∫

Ω12 |∇u|2 =

∫Ω

12(u2

x + u2y )

∂f∂ux

= ux , ∂f∂uy

= uy

∂f∂ux

n1 + ∂f∂uy

n2 = uxn1 + uyn2 = ∂u∂n = 0

Variational Methods

Page 72: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Boundary conditions

using “per partes” on u(x , y), n(x , y) ≡ [n1(x , y),n2(x , y)]normal vector at the boundary ∂Ω

∂εF (u + εv) =

∫(·)dxdy +

∫∂Ω

[∂f∂ux

n1 +∂f∂uy

n2

]v ds

Dirichlet b.c.u is predefined at the boundary ∂Ω→ v(∂Ω) = 0Neumann b.c.derivative in the direction of normal ∂u

∂n = 0

Example

Consider F (u) =∫

Ω12 |∇u|2 =

∫Ω

12(u2

x + u2y )

∂f∂ux

= ux , ∂f∂uy

= uy∂f∂ux

n1 + ∂f∂uy

n2 = uxn1 + uyn2 = ∂u∂n = 0

Variational Methods

Page 73: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

E-L equation example

Smoothing functional:

F (u) =

∫Ω|∇u|2dx , f = u2

x + u2y

E-L equation:

F ′(u) = −∆u = −u2xx − u2

yy

Variational Methods

Page 74: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

E-L equation example

Smoothing functional:

F (u) =

∫Ω|∇u|2dx , f = u2

x + u2y

E-L equation:

F ′(u) = −∆u = −u2xx − u2

yy

Variational Methods

Page 75: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

More examples

Total variation of an image function u(x,y):

F (u) =

∫Ω|∇u|dx , f =

√u2

x + u2y

E-L equation:

∂f∂u− ∂

∂x∂f∂ux− ∂

∂y∂f∂uy

− ∂

∂xux√

u2x + u2

y

− ∂

∂yuy√

u2x + u2

y

= −div(∇u|∇u|

)

Variational Methods

Page 76: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

More examples

Total variation of an image function u(x,y):

F (u) =

∫Ω|∇u|dx , f =

√u2

x + u2y

E-L equation:

∂f∂u− ∂

∂x∂f∂ux− ∂

∂y∂f∂uy

− ∂

∂xux√

u2x + u2

y

− ∂

∂yuy√

u2x + u2

y

= −div(∇u|∇u|

)

Variational Methods

Page 77: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

More examples

Total variation of an image function u(x,y):

F (u) =

∫Ω|∇u|dx , f =

√u2

x + u2y

E-L equation:

∂f∂u− ∂

∂x∂f∂ux− ∂

∂y∂f∂uy

− ∂

∂xux√

u2x + u2

y

− ∂

∂yuy√

u2x + u2

y

= −div(∇u|∇u|

)

Variational Methods

Page 78: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

More examples

Total variation of an image function u(x,y):

F (u) =

∫Ω|∇u|dx , f =

√u2

x + u2y

E-L equation:

∂f∂u− ∂

∂x∂f∂ux− ∂

∂y∂f∂uy

− ∂

∂xux√

u2x + u2

y

− ∂

∂yuy√

u2x + u2

y

= −div(∇u|∇u|

)

Variational Methods

Page 79: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

More examples

Total variation of an image function u(x,y):

F (u) =

∫Ω|∇u|dx , f =

√u2

x + u2y

E-L equation:

∂f∂u− ∂

∂x∂f∂ux− ∂

∂y∂f∂uy

− ∂

∂xux√

u2x + u2

y

− ∂

∂yuy√

u2x + u2

y

= −div(∇u|∇u|

)

Variational Methods

Page 80: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Outline

1 IntroductionMotivationDerivation of Euler-Lagrange EquationVariational Problem and P.D.E.

Variational Methods

Page 81: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Steepest Descent

Classical optimization problem

g : R → R, x = minx

g(x)

Must satisfy g′(x) = 0Imagine, analytical solution is impossible.Let us walk in the direction opposite to the gradient

xk+1 = xk − αg′(xk ) ,

where α is the step length

Variational Methods

Page 82: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Steepest Descent

Classical optimization problem

g : R → R, x = minx

g(x)

Must satisfy g′(x) = 0

Imagine, analytical solution is impossible.Let us walk in the direction opposite to the gradient

xk+1 = xk − αg′(xk ) ,

where α is the step length

Variational Methods

Page 83: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Steepest Descent

Classical optimization problem

g : R → R, x = minx

g(x)

Must satisfy g′(x) = 0Imagine, analytical solution is impossible.

Let us walk in the direction opposite to the gradient

xk+1 = xk − αg′(xk ) ,

where α is the step length

Variational Methods

Page 84: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Steepest Descent

Classical optimization problem

g : R → R, x = minx

g(x)

Must satisfy g′(x) = 0Imagine, analytical solution is impossible.Let us walk in the direction opposite to the gradient

xk+1 = xk − αg′(xk ) ,

where α is the step length

Variational Methods

Page 85: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Steepest Descent

Classical optimization problem

g : R → R, x = minx

g(x)

Must satisfy g′(x) = 0Imagine, analytical solution is impossible.Let us walk in the direction opposite to the gradient

xk+1 = xk − αg′(xk ) ,

where α is the step length

Variational Methods

Page 86: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Steepest Descent

Classical optimization problem

g : R → R, x = minx

g(x)

Must satisfy g′(x) = 0Imagine, analytical solution is impossible.Let us walk in the direction opposite to the gradient

xk+1 = xk − αg′(xk ) ,

where α is the step length

Variational Methods

Page 87: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Steepest Descent

Classical optimization problem

g : R → R, x = minx

g(x)

Must satisfy g′(x) = 0Imagine, analytical solution is impossible.Let us walk in the direction opposite to the gradient

xk+1 = xk − αg′(xk ) ,

where α is the step length

Variational Methods

Page 88: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Steepest Descent

Classical optimization problem

g : R → R, x = minx

g(x)

Must satisfy g′(x) = 0Imagine, analytical solution is impossible.Let us walk in the direction opposite to the gradient

xk+1 = xk − αg′(xk ) ,

where α is the step length

Variational Methods

Page 89: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Steepest Descent

∀αxk+1 − xk

α= −g′(xk ) ,

Define x(t) as a function of time such that x(tk ) = xk andtk+1 = tk + α

dxdt

(tk ) = limα→0

x(tk + α)− x(tk )

α= lim

α→0

xk+1 − xk

α= −g′(xk )

Finding the solution with the steepest-descent method isequivalent to solving P.D.E.:

dxdt

= −g′(x)

Variational Methods

Page 90: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Steepest Descent

∀αxk+1 − xk

α= −g′(xk ) ,

Define x(t) as a function of time such that x(tk ) = xk andtk+1 = tk + α

dxdt

(tk ) = limα→0

x(tk + α)− x(tk )

α= lim

α→0

xk+1 − xk

α= −g′(xk )

Finding the solution with the steepest-descent method isequivalent to solving P.D.E.:

dxdt

= −g′(x)

Variational Methods

Page 91: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Steepest Descent

∀αxk+1 − xk

α= −g′(xk ) ,

Define x(t) as a function of time such that x(tk ) = xk andtk+1 = tk + α

dxdt

(tk ) = limα→0

x(tk + α)− x(tk )

α= lim

α→0

xk+1 − xk

α= −g′(xk )

Finding the solution with the steepest-descent method isequivalent to solving P.D.E.:

dxdt

= −g′(x)

Variational Methods

Page 92: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

P.D.E

Variational problem

u = minu

F (u(x))

Must satisfy E-L equation

⇒ F ′(u) = 0

Find the solution with the steepest-descent method

uk+1 = uk − αF ′(uk ) ,

where α is the step length and must be determined∀α

uk+1 − uk

α= −F ′(uk ) ,

Variational Methods

Page 93: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

P.D.E

Variational problem

u = minu

F (u(x))

Must satisfy E-L equation

⇒ F ′(u) = 0

Find the solution with the steepest-descent method

uk+1 = uk − αF ′(uk ) ,

where α is the step length and must be determined∀α

uk+1 − uk

α= −F ′(uk ) ,

Variational Methods

Page 94: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

P.D.E

Variational problem

u = minu

F (u(x))

Must satisfy E-L equation

⇒ F ′(u) = 0

Find the solution with the steepest-descent method

uk+1 = uk − αF ′(uk ) ,

where α is the step length and must be determined

∀αuk+1 − uk

α= −F ′(uk ) ,

Variational Methods

Page 95: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

P.D.E

Variational problem

u = minu

F (u(x))

Must satisfy E-L equation

⇒ F ′(u) = 0

Find the solution with the steepest-descent method

uk+1 = uk − αF ′(uk ) ,

where α is the step length and must be determined∀α

uk+1 − uk

α= −F ′(uk ) ,

Variational Methods

Page 96: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

P.D.E

Make u also function of time, i.e., u(x , t)

uk (x) ≡ u(x , tk )

and tk+1 = tk + α

limα→0

uk+1 − uk

α≡ ∂u∂t

(x , tk )

Solving the variational problem with the steepest-descentmethod is equivalent to solving P.D.E.:

∂u∂t

= −F ′(u)

+boundary conditions.

Variational Methods

Page 97: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

P.D.E

Make u also function of time, i.e., u(x , t)

uk (x) ≡ u(x , tk )

and tk+1 = tk + α

limα→0

uk+1 − uk

α≡ ∂u∂t

(x , tk )

Solving the variational problem with the steepest-descentmethod is equivalent to solving P.D.E.:

∂u∂t

= −F ′(u)

+boundary conditions.

Variational Methods

Page 98: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Steepest descent

Variational Methods

Page 99: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Steepest descent

Variational Methods

Page 100: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Steepest descent

Variational Methods

Page 101: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Steepest descent

Variational Methods

Page 102: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Steepest descent

Variational Methods

Page 103: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Steepest descent

Variational Methods

Page 104: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Steepest descent

Variational Methods

Page 105: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Differential Calculus x Variational Calculus

Differential Calculus Variational Calculus

Problem Spec. function function of function= functional

Necess. Cond. 1st derivative = 0 1st variation = 0Result one number (or vector) function

Variational Methods

Page 106: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Optimization Problem

Solving PDE’s is equivalent to optimization of integralfunctionals

ut − F ′(u) = 0 ⇔ min F (u)

Example

ut = ∆u ⇔ min∫

Ω|∇u|2

Does every PDE have its corresponding optimizationproblem?

Variational Methods

Page 107: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Optimization Problem

Solving PDE’s is equivalent to optimization of integralfunctionals

ut − F ′(u) = 0 ⇔ min F (u)

Example

ut = ∆u ⇔ min∫

Ω|∇u|2

Does every PDE have its corresponding optimizationproblem?

Variational Methods

Page 108: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Optimization Problem

Solving PDE’s is equivalent to optimization of integralfunctionals

ut − F ′(u) = 0 ⇔ min F (u)

Example

ut = ∆u ⇔ min∫

Ω|∇u|2

Does every PDE have its corresponding optimizationproblem?

Variational Methods

Page 109: Variational Methods in Image Processing - CASzoi.utia.cas.cz/files/var_calculus_2.pdfVariational Methods in Image Processing ... Motivation Derivation of Euler-Lagrange Equation Variational

Introduction Motivation E-L PDE

Optimization Problem

Solving PDE’s is equivalent to optimization of integralfunctionals

ut − F ′(u) = 0 ⇔ min F (u)

Example

ut = ∆u ⇔ min∫

Ω|∇u|2

Does every PDE have its corresponding optimizationproblem?

Variational Methods