p. pegon & ph. on & ph. cacapéran - cea/ceaclub cast3m, 21th of november 2008 an amazing...

Club Cast3M, 21th of November 2008Club Cast3M, 21th of November 2008

An amazing optimisation problemAn amazing optimisation problemAn amazing optimisation problemAn amazing optimisation problem

P. Pegon & Ph. P. Pegon & Ph. CapéranCapérangg ppEuropean Laboratory for Structural AssessmentEuropean Laboratory for Structural Assessment

Joint Research CentreJoint Research CentreIspraIspra, Italy, Italy

An amazing An amazing optimisationoptimisation problemproblem

OUTLINE:

• Background: some results from photogrammetry

Th (VERY) i lifi d ti i ti bl• The (VERY) simplified optimisation problem

• EXCE, steepest descent, conjugate gradient & BFGS

• The Gauss-Newton method (+ new operator GANE)The Gauss Newton method (+ new operator GANE)

• Conclusions, further works & bibliography

Background: some results from Background: some results from photogrammetryphotogrammetry

The wall The cameraThe structureESECMASE

20cmm

4096 grey levels, S/N 64 dB, 1536x1024 pixels

Kodak CCD KAF 1602EKodak CCD KAF 1602E

+-1.3mm/pixel

Current tiI iti l t


Current state

i

Set of M interpolation

Reference stateInitial t0

Set of M interpolation functions on M reference

sub-windowsSet of M current sub-windows

under tracking

Given one reference, C3 interpolation function,

( )0W X

Consider its corresponding current discrete function

( )iw x

Optimisation on the Quadratic Error:

( ) ( )( )( )2

∑C3 Analytic Template { } { }1.. ... ..i i

n nP P +

( )With respect to { }nP

( ) ( )( )( )2

01..

( ) ) ;k kk K

E w x W T x=

= −∑P P( )0 ( ; )kW T x P

Tracking MethodCoordinate Transformation

parametrised by jdijd( );T X P

{ }nP


Comparison with Heidenhain measurement (K14 - 0.16g)


Displacement on targets


Displacement on texture


Displacement fields (K12 - 0.12g)


OUTLINE:






The (VERY) simplified The (VERY) simplified optimisationoptimisation problemproblem

In the elastic case…

65 bricks , 1,...,65 0.3iE i ν= =

Joints 66 66

66 ,2 (1 )n s

E EE K K

e e ν→ = =

+

Identification of ??? E


d is imposed

d is measured

1+

=EThe reference solution is obtained with and( )+ +d E ( ) 5%σ + =E1EThe reference solution is obtained with and ( )d E ( ) 5%σ E


{ }* argmin ( )F= EE EFind

( ) ( ) ( )T( )F = =E f E f E f E( ) ( ) ( )( )( )

( ) ( )+ += −f E d E d EWhere

(first) problem !!! ( ) ( )λ=d E d E


Numerical derivation of the gradients

( ) ( )( ) F E E E E F E E E EF δ δ+∂ Ε 1 1( ,..., ,..., ) ( ,..., ,..., )( )2

i n i n

i

F E E E E F E E E EFE E

δ δδ

+ − −∂ Ε ≈∂

And also (later)

1 1( ,..., ,..., ) ( ,..., ,..., )( ) i n i nE E E E E E E Eδ δ+ − −∂ Ε f ff 1 1( , , , , ) ( , , , , )( )2

i n i n

iE Eδ≈

∂


OUTLINE:






EXCE, EXCE, steepeststeepest descentdescent, , conjugateconjugate gradient & BFGSgradient & BFGS

EXCE

The EXCELL operator computes the minimum of a function F(Xi),the usemethod is the well-known MMA (Method of Moving Asymptotes) proposed by K.Svanberg. The purpose is to find the minimum of a function F(Xi) with i=1,N given that :

- the relations Cj(Xi) < Cjmax (j>0 and j=1,M) must be satisfied

- there are relations on each unknown Ximin < Xi < Ximax

The functions F and Cj are defined by the function values and by their derivatives at the starting point X0.

( ) , ( )i j iF E C E ?????


( ) ( ) ( )F + += −E d E d E

EXCE

1st case 1

( ) ( ) ( )

( ) ( ) ( ) 0

( ) 0 ( ) 0

C

C C

σ σ +

+ +

= − <

= − < = − <

E E E

E E E E E E2 3( ) 0 , ( ) 0C C= < = <E E E E E E

( ) ( ) ( )F + += −E d E d E2nd case

21 2

( ) ( ) ( )

( ) ( ) ( ) 0 , ( ) ( ) ( /100) 0

F

C C orσ σ+ ++= − < = − <

E d E d E

E E E E E E E

3rd case … but no4

( ) ( )

( ) ( ) ( ) / ( ) 1 0 10

F

C

σ+ + + + −

=E E

E d E d E d E convincing results41( ) ( ) ( ) / ( ) 1.0 10C + + + + −= − < ×E d E d E d E


Descent method: general format

D t di ti ithiE 0i T F∇ <EDescent direction with

Line search Robust line search???

idE 0id F∇ <

EE

{ }0argmin ( )i i idFα α= +E ELine search Robust line search???

and loop on until convergence

{ }0argmin ( )dFαα α> +E E

i1i i idα+ = +E E E

Meaning of convergence???

Here we concentrate on !!!!!!!iEHere we concentrate on !!!!!!! dE


Steepest descent iist F= −∇

EE

( ) ( ) / ( )+ + + +−d E d E d E

i

The convergenceThe convergence can be very slow!s


Conjugate gradient : the descent directions are mutually orthogonal

(with precise line search)1

121

( )i

i i T ii i i i i igc gc

iFγ γ

−−

−

−= + = −∇ =E

g g gE g E ggg

( ) ( ) / ( )+ + + +−d E d E d E

i

Surprisingly slow convergences

i


BFGS: the Hessian matrix is iteratively constructed

1 ...ii i i ibfgs bfgs bfgs bfgsF −= − ∇ = +

EE H H H

( ) ( ) / ( )+ + + +−d E d E d E

i

BFGS=Broyden-Fletcher-Goldfarb-ShannoSurprisingly slow

convergences


Summary:

• Very slow convergence

• “High” level asymptotic convergence plateau

Conclusion: we need to do better!Conclusion: we need to do better!


OUTLINE:






The GaussThe Gauss--Newton method (+ new operator GANE)Newton method (+ new operator GANE)

Write the problem as a least square problem…

{ }* argmin ( )F= xE EFind

( )( ) ( ) ( ) ( )2 2 T1 1 1( )2 2 2

m

iF f= = =∑E E f E f E f E( )( ) ( ) ( ) ( )

( )12 2 2

( ) ( )i=

+ += −

∑f E d E d E

Where

… and use it!


The Gauss-Newton MethodSame cost as

computing s/ jF E∂ ∂

Using with( ) ( ) ( )δ δ+ +f E E f E J E E ( ) ( )i

ijj

fE

∂⎡ ⎤ =⎣ ⎦ ∂J E E

is given by

j

{ }argmin ( )gn Fδ δ= +EE E E ( )T Tgn F= − = −∇J J E J f

Use as a descent directiongnE

Warning: no indetermination if the columns of are linearly independent

Jindependent


Operator GA(us-)NE(wtonTAB2=GANE TAB1;

CHPO1 RIGI1=GANE TAB1 'MATR';;

Description:____________

The function to minimize is 2F(X)=f(X).f(X), and J(X)=df/dX

The descent direction is H solution of:transpose(J)J H = -transpose(J)f

Contents:_________

TAB1 : Table of type 'VECTEUR' containingTAB1 . 0 = f(X)TAB1 . 1 = df/dX1

.TAB1 . n = df/dXn/

CHPO1 : -transpose(J)fRIGI1 : transpose(J)J

TAB2 : TABLE of type 'VECTEUR' containingTAB2 1 H1TAB2 . 1 = H1

. TAB2 . n = Hn


( ) ( )λ=d E d EThere is a second problem!!! (first problem )( ) ( )λ=d E d EThere is a second problem!!! (first problem )

• is almost always rank 1 deficient (use of RESO ‘ENSE’)( )TJ J

{ }65∂ ∂∑f f

• For instance: ( for )

BUT

1ia =− { }1=E66166

( )ii i

aE E=

∂ ∂=∂ ∂∑f fE

66 65( )E g= E

• GANE can be used to find out the ( )

• Assuming with with

ia 65 65 6566

( )T

E∂=

∂fJ J a J

/ i ig E a∂ ∂ = 65J ia∂ ∂ ∂= +f f f66 65( )gAssuming with with

• Use GANE again for finding and complete with 65

661

gn i gnii

E aE=

=∑i ig 65

66i

i iE E E∂ ∂ ∂

65gnE


And finally!

( ) ( ) / ( )+ + + +−d E d E d E

i

… with

i

7max( ) 10i iE E+ −− ≈( )i i


66E+Substituting with (and still starting from ) 66 , 1,2, 4,8nE n+ = { }1=E

1n 1n =2n =

3n =4n =

WARNING

• Sometimes RESO takes care itself of the indetermination…7E E+ +

• with 7max( ) 10i iE E −+− ≈

E E≠E E


OUTLINE:






Conclusions, further works & bibliographyConclusions, further works & bibliography

Conclusions & further works

• Nothing is given for free!

• Cast3M demonstrates again its ability to help to understand (and solve) problems!(and solve) problems!

• Gauss-Newton is the root for other classical methods

• Damped Gauss-Newton = Levenberg-Marquardt methodT T (already in GANE)

• Powell’s Dog Leg Method

• Use of noisy experimental data

( )T Tlm Fμ+ = − = −∇J J I E J f

• Use of noisy experimental data

• Identification of non-linear models with more refined meshes

• Line search, convergence tests…, g

Conclusions, further works & bibliographyConclusions, further works & bibliography

Short bibliography

• W.H. Press, S.A. Teukolsky, W.T. Vetterling & B.P. Flannery,Numerical Recipes : The Art of Scientific Computing, 1986u e ca ec pes e t o Sc e t c Co put g, 986

• H. Matthis & G. Strang,The Solution of Non-Linear Finite Element EquationsIJNME, 14, 1613-1626, 1979IJNME, 14, 1613 1626, 1979

• K. Madsen, H.B. Nielsen & O. TingleffMethods for Non-Linear Least Squares ProblemsTechnical University of Denmark 2004Technical University of Denmark, 2004

• T. Charras & J. KicheninL’optimisation dans CAST3M

p. pegon & ph. on & ph. cacapéran - cea/ceaclub cast3m, 21th of november 2008 an amazing...

Documents