engineering optimization – concepts and applications engineering optimization concepts and...

Engineering Optimization – Concepts and Applications

Engineering Optimization

Concepts and Applications

Fred van KeulenMatthijs Langelaar

CLA [email protected]

http://www.tudelft.nl/


Contents

● Sensitivity analysis



Sensitivity analysis

● Sensitivity: derivative of response w.r.t. design variable:

ssu );(VMVM

i

VM

i

VM

i

VM

ssds

d

u

u

● Note: components of s should be independent!

u constant si constant

Sensitivity of system response (state variables)

Total derivative

Partial derivatives



Sensitivity analysis (2)

● What for?

– Avoid curse of dimensionality by using higher-order

optimization algorithms (gradient-based, Newton, …)

– Examine sensitivity / robustness of optimized design

solutions (parameter sensitivity)

● When?

– Attractive when sensitivity information can be obtained

relatively cheaply



Logarithmic sensitivity

● Definition:

● Advantages:

– Dimensionless, allows comparisons between parameters

– Clearly indicate the relative “strength” of the influence of

parameters:

>1: influential, important parameter

<<1: not very influential parameter

xd

fd

xd

fd

L

L

log

log

dx

df

f

x

dxx

dff 1

1

xdx

xd 1log



Example logarithmic sensitivity

● Logarithmic sensitivity gives information on relative importance

● Always use logarithmic sensitivities when comparing sensitivity

values of different variables!

af x 1adfax

dx

1 1a

a aLa a

L

d f x x xax ax a a

d x f x x



Aspects of sensitivity analysis

● Implementation effort

● Efficiency

● Accuracy and consistency

Response

Design variable

ExactNumericalmodel



Sensitivity analysis approaches

● Global finite differencesInvolves repetitive design evaluations

● Discrete derivativesBased on differentiation of numerical model

● Continuum derivativesBased on differentiation of governing equations

Implementation Efficiency

Very easy Terrible*

Moderate As good

as it gets

Lots of work As good

as it gets



Sensitivity analysis approaches (2)

Schematically:

Model

Model

xx

x+x

f

f+f +-

xd

df GFD

Governingequations

Discretization Differentiationx

xd

df Discrete

Governingequations

DiscretizationDifferentiationx

xd

df Continuum



Automated differentiation

● Automatic generation of code that computes

sensitivities:

● Many different tools exist: ADIFOR, ADOL-F (Fortran),

ADIC, ADOL-C (C/C++), …

● Convenient, but generally code is several times slower

than hand-coded derivatives

Analysiscode

Automaticdifferentiation

Derivativecode



Finite difference derivatives

● Finite differences for sensitivity analysis (GFD):

– Simple

– Computationally inefficient (however …)

– Accuracy depends on design perturbation

32 '''!3

1''

!2

1'

!1

1)( hfhfhffhxf

● Based on Taylor series:

)(''!2

1'

!1

1)()( 32 hOhfhfxfhxf

)(''!2

1)()(' 2hOhf

h

xfhxff



Finite difference derivatives (2)

● First order forward / backward FD:

)('''!3

1''

!2

1'

!1

1)( 432 hOhfhfhffhxf

)('''!3

2'2)()( 43 hOhfhfhxfhxf

-

)(''!2

1)()(' 2hOhf

h

xfhxff

)(''

!2

1)()(' 2hOhf

h

hxfxff

)('''!3

1''

!2

1'

!1

1)( 432 hOhfhfhffhxf ● Central FD:

)('''!3

1

2

)()(' 32 hOhf

h

hxfhxff



Finite difference derivatives (3)

● Similarly:

)('''!3

6)()(2)2('' 2

2hOhf

h

xfhxfhxff

(forward))(

!4

2)()(2)('' 32)4(

2hOhf

h

hxfxfhxff

(central)

● Forward FD error analysis:

)()()( xxfxf mm

'mf

)(''!2

1)()()()()(' 2hOhf

h

xhx

h

xfhxfxf m

mmmm

Condition error Truncation error



FD accuracy

● Perturbation h determines error:

2( ) ( )'( ) (

1''

(

2!

()

) )m m m mm

f x h f xf x Of h

hh

x h x

h

h

Error



Practical aspect: noise

● Numerical noise can spoil FD accuracy!

● Example of noise source: effect of remeshing

Normalized stress constraint

Hole radius



Nonlinear elastic case

● Relatively cheap FD

sensitivities (exception):

– Solution technique:

incremental-iterative approach

Involves solution of many linear

systems, e.g.

Load

Displacement

T K u R

– FD: start the solution process for

the perturbed case from the

unperturbed solution● Much less expensive than full analysis!



Nonlinear path-independent case

● Consider: 0ssug ));(( 0ffR intext(e.g. )

● Solution obtained by Newton iterations:

uuugu

gu

kkd

d1

1

,

● For FD, solve perturbed case by iterating from nominal

solution:

0δεssug0εsug 11 )*;()*;(

2));(( εssssug



Nonlinear path-independent case● Pitfall: make sure to include the finite residuals in the

FD calculation!

– Consider first iteration for perturbed case:

ssssssd

dss kk

);();()(1

1 uguu

gu

– For small design perturbation, this approaches:

ssssd

ds kk );();()(

1

1 uguu

gu

Original

residual

– Interpretation: just an additional Newton iteration

Original residual dominates over effect of design perturbation



Finite residual problem: solution

● To improve FD accuracy with finite residuals:

instead of solving

solve

i.e. subtract original residual from new residual.

0ssssug );(

0ssugssssug );();(

● Ok for s = 0. Original residual no longer dominates



Finite difference summary

● Easy to implement, black box approach

● Inefficient, except for nonlinear path-independent and

explicitly solved transient case

● Choice of proper relative design perturbation critical

● No adjoint formulation possible: unattractive in cases

with many design variables and few responses



State variable vector sensitivity

● Then:

sds

d

sds

du

uds

d Ti

i

u

u

Discrete derivatives

● Consider linear discretized equations

(e.g. linear elastic FE model):

and response (e.g. equivalent stress):

fKufuK 1)()()( sss

( );VonMises s s u



State variable sensitivity

● State variable derivatives follow from differentiation of

original equation:

ds

d

ds

d

ds

dsss

fuKu

KfuK )()()(

uKfu

Kds

d

ds

d

ds

d

u

KfK

u

ds

d

ds

d

ds

d 1

Pseudo-load vector

● Decomposed K

already available (direct solver)!



State variable sensitivity (2)

● Nonlinear case similar:

sds

dss

g

u

gu0ug

1

));((

u

KfK

u

ds

d

ds

d

ds

d 1

Already decomposed tangent matrix(direct solver)



Semi-analytical approach

● Semi-analytical: use FD to compute pseudo-load:

u

KfK

u

ds

d

ds

d

ds

d 1

u

KKffK

s

sss

s

sss )()()()(1

● Advantages:

– Easy implementation (can be done at top level)

– Efficient computation



Alreadydecomposed

SA: nonlinear case

● Geometrically nonlinear (history-independent) setting:

0ug ));(( ss

0u

u

ggg

ds

d

sds

d

sds

d

g

u

gu1

s

sssss

s

);();( ugugg

● SA approach: computed using FD:s

g



CHEAP!!


● Note, computation of discrete derivatives

– Only involves a linear equation, also in nonlinear case

– Allows re-use of the decomposed system matrix

Sensitivity analysis much cheaper than analysis itself!

● Options for calculation of pseudo-load vector:

a) Analytical differentiation (lots of work)

b) Automated differentiation (code generator programs)

c) Finite difference approach



SA accuracy problem

● Accuracy of semi-analytical (SA) sensitivities w.r.t.

shape variables reduces for cases with substantial

rotations (slender structures)

● Problem increases with mesh refinement!?!

SEE APPENDIX



Eigenvalue sensitivities

● Important class of responses: eigenvalues

● Discrete sensitivity analysis:

0vKM

0vKMMvvKM '''' :s

:*Tv 0vKMvMvvvKMv '''' TTT

Mvv

vKMvT

T '''

0



Eigenvalue sensitivities (2)

● Result:

● Note, no need to compute eigenvector sensitivities v’!

If needed, one can use Nelson’s method

(but rather expensive)

Mvv

vKMvT

T '''

● Difficulties: eigenvalue multiplicity, mode switching ...



Contents

● Sensitivity analysis:

– Brief recap discrete / SA approach

– Adjoint method

– Continuum sensitivities

● Topology optimization

● Closure



Adjoint discrete sensitivities

● Discussed direct approach:

j

i

j

i

j

i

ds

dh

s

h

ds

dh u

ussuhh

));((

and

uKf

Ku

sfsusKjjj ssds

d 1)()()(

● One backsubstitution needed for every design variable:

not attractive for many design variables

● Alternative: adjoint formulation



Adjoint sensitivities

● Starting point: augmented response:

)()()());((* susKsfλssuhh T

jjj

Ti

j

i

j

i

j

i

ds

d

ssds

dh

s

h

ds

dh uKu

Kfλ

u

u

*

= 0

j

Ti

i

jj

Ti

j

i

ds

dh

sss

h uKλ

uu

Kfλ

● To avoid computation of state vector derivatives,

choose i such that vanishes!jds

du



Adjoint sensitivities (2)

● Result:

j

Ti

jj

T

j

i

j

i

ds

dh

sss

h

ds

dh uKλ

uu

Kfλ

T

iTi

Ti

i hh

uKλ0Kλ

u

uKf

λjj

Ti

j

i

j

i

sss

h

ds

dh

● One backsubstitution per response: attractive in case of

many design variables and few responses



Adjoint vs. direct

● Direct method attractive when #variables < #responses,

adjoint method attractive when #variables > #responses

● Note, adjoint method requires load vector composed of

response derivatives (specific implementation)● Difference consists of order of computations:

uKf

Ku jj

i

j

i

j

i

ss

h

s

h

ds

dh 1

Tiλ jds

du



Sensitivities in transient case

● Transient analysis: );,();();( 11 sttstst iiii

iii ttt ds

d

ds

d

ds

d

1

● Sensitivities at time ti depend on sensitivities at

previous instants

– Direct method: forward time integration of sensitivities

– Adjoint method: backward time integration of sensitivities

(unattractive, storage problem)

● FD often preferred for explicitly solved transient problems



Discrete derivative summary

● Generally efficient and easy to implement, particularly semi-analytical case (combination with FD)

● Reuse of decomposed stiffness matrix (with direct solver – with iterative solver, reuse of preconditioner)

● Direct and adjoint versions

● SA: accuracy problems for structures under large rotations (beams, shells)

Governingequations

Discretization Differentiationx

xd

df



Contents

● Sensitivity analysis:

– Brief recap discrete / SA approach

– Adjoint method

– Continuum sensitivities

● Topology optimization

● Closure



Continuum derivatives

● Example: beam bending

(Euler-Bernoulli beam)

Governingequations

DiscretizationDifferentiationx

xd

df

q(x)

x I(x,s)

),(,2

2

sxEIMdx

wd

0

2

2

qdx

EId

+ boundary conditions

Governing equation:



Continuum derivatives (2)

● Now differentiate w.r.t. s:

02

2

2

2

ds

dEI

dx

d

ds

dIE

dx

d Sensitivity equation

● For nonlinear / complex problems, the continuum

sensitivity equations are often simpler

02

2

qEIdx

d Governing equation

● Compare:



Sensitivity analysis summary

● Sensitivities important in optimization:

– Efficient higher-order optimization algorithms

– Evaluation of robustness of results

● Choice of sensitivity analysis method depends on:

– Number of design variables vs. number of responses (adjoint vs. direct)

– Type of model (cheap / expensive, linear / nonlinear / transient)

– Implementation effort, access to source code



Sensitivity analysis summary (2)

Finite difference


Semi-analytical

Continuum derivatives

Impl

emen

tatio

nEf

ficie

ncy

Accu

racy

Adjo

int m

ode

Poin

ts o

f atte

ntio

n

• Perturbation size critical• Efficient for nonlinear elastic

& explicit transient case

• Inaccurate for large rotations• Remedies: exact / refined version


engineering optimization – concepts and applications engineering optimization concepts and...

Documents

sensitivity information

derivative of response

u constant s

design variable

influence of parameters

components of s

important parameter

relative strength