L10 Optimal Design L.Multipliers• Homework• Review• Meaning & of Lagrange Multiplier • Summary

1

Homework 4.44

2

)()(

)()()()(

xxνx,

xxνx,xx

hυFL

hυfLFf

Now is a “minimize”

We have only used “necessary conditions”We cannot yet conclude that the pt is a MIN!

4.44 cont’d

3

H(x) is negative definite, therefore the candidate pt is not a local min. (therefore Pt A is NOT a max of the original F(x)). Unbounded?

33.8)( xF

4.0)( xF

0.6)( xF

Prob 4.54

4

))(()(

))(()()()(

2

2

sguFL

sgufLFf

xxνx,

xxνx,xx

OKs ,42

Gaussian Elimination Case 2

5

4011

001113

0158

4011

0165

0158

401101650158

21

21

21

uxxuxxuxx

833.1)1666.2(1113

166.2)4(1324

)4(130240

001113

0158

1

1

2

2

xx

xx

x R1 by -1+ to R2

x R3 by -13+ to R2

Prob 4.57

6

Gaussian Elmination 4.57 Case 1 u=0

7

240)0(22

0002040222102

201140222102

201121202102

1

1

2

21

21

21

21

21

21

21

21

21

xυx

xυxxυxxυxx

υxxυxxυxx

υxxυxxυxx

+R1 to R2

x R2 by -1/2+ to R3

BacksubUsing R2 infeasible therefore2

0402

04

21

21

2121

s

sg

sxxgCheck feasibility

Gaussian Elmination 4.57 Case 2 s=0

8

+R3 to R4

BacksubUsing R3

0sCheck feasibility

320)1(11

12020

20112112021102

40112011

2112021102

2

1

2

21

21

21

21

21

21

21

21

xυx

xxx

υxxuυxxuυxx

xxυxx

uυxxuυxx

Prob 4.57

9

2υ,0,0,2,0:1 Case 21 sxxu

2υ,2,1,3,0:2 Case 21 uxxs

Prob 4.59

10

]2[]4[

...)1()1(

)()(),(

22212

21211

22

21

2222

211121

sxxusxxu

xxL

sgusguxxfL

2

4

)1()1(),(

212

211

22

2121

xxg

xxgST

xxxxfMin

Prob 4.59

11

0,0

0,0

0,0

02

04

0)1(2

0)1(2

2211

222

121

22212

21211

2122

2111

susu

us

us

sxxg

sxxg

uuxx

L

uuxx

L

Prob 4.59

12

Gaussian Elmination 4.59 Case 4 s1,2=0

13

+R3 to R4

BacksubUsing R3

Check feasibilityBoth s1 and s2 =01

3

20114011

2112021102

211

411

2112021102

2

1

21

21

2121

2121

2221

2121

2121

2121

xx

xxxx

uuxxuuxx

sxx

sxx

uuxxuuxx

Prob 4.59

14

]2[]4[

...)1()1(22212

21211

22

21

sxxusxxu

xxL

Where is: Case 1Case 2Case 3Case 4

MV OptimizationInequality & Equality Constrained

15

nkx x x

mjg

pi= h

f :MINIMIZE

(L )kk

(U )k

j

i

1

...10)(

...10)(: ToSubject

) (

x

x

x

KKT Necessary Conditions for Min

16

))(( )()( 2

11i

m

iii

p

iii sguhυfL

xxxsu,v,x,

nkx

gu

x

hυ

x

f m

j k

jj

p

i k

ii

k

to1for 011

mjsg

ph

jj

i

to1for 0*)(

to1ifor 0*)(2

x

x

Regularity check - gradients of active inequality constraints are linearly independent

mjs j to1for 02 mjsu jj to1for 0*

mju j to1for 0*

Relax both constraints (Prob 4.59)

171

)11()12(

)1()1(22

22

21

ff

xxf

2

4

212

211

xxg

xxg

1

3

then,1,let

2

4

02

04

212

211

21

2212

1211

212

211

xxg

xxg

ee

exxg

exxg

xxg

xxg

Constraint Variation Sensitivity Theorem

18

ijj

iii

ub

f

e

f

υb

f

b

f

)(*

)(*

x*

x*

The instantaneous rate of change in the objective function with respect to relaxing a constraint IS the LaGrange multiplier!

Practical Use of Multipliers in 4.59

19

jjiiji

jj

ii

ji

eubυfebf

ee

fb

b

ffebf

)0,0(*),(

)0.0(*)0,0(*)0,0(*),(

0)1(2)1(24)1,1(1,2

)0,0(*),(

2,1

fbu

eubυfebf

j

jjiiji

The first-order approximation on f(x), of relaxing a constraint is obtained from a Taylor Series expansion:

f(actual)=1versusf(approx)=0

Summary• Min =-Max, i.e. f(x)=-F(x)• Necessary Conditions for Min• KKT point is a CANDIDATE min!

(need sufficient conditions for proof)

• Use switching conditions, Gaussian Elimination to find KKT pts

• LaGrange multipliers are the instantaneous rate of change in f(x) w.r.t. change in constraint relaxation.

20