non linear programming problems
DESCRIPTION
OptimisationTRANSCRIPT
![Page 1: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/1.jpg)
Non-linear Programming Problem Problem
![Page 2: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/2.jpg)
Classical Optimization
• Classical optimization theory uses differential calculus to determine points of maxima and minima for unconstrained and constrained functions.
• This chapter develops necessary and sufficient conditions for determining maxima and minima of unconstrained and constrained functions.
2
![Page 3: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/3.jpg)
Quadratic forms
Consider the function
nnnn
nnn
xxaxxaxaxaxaXf
1,12112
22222
2111)(
−−++
++++=
⋯⋯
3
The above function is called the quadratic or quadratic form in n-variables.
The above function can be written in the form XTAX, where X = [x1, x2, …, xn]T be a n-vector and A = (aij) be a n × n symmetric matrix.
nnnn xxaxxa 1,12112 −−++⋯
![Page 4: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/4.jpg)
Quadratic formsThen the quadratic form
(or the matrix A (symmetric matrix)) is called• Positive semi definite if XTAX ≥ 0 for all X ≠ 0.
2
1( ) 2T
ii i ij i ji i j n
Q X X A X a x a x x≤ ≤ ≤
= = +∑ ∑∑
• Positive semi definite if XTAX ≥ 0 for all X ≠ 0.• Positive definite if XTAX > 0 for all X ≠ 0.• Negative semi definite if XT A X ≤ 0 for all X ≠ 0.• Negative definite if XTA X < 0 for all X ≠ 0.The above quantities of the quadratic form depend on the symmetric matrix A, therefore we have
4
![Page 5: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/5.jpg)
1. Matrix minor testA necessary and sufficient condition for A (any square matrix ) to be :Positive definite (positive semi definite) is that all the nprincipal minors of A are > 0 ( ≥ 0).
Negative definite (negative semi definite) if kth principal
5
Negative definite (negative semi definite) if kth principalminor of A has the sign of (-1)k, k = 1, 2, …,n(kth principal minor of A is zero or has the sign of (-1)k,k=1,2,…,n )
Indefinite, if none of the above cases happen
![Page 6: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/6.jpg)
2. Eigenvalue Test
Since matrix A is a real symmetric matrix in XTAX, it follows that its eigenvalues ( λi) are real. Then XTAX is
Positive definite (positive semi definite) if λi > 0(λi ≥ 0) i =1, 2, …,n.
6
Negative definite (negative semi definite) if λi < 0 ,(λi ≤ 0) i=1,2,…,n
Indefinite, if A has both positive and negative eigenvalues.
![Page 7: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/7.jpg)
Examples : Decide the definiteness of the following quadratic functions:
(1)
(2)
23
22
21 53 xxx ++
32312123
22
21 4247107 xxxxxxxxx +−+−−−
7
(2)
(3)
323121321 4247107 xxxxxxxxx +−+−−−
22
21 xx −
![Page 8: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/8.jpg)
Local maximum/Local minimum
Let f (X) = f (x1, x2,…,xn) be a real-valued function of the n variables
x1, x2, …, xn (we assume that f (X) is at least twice differentiable ).
A point X0 is said to be a local maximum of f (X) if there exists an
ε > 0 such thatε > 0 such that
Here
A point X0 is said to be a local minimum of f (X) if there exists an
ε > 0 such that
0 0( ) ( ) for all jf X h f X h ε+ ≤ ≤"
0 0 0 0 0 01 2 0 1 2 0 1 1 2 2( , ,.., ) , ( , ,.., ) and ( , ,.., ) .T T T
n n n nh h h h X x x x X h x h x h x h= = + = + + +""
8
0 0( ) ( ) for all jf X h f X h ε+ ≥ ≤"
![Page 9: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/9.jpg)
Absolute maximum/Absolute minimum
X0 is called an absolute maximum or global maximum of f (X) if
X0 is called an absolute minimum or global minimum of f (X) if
0( ) ( ) .f X f X X≤ ∀
X0 is called an absolute minimum or global minimum of f (X) if
0( ) ( ) .f X f X X≥ ∀
⋯++∇=−+ HhhhXfXfhXf T
21)()()( 000
Taylor Series expansion for functions of several variables
![Page 10: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/10.jpg)
TheoremA necessary condition for X0 to be an optimum
point of f (X) is that
(that is all the first order partial derivatives arezero at X0.)
0)( 0 =∇ Xf
ixf
∂
∂
Definition: A point X0 for which is calleda stationary point of f (X) (potential candidate forlocal maximum or local minimum).
0)( 0 =∇ Xf
10
![Page 11: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/11.jpg)
Let X0 be a stationary point of f (X). A sufficient conditionfor X0 to be a
local minimum of f (X) is that the Hessian matrixH(X0) is positive definite;
Theorem
11
H(X0) is positive definite;
local maximum of f (X) is that the Hessian matrixH(X0) is negative definite.
![Page 12: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/12.jpg)
Hessian matrix
2 2 2
21 1 2 1
2 2 2
22 1 2 2
. .
. .( ) .
n
n
f f fx x x x xf f f
x x x x xH x
∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ ∂ = 2 1 2 2
2 2 2
21 2
( ) ...
. . .
n
n n n
x x x x xH x
f f fx x x x x
∂ ∂ ∂ ∂ ∂ = ∂ ∂ ∂
∂ ∂ ∂ ∂ ∂
12
![Page 13: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/13.jpg)
Example 1
Examine the following function for extreme points
f (x , x ,x ) = x +2 x +x x –x 2 – x 2 – x 2f (x1, x2,x3) = x1 +2 x3 +x2 x3 –x12 – x 22 – x3
2
13
![Page 14: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/14.jpg)
Definition:
A function f (X)=f (x1,x2,…xn) of n variables is said to beconvex if for each pair of points X, Y on the graph, the linesegment joining these two points lies entirely above or onthe graph.
14
the graph.
i.e. f((1-λ) X + λY) ≤ (1-λ) f (X)+ λ f (Y)
for all 0 ≤ λ ≤ 1.
![Page 15: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/15.jpg)
f is said to be strictly convex if for each pair of points X,Y on the graph,
f ((1-λ) X + λ Y) < (1-λ) f (X)+ λ f (Y)
for all λ such that 0 < λ < 1.
15
for all λ such that 0 < λ < 1.
f is called concave (strictly concave) if – f is convex (strictly convex).
![Page 16: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/16.jpg)
2
2 0d fdx
≥
Convexity test for function of one variable
convex if
A function of one variable f(x) is
2
2 0d fdx
≤
16
concave if
![Page 17: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/17.jpg)
quantity convex Strictly convex
concave Strictly concave
f f - (f )2 ≥ 0 > 0 ≥ 0 > 0
Convexity test for functions of 2 variables
fxx fyy - (fxy)2 ≥ 0 > 0 ≥ 0 > 0
fxx ≥ 0 > 0 ≤ 0 < 0
fyy ≥ 0 > 0 ≤ 0 < 0
17
![Page 18: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/18.jpg)
Results(1)The Sum of two convex functions is convex.
(2) Let f(X) = XTAX. Then f(X) is convex if XTAX is positive semi-definite.is positive semi-definite.
Example: Show that the following function is convex f (x1, x2) = -2 x1 x2 +x1
2 + 2x22
18
![Page 19: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/19.jpg)
Constraint Optimization ProblemsProblems
![Page 20: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/20.jpg)
Constrained optimization ProblemsKarush–Kuhn–Tucker (KKT) conditions:
Consider the problem maximize z = f(X) = f (x1, x2,…, xn)subject to g(X) ≤ 0 ⇒ [g (X) ≤ 0
20
subject to g(X) ≤ 0 ⇒ [g1(X) ≤ 0g2(X) ≤ 0
.gm(X) ≤ 0]
(the non-negativity restrictions, if any, are included in the above).
![Page 21: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/21.jpg)
We define the Lagrangian function
L(X, S, λ) = f(X) – λ [g(X) + s2]
where s, s12, s2
2,..,sm2 ,are the non negative slack variables
added to g1(X) ≤ 0 ,…. gm(X) ≤ 0 to make them into equalities. Therefore
21
L(X, S, λ) = f(X) – [λ1{g1(X) + s12} + λ2{g2(X) +s2
2}
+… +λm{gm(x) + sm2}]
![Page 22: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/22.jpg)
KKT conditions …• KKT necessary conditions for optimality are given
by ,
( ) ( ) 0L f X g XXL
λ∂
= ∇ − ∇ =∂∂
2
2 0, 1, 2, ... ,
( ( ) ) 0
i ii
L S i mSL g X S
λ
λ
∂= − = =
∂
∂= − + =
∂
22
![Page 23: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/23.jpg)
0)()( =∇−∇ XgXf λ
The KKT necessary conditions for maximization problem are :
λ≥0
KKT conditions …
0)(
0)(
0)()(
≤
=
=∇−∇
Xg
Xg
XgXf
i
iiλ
λ
23
i=1,2…m
These conditions apply to the minimization case as well,except that λ≤ 0.
![Page 24: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/24.jpg)
KKT conditions …
gggf m 0..21 =∂
−−∂
−∂
−∂
λλλ
In scalar notation, this is given by
λi ≥ 0 i=1,2,….m
njxg
xg
xg
xf
j
mm
jjj
,...,2,1
0..22
11
=
=∂
∂−−
∂
∂−
∂
∂−
∂
∂λλλ
0)(
0)(
≤
=
Xg
Xg
i
iiλ
24
i=1,2,….m
i=1,2,….m
![Page 25: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/25.jpg)
IMPORTANT: The KKT condition can be satisfied at a local minimum (or max.), a global minimum (or max.) as well as at a saddle point.
We can use the KKT condition to characterize all the stationary pointsof the problem, and then perform some additional testing to determine
25
of the problem, and then perform some additional testing to determinethe optimal solutions of the problem.
![Page 26: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/26.jpg)
Sufficiency of the KKT conditions:
Sense of optimization
maximization
Required conditions
Objective function Solution space
Concave Convex set
26
maximization
minimization
ConcaveConvex
Convex set
Convex set
![Page 27: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/27.jpg)
Example: Use the KKT conditions to find the optimal solution for the following problem:
maximize f(x1, x2) = x1+ 2x2 – x23
subject to x1 + x2 ≤ 1
27
x1 ≥ 0
x2 ≥ 0
![Page 28: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/28.jpg)
Solution: Here there are three constraints ,
g1(x1,x2) = x1+x2 - 1 ≤ 0
g2(x1,x2) = -x1 ≤ 0
g3(x1,x2) = -x2 ≤ 0The KKT necessary conditions for maximization problem are :
28
problem are :
λi ≥ 0
0)(
0)(
0)()(
≤
=
=∇−∇
Xg
Xg
XgXf
i
iiλ
λ
i=1,2…m
![Page 29: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/29.jpg)
Hence the KKT conditions become
01
33
1
22
1
11
1
=∂
∂−
∂
∂−
∂
∂−
∂
∂
xg
xg
xg
xf
λλλ
02
33
2
22
2
11
2
=∂
∂−
∂
∂−
∂
∂−
∂
∂
xg
xg
xg
xf
λλλ
λ1g1(x1,x2) = 0
λ g (x ,x ) = 0 (note: f is concave, g are
29
λ2g2(x1,x2) = 0 (note: f is concave, gi are
λ3g3(x1,x2) = 0 convex, maximization problem
g1(x1,x2) ≤ 0 these KKT conditions are
g2(x1,x2) ≤ 0 sufficient at the optimum point)
g3(x1,x2) ≤ 0 and λ1≥0, λ2≥0, λ3≥0
⇒
![Page 30: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/30.jpg)
i.e. 1 – λ1 + λ2 = 0 (1)
2 – 3x22 – λ1 + λ3 = 0 (2)
λ1(x1 + x2 – 1) = 0 (3)
λ2 x1 = 0 (4)
λ3 x2 = 0 (5)
30
x1 + x2 – 1 ≤ 0 (6)
x1 ≥ 0 (7)
x2 ≥ 0 (8) λ1 ≥ 0 (9)
λ2 ≥ 0 (10) and λ3 ≥ 0 (11)
![Page 31: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/31.jpg)
(1) gives λ1 = 1 + λ2 ≥ 1 >0 (using 10)
Hence (3) gives x1 + x2 = 1 (12)
Thus both x1, x2 cannot be zero, therefore let x1>0, then (4)gives λ2 = 0. therefore λ1 = 1
if now x2 = 0, then (2) gives 2 – 0 – 1 + λ3 = 0
=> λ < 0 not possible
31
=> λ3 < 0 not possible
Therefore x2 > 0, hence (5) gives λ3 = 0 and then (2) gives
x22 = 1/3 => x2 =1/√3.
And so x1 = 1- 1/√3. Therefore
Max f = 1 - 1/√3 + 2/√3 – 1/3√3 = 1 + 2/3√3.
![Page 32: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/32.jpg)
Example: Use the KKT conditions to derive an optimal solution for the following problem:
minimize f (x1, x2) = x12+ x2
subject to x12 + x2
2 ≤ 9
x + x ≤ 1
32
x1 + x2 ≤ 1
Solution: Here there are two constraints,
g1(x1,x2) = x12+x2
2 - 9 ≤ 0
g2(x1,x2) = x1 + x2 -1 ≤ 0
![Page 33: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/33.jpg)
Thus the KKT conditions are:
1 20, 0λ λ≤ ≤ as it is a minimization problem
2x1 - 2λ1x1 - λ2 = 0
1 - 2λ1x2 - λ2 = 02 2( 9) 0x xλ + − =
33
1 1 2
2 1 2
( 9) 0( 1) 0x xx x
λ
λ
+ − =
+ − =
2 21 2
1 2
91
x xx x+ ≤
+ ≤
![Page 34: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/34.jpg)
Now (from 2) gives 01 =λ 12 =λ Not possible.
Hence 01 ≠λ and so 922
21 =+ xx
Assume . So (1st equation of ) (2) gives02 =λ
0)1(2 11 =−λx Since we get x1= 001 ≤λ
(5)
34
0)1(2 11 =−λx Since we get x1= 001 ≤λ
From (5), we get 32 ±=x
2nd equation of (2) says (with ) x2 = -3,01 <λ 02 =λ
Thus the optimal solution is: The optimal value is :
1 2 1 210, 3, , 06
x x λ λ= =− =− = 3z = −
![Page 35: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/35.jpg)
Maximize f(x) = 20x1 + 10 x2
Subject to x12 + x2
2 ≤ 1
x1 + 2x2 ≤ 2
Use the KKT conditions to find an optimal solution of the following problem:
35
x1≥ 0, x2 ≥ 0
max f occurs at x1 = 2/√5, x2 = 1/√5
![Page 36: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/36.jpg)
Quadratic Programming
36
Quadratic Programming
![Page 37: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/37.jpg)
Quadratic Programming …
A quadratic programming problem is a non-linear programming problem of the form
Maximize subject to , 0
Tz CX X DXAX b X= +
≤ ≥
[ ] [ ] [ ], ,..., , , ,..., , , ,...,T TX x x x b b b b C c c c= = =where [ ] [ ] [ ]1 2 1 2 1 2, ,..., , , ,..., , , ,...,T Tn n nX x x x b b b b C c c c= = =
=
mnmm
n
n
aaa
aaaaaa
A
....
..
..
21
22221
11211
=
nnnn
n
n
ddd
dddddd
D
....
..
..
21
22221
11211
where
![Page 38: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/38.jpg)
Quadratic Programming …
! Assume that the n × n matrix D is symmetric and negative-definite.
! This means that the objective function is strictly
38
! This means that the objective function is strictly concave.
! Since the constraints are linear, the feasible region is a convex set.
![Page 39: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/39.jpg)
Quadratic Programming …
In scalar notation, the quadratic programming problem reads:
2
1 1 1Maximize 2
n n
j j jj j ij i jj j i j n
z c x d x d x x= = ≤ < ≤
= + +∑ ∑ ∑ ∑
39
11 1 12 2 1 1
21 1 22 2 2 2
1 1 2 2
1 2
subject to . . .. . .
. . ., , . . . , 0
n n
n n
m m mn n m
n
a x a x a x ba x a x a x b
a x a x a x bx x x
+ + + ≤
+ + + ≤
+ + + ≤
≥
⋮
![Page 40: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/40.jpg)
Wolfe’s Method to solve a Quadratic Programming Problem:
! The solution to this problem is based on the KKTconditions. Since the objective function is strictlyconcave and the solution space is convex, theKKT conditions are also sufficient for optimum.
40
KKT conditions are also sufficient for optimum.
! Since there are m + n constraints, we have m + nLagrange multipliers; the first m of them aredenoted by λ1, λ2 , …, λm and the last n of themare denoted by µ1, µ2 , …, µn.
![Page 41: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/41.jpg)
Wolfe’s Method…
The KKT (necessary) conditions are:
0,...,,,,...,,.1 2121 ≥nm µµµλλλ
njaxdc j
m
iiji
n
iiijj ,...,2,1,02.2
11==+−+ ∑∑
==
µλ
41
13. 0, 1, 2,. . .,
0 , 1, 2,. . .,
n
i ij j ij
j j
a x b i m
x j n
λ
µ=
− = =
= =
∑
14. , 1, 2,. . ., and 0, 1,2,. . .,
n
ij j i jj
a x b i m x j n=
≤ = ≥ =∑
![Page 42: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/42.jpg)
Wolfe’s Method…
Denoting the (non-negative) slack variable for the ith constraint
by si, the 3rd condition can be written in an equivalent form
i
n
jjij bxa ≤∑
=1
42
as:
(Referred to as " Restricted Basis " conditions).
3. 0, 1, 2, . . .,0 , 1, 2,. . .,
i i
j j
s i mx j n
λ
µ
= =
= =
![Page 43: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/43.jpg)
Wolfe’s Method…
Also condition(s) (2) can be rewritten as:
nj
caxd jj
m
iiji
n
iiij
,...,2,1
,211
=
=−+− ∑∑==
µλ
43
and condition(s) (4) can be rewritten as:
14. , 1, 2, . . .,
0 , 1, 2,. . .,
n
ij j i ij
j
a x s b i m
x j n=
+ = =
≥ =
∑
nj ,...,2,1=
![Page 44: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/44.jpg)
Wolfe’s Method…
Thus we have to find a solution of the following m + nlinear equations in the 2n + m unknowns jijx µλ ,,
njcaxd jj
m
iiji
n
iiij ,...,2,1,2
11==−+− ∑∑
==
µλ
n
∑
44
1, 1 , 2 , . . . ,
n
i j j i ij
a x s b i m=
+ = =∑0 , 1 , 2 , . . . , ,
0 , 1 , 2 , . . . ,i i
j j
s i mx j n
λ
µ
= =
= =
0 , 0 , 1 , 2 , . . . , ,0 , 0 , 1 , 2 , . . . ,
i i
j j
s i mx j n
λ
µ
≥ ≥ =
≥ ≥ =
![Page 45: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/45.jpg)
Wolfe’s Method…
Since we are interested only in a " feasible solution“of the above system of linear equations, we usePhase-I method to find such a feasible solution. Bythe sufficiency of the KKT conditions, it will be
45
the sufficiency of the KKT conditions, it will beautomatically the optimum solution of the givenquadratic programming problem.
![Page 46: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/46.jpg)
Example-1:
2 21 2 1 2Maximize 8 4z x x x x= + − −
1 2subject to 2x x+ ≤
46
1 2
1 2
subject to 2, 0.
x xx x+ ≤
≥
![Page 47: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/47.jpg)
Denoting the Lagrange multipliers by λ1, µ1, and µ2, the KKT conditions are:
0,,.1 211 ≥µµλ
1 1 12. 8 2 04 2 0
xx
λ µ
λ µ
− − + =
− − + =
47
2 1 2
1 1 1
2 1 2
4 2 0i.e. 2 8
2 4
xxx
λ µ
λ µ
λ µ
− − + =
+ − =
+ − =
1 2 1 1 1 1 1 2 23. 2, 0x x s s x xλ µ µ+ + = = = =
All variables ≥ 0.
![Page 48: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/48.jpg)
Introducing artificial variables R1, R2, we thus have toMinimizesubject to the constraints
21 RRr +=
4282
2212
1111
=+−+
=+−+
RxRx
µλ
µλ
48
All variables ≥ 0 (We solve by " Modified Simplex " Algorithm).
242
121
2212
=++
=+−+
SxxRx µλ
0221111 === xxS µµλ
![Page 49: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/49.jpg)
Basic r x1 x2 λ1 µ1 µ2 R1 R2 s1 Sol
r 1 0 0 0 0 0 -1 -1 0 0 2 2 2 -1 -1 0 0 0 12
R1 0 2 0 1 -1 0 1 0 0 8 R2 0 0 2 1 0 -1 0 1 0 4 S1 0 1 1 0 0 0 0 0 1 2 r 1 0 0 2 -1 -1 0 0 -2 8
49
r 1 0 0 2 -1 -1 0 0 -2 8 R1 0 0 -2 1 -1 0 1 0 -2 4 R2 0 0 2 1 0 -1 0 1 0 4 x1 0 1 1 0 0 0 0 0 1 2 r 1 0 4 0 1 -1 -2 0 2 0 λ 1 0 0 -2 1 -1 0 1 0 -2 4 R2 0 0 4 0 1 -1 -1 1 2 0 x1 0 1 1 0 0 0 0 0 1 2
![Page 50: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/50.jpg)
Thus we have got the feasible solution
x1 = 2, x2 = 0, λ1 = 4, µ1 = 0, µ2 = 0
50
and the optimal value is: z = 12
![Page 51: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/51.jpg)
Example-2
Maximize 21 1 2 38 2z x x x x= − + +
subject to
51
1 2 3
1 2 3
3 2 12, , 0
x x xx x x+ + ≤
≥
![Page 52: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/52.jpg)
Denoting the Lagrange multipliers by λ1,µ1, µ2, and µ3, the KKT conditions are:
1 1 12. 8 2 02 3 0
x λ µ
λ µ
− − + =
− + =
1 1 2 31. , , , 0λ µ µ µ ≥
Example-2: Solution
52
1 2
1
1 1 1
1 2
1 3
2 3 01 2 3 0. . 2 8
3 22 1
i e x
λ µ
λ µ
λ µ
λ µ
λ µ
− + =
− + =
+ − =
− =
− =
![Page 53: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/53.jpg)
All variables ≥ 0.
1 2 3 1
1 1
1 1 2 2 3 3
3. 3 2 12,0
0
x x x SSx x xλ
µ µ µ
+ + + =
=
= = =
53
Solving this by " Modified Simplex Algorithm ", the optimal solution is:
and the optimal z =
1 2 311 25, , 03 9
x x x= = =
1939
![Page 54: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/54.jpg)
Basic r x1 x2 x3 λ1 µ1 µ2 µ3 R1 R2 R3 S1 Sol
r 1 0 0 0 0 0 0 0 -1 -1 -1 0 02 0 0 6 -1 -1 -1 0 0 0 0 11
R1 0 2 0 0 1 -1 0 0 1 0 0 0 8 R2 0 0 0 0 3 0 -1 0 0 1 0 0 2
R3 0 0 0 0 2 0 0 -1 0 0 1 0 1
54
S1 0 1 3 2 0 0 0 0 0 0 0 1 12
Since λ1 S1 = 0 and S1 is in the basis, λ1 cannot enter.
So we allow x1 to enter the basis and of course by minimum ratio test R1 leaves the basis.
![Page 55: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/55.jpg)
Basic r x1 x2 x3 λ1 µ1 µ2 µ3 R1 R2 R3 S1 Sol
r 1 0 0 0 5 0 -1 -1 -1 0 0 0 3
x1 0 1 0 0 1/2 -1/2 0 0 1/2 0 0 0 4 R2 0 0 0 0 3 0 -1 0 0 1 0 0 2
R3 0 0 0 0 2 0 0 -1 0 0 1 0 1
55
S1 0 0 3 2 -1/2 1/2 0 0 -1/2 0 0 1 8
Since λ1 S1 = 0 and S1 is in the basis, λ1 cannot enter.
So we allow x2 to enter the basis and of course by minimum ratio test S1 leaves the basis.
![Page 56: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/56.jpg)
Basic r x1 x2 x3 λ1 µ1 µ2 µ3 R1 R2 R3 S1 Sol
r 1 0 0 0 5 0 -1 -1 -1 0 0 0 3
x1 0 1 0 0 1/2 -1/2 0 0 1/2 0 0 0 4 R2 0 0 0 0 3 0 -1 0 0 1 0 0 2
R3 0 0 0 0 2 0 0 -1 0 0 1 0 1
56
x2 0 0 1 2/3 -1/6 1/6 0 0 -1/6 0 0 1/3 8/3
As S1 is not in the basis, now λ1 enters the basis .
And by minimum ratio test R3 leaves the basis.
![Page 57: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/57.jpg)
Basic r x1 x2 x3 λ1 µ1 µ2 µ3 R1 R2 R3 S1 Sol
r 1 0 0 0 0 0 -1 3/2 -1 0 -5/2 0 1/2
x1 0 1 0 0 0 -1/2 0 1/4 1/2 0 -1/4 0 15/4 R2 0 0 0 0 0 0 -1 3/2 0 1 -3/2 0 1/2λ1 0 0 0 0 1 0 0 -1/2 0 0 1/2 0 1/2
57
x2 0 0 1 2/3 0 1/6 0 -1/12 -1/6 0 1/12 1/3 11/4
Now µ3 enters the basis .
And by minimum ratio test R2 leaves the basis.
![Page 58: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/58.jpg)
Basic r x1 x2 x3 λ1 µ1 µ2 µ3 R1 R2 R3 S1 Sol
r 1 0 0 0 0 0 0 0 -1 -1 -1 0 0
x1 0 1 0 0 0 -1/2 1/6 0 1/2 -1/6 0 0 11/3
µ3 0 0 0 0 0 0 -2/3 1 0 2/3 -1 0 1/3λ1 0 0 0 0 1 0 -1/3 0 0 1/3 0 0 2/3
58
x2 0 0 1 2/3 0 1/6 -1/18 0 -1/6 1/18 0 1/3 25/9
This is the end of Phase I.
Thus the optimal solution is:
1 2 311 25, , 03 9
x x x= = =
Thus the optimal value z is: 193
9
![Page 59: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/59.jpg)
2221
2121 34236 xxxxxxz −−−+=Maximize
subject to 1 2 1x x+ ≤
Example-3
59
subject to 1 2
1 2
1 2
12 3 4
, 0
x xx x
x x
+ ≤
+ ≤
≥
![Page 60: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/60.jpg)
Denoting the Lagrange multipliers by λ1, λ2, µ1, and µ2, the KKT conditions are:
1 2 1 21. , , , 0λ λ µ µ ≥
λ λ µ− − − − + =
60
1 2 1 2 1
1 2 1 2 2
1 2 1 2 1
1 2 1 2 2
2. 6 4 4 2 03 4 6 3 0i.e. 4 4 2 6
4 6 3 3
x xx xx xx x
λ λ µ
λ λ µ
λ λ µ
λ λ µ
− − − − + =
− − − − + =
+ + + − =
+ + + − =
![Page 61: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/61.jpg)
1 2 1
1 2 2
1 1 2 2
1 1 2 2
3. 1,2 3 4,
00
x x Sx x SS Sx x
λ λ
µ µ
+ + =
+ + =
= =
= =
61
and all variables ≥ 0.
1 1 2 2
Solving this by " Modified Simplex Algorithm ", the optimal solution is:
x1 = 1, x2 = 0 and the optimal z = 4.
![Page 62: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/62.jpg)
Basic r x1 x2 λ1 λ2 µ1 µ2 R1 R2 S1 S2 Sol
r 1 0 0 0 0 0 0 -1 -1 0 0 0 8 10 2 5 -1 -1 0 0 0 0 9
R1 0 4 4 1 2 -1 0 1 0 0 0 6 R2 0 4 6 1 3 0 -1 0 1 0 0 3
S1 0 1 1 0 0 0 0 0 0 1 0 1
62
r 1 4/3 0 1/3 0 -1 2/3 0 -5/3 0 0 4
x2 0 2/3 1 1/6 1/2 0 -1/6 0 1/6 0 0 1/2 S1 0 1/3 0 -1/6 –1/2 0 1/6 0 -1/6 1 0 1/2
S2 0 0 0 -1/2 -3/2 0 1/2 0 -1/2 0 1 5/2
S2 0 2 3 0 0 0 0 0 0 0 1 4
R1 0 4/3 0 1/3 0 -1 2/3 1 -2/3 0 0 4
![Page 63: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/63.jpg)
Basic r x1 x2 λ1 λ2 µ1 µ2 R1 R2 S1 S2 Sol
r 1 0 -2 0 -1 -1 1 0 -2 0 0 3
R1 0 4 4 1 2 -1 0 1 0 0 0 6
x1 0 1 3/2 1/4 3/4 0 -1/4 0 1/4 0 0 3/4
S1 0 0 -1/2 -1/4–3/4 0 1/4 0 -1/4 1 0 1/4
63
r 1 0 0 1 2 -1 0 0 -1 -4 0 2
x1 0 1 1 0 0 0 0 0 0 1 0 1µ2 0 0 -2 -1 –3 0 1 0 -1 4 0 1S2 0 0 1 0 0 0 0 0 0 -2 1 2
S2 0 0 0 -1/2 -3/2 0 1/2 0 -1/2 0 1 5/2
R1 0 0 0 1 2 -1 0 1 0 -4 0 2
![Page 64: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/64.jpg)
Basic r x1 x2 λ1 λ2 µ1 µ2 R1 R2 S1 S2 Sol
r 1 0 0 0 0 0 0 -1 -1 0 0 0
λ1 0 0 0 1 2 -1 0 1 0 -4 0 2 x1 0 1 1 0 0 0 0 0 0 1 0 1
µ2 0 0 -2 0 –1 -1 1 1 -1 0 0 3
64
S2 0 0 1 0 0 0 0 0 0 -2 1 2
Thus the optimum solution is:
x1 = 1, x2 = 0, λ1 = 2, λ2 = 0, µ1 = 0, µ2 = 3
And the optimal value is: z = 4
![Page 65: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/65.jpg)
Solve the following quadratic programming problem
Maximize 2221
2121 518205020 xxxxxxz −+−+=
subject to 6≤+ xx
65
subject to
0,1846
21
21
21
≥
≤+
≤+
xxxxxx
Using Excel solver, the opt solution is: x1=2, x2=4Max z = 224
![Page 66: Non linear Programming Problems](https://reader034.vdocuments.net/reader034/viewer/2022042500/56d6be8a1a28ab301692924c/html5/thumbnails/66.jpg)
Remark:
If the problem is a minimization problem, say, Minimize z,
66
we convert it into a maximization problem, Maximize -z.