kuhn-tucker theory slides

8/9/2019 Kuhn-tucker Theory Slides

1/21

3/26/201

INTRODUCTION TO KUHN-TUCKER THEORY

OPTIMIZATION WITH INEQUALITYCONSTRAINTS

Max f(x)

s.t. gk(x) bk, k= 1,2, , m

Remarks:

If the objective is Minf(x), we may changeit to Max f(x).

If the constraint is gk(x) bk, we maychange it to gk(x) bk.


2/21

3/26/201

Graphical Solution

Max f x x x

s t x x x

x

x

x x

( )

. .

,

x

12

22

2

12

1 2

2

1

1 2

2 1

2 0

1 0

1 0

0

Example

Graphical Solution

f x x x

x x

( )

( )

x

12

22

2

12

22

2 1

1

Rewrite the objective function as

This is the equation of a circle of radius rwith center

at (0,1).

x x r12

22 21 ( )

Let the objective function be equal to r2, i.e.,


3/21

3/26/201

Level (Contour) Sets of the

Objective Function

0 0.2 0.4 0.6 0.8 1

0

0.2

0.4

0.6

0.8

1

Center:

(0,1) r

Constraint Curve x2= x12+ 2x1

0 0.2 0.4 0.6 0.8 1

0

0.2

0.4

0.6

0.8

1


4/21

3/26/201

The Feasible Set

0 0.2 0.4 0.6 0.8 1

0

0.2

0.4

0.6

0.8

1

Circle of radius r= 1

0 0.2 0.4 0.6 0.8 1

0

0.2

0.4

0.6

0.8

1

r= 1


5/21

3/26/201

Optimal Solution

0 0.2 0.4 0.6 0.8 1

0

0.2

0.4

0.6

0.8

1

r= 1

(x1*,x2*)= (1,1)

(x1**,x2**)

= (0,0)

Optimal Solution

There are two optimal solutions:

(x1*,x2*) = (1,1)

VOF(x1*,x2*) =x1*2+x2*

22x2* + 1 = 1

(x1**,x2**) = (0,0)

VOF(x1**,x2**) =x1**2

+x2**2

2x2** + 1 = 1


6/21

3/26/201

Example 2

Min x y

s t x y

x y

x y

( ) ( )

. .

,

5 6

2 6

2 8

0

2 2

( ) ( )x y r 5 62 2 2Let

Example 2: The Objective Function

( ) ( )x y r 5 6 2 2

This is the equation of a circle centered at (5,6) with radius r.

Thus, we wish to minimize r, i.e., we want to determine the

circle centered at (5,6) with the smallest radius that touchesthe feasible set.


7/21

3/26/201

Example 2: The Objective Function

0 3 5 8 x

4

6

y

Example 2: The Feasible Set

0 3 5 8 x

4

6

y

2x+y= 6

x+ 2y= 8


8/21

3/26/201

Example 2: The Optimal Solution

0 3 5 8 x

4

6

y

2x+y= 6

x+ 2y= 8

(x*,y*) (optimal solution)

Example 2. The Optimal Solution

The optimal solution is the intersection of the

two lines given by the equations

2x+y= 6

x+ 2y= 8

The optimal solution isx* = 4/3,y* = 10/3

VOF(x*,y*) = (4/35)2+ (10/36)2= 185/9


9/21


10/21

3/26/201

1

Slaters Constraint Qualification:

Example

2 6x y

x y 2 8

g x y x y1 6 2 0( , )

g x y x y2 8 2 0( , )

Consider the constraints:

Rewrite the constraints as:

Slaters Constraint Qualification:Example

Both functions are concave (because they are

linear) on R2+, the nonnegative quadrant.

Choose (x0,y0) = (1,1). Then

g1(x0,y0) = 62(1)1 = 3 > 0

g2(x0,y0) = 812(1) = 5 > 0 Hence, the constraints satisfy Slaters

Constraint Qualification.


11/21

3/26/201

Kuhn-Tucker Necessary Conditions

for Optimality

Given the problem

Max f(x), x Rn

s.t. gk(x) bk, k= 1,2, . . ., m

wheref,gk(k= 1,2, ..., m) have continuous partial

derivatives on an open convex subset D of Rnand

thegks satisfy Slater's constraint qualification.

If x* is an optimal solution, then there exist scalars

1*, 2*, ..., m* such that

Kuhn-Tucker Necessary Conditionsfor Optimality

( ) ( *) ( *) , , , . . . ,*a f g j nj kk

m

jk

x x

1

0 1 2

( ) ( *) , , , . . . ,*b g b k mk k k x 0 1 2

( ) , , , . . . ,*c k mk 0 1 2

( ) ( *) , , , . . . ,d g b k mk kx 0 1 2


12/21

3/26/201

1

Example 1

1. Min 4 + 2 4

s.t. 4

2. Max 2 2 + +

s.t. 1

3. Max 4 2 + +

s.t. + 0.25

Steps

1. Write in standard form.

2. Check if constraints are concave. If

yes,

3. Check if Slaters constraint qualification

is met. If yes,4. Findxs that satisfy Kuhn-Tucker

Necessary Conditions


13/21


14/21

3/26/201

1

The Expenditure Minimization

Problem

Min E(x, y) =p1x+p2y

s.t. U(x,y) u

x, y 0

The optimal solution is called an expenditure-minimizer.

Transform to a max problem:

Max E(x, y) = p1xp2y

s.t. U(x,y) u 0 x, y 0

Kuhn-Tucker Conditions whenNonnegativity Constraints are Explicit

Given the problem

Max f(x), x Rn

s.t. gk(x) bk, k= 1,2, . . ., m

x 0

wheref,gk

(k= 1,2, ..., m) have continuous partialderivatives on an open convex subset D of Rnand

thegks satisfy Slater's constraint qualification.

If x* is an optimal solution, then there exist scalars

1*, 2*, ..., m* such that


15/21

3/26/201

1

Kuhn-Tucker Conditions when

Nonnegativity Constraints are Explicit

( ) ( *) ( *) ,*a f gj kk

m

jk

x x

1

0

( ) ( *) ( *)* *b f g xj k jk

k

m

jx x

1

0

( ) ( *)*c g bk k k x 0

( ) *d k 0

( ) ( *)e g bk

kx 0( ) *f x 0

Example

Maxf(x,y) =x+ 2y

s.t. g(x,y) = x2y+ 2 0

x,y 0

It is clear thatfandghave continuous partial

derivatives.

We have already shown thatgis concave

and satisfies Slaters ConstraintQualification.


16/21

3/26/201

1

Kuhn-Tucker Conditions when

Nonnegativity Constraints are Explicit

(a) 1 2x 0 2 0 (b) (1 2x)x= 0 (2)y= 0 (c) (x2y+ 2) = 0 (d) 0 (e) x2y+ 2 0 (f ) x,y 0

Case 1. = 0 This is not possible from

(a).

Case 2. > 0 x= 0; not possible from

(a); hence,x> 0.

From (c): x2y+ 2 = 0 y= 0 => x2+ 2 = 0 => x= 2 => 1 22 = 0

=> = 1/(22), contrary to 2 from (a); hence,y> 0.

Kuhn-Tucker Conditions whenNonnegativity Constraints are Explicit

(a) 1 2x 0 2 0 (b) (1 2x)x= 0 (2)y= 0 (c) (x2y+ 2) = 0 (d) 0

(e) x2y+ 2 0 (f ) x,y 0

From (b),

(2) = 0 => * = 2 =>12(2)x= 0; => x* = ;

Solving foryin (c):

y* = 31/16

It can be shown that

x* = ,y* = 31/16

is an optimal solution byshowing that the objectivefunction is concave (Kuhn-Tucker SufficiencyCondition).


17/21

3/26/201

1

Example

1. Min 3 + 4

s.t. + 15

, 0

Kuhn-Tucker Sufficient Condition forOptimality: Example

Min x y( ) ( ) 2 32 2

s t x y. . 3 2 6

x y, 0

Transform the problem to conform to the standard form.

Max f x y x y( , ) ( ) ( ) 2 32 2

s t g x y x y. . ( , ) 6 3 2 0

x y, 0


18/21

3/26/201

1

Kuhn-Tucker Sufficient Condition for

Optimality: Example

Max f x y x y( , ) ( ) ( ) 2 32 2

s t g x y x y. . ( , ) 6 3 2 0

x y, 0

Note: Constraintgis concave since it is linear.

Choosex0= 0,y0= 0.

g(x0,y0) =g(0,0) = 6 > 0.

Hence, Slaters Constraint Qualification is satisfied.


Kuhn-Tucker conditions

(a) 2(x2)3 0 2(y3)20 (b) [2(x2)3]x= 0 [2(y3)2]y= 0

(c) (63x2y) = 0 (d) 0 (e) 63x2y 0 (f ) x,y 0

Case 1. = 0 From (a):

2(x2) 0 => x 2 2(y3) 0 => y 3 From (b):

2(x2) = 0 => x= 2 2(y3) = 0 => y= 3

From (e): 63x2y= 63(2)2(3)

=6 Hence,x= 2,y= 3; this

violates (e).

Therefore, = 0 is notpossible.


19/21

3/26/201

1

Kuhn-Tucker Sufficient Condition for

Optimality: Example

Kuhn-Tucker conditions

(a) 2(x2)3 0 2(y3)20 (b) [2(x2)3]x= 0 [2(y3)2]y= 0 (c) (63x2y) = 0 (d) 0 (e) 63x2y 0 (f ) x,y 0

Case 2. > 0 From (c):

63x2y= 0 x= 0 => y= 3

=> = 0 (from (b))contrary to > 0.

Hence,x> 0.

y= 0 => x= 2

=> = 0 (from (b))contrary to > 0.

Hence,y> 0.

From (b) & (c):

2(x2)3= 0 2(y3)2= 0 63x2y= 0


x y* , * , * 8

13

27

13

12

13

H( , )x y

2 0

0 2

The Hessian matrix of the objective functionfis:

His a negative definite matrix; hencefis strictly concave.

By the Kuhn-Tucker sufficiency theorem, (x*,y*) is an

optimal solution.

Solution:


20/21

3/26/201

2

Example

Min ( 5) + ( 4)

s.t. 2 + 3 10

, 0

Example

Min 42 + 5 3

s.t. + 2 20

, 0


21/21

3/26/201

Example

Max 52 4 + 10 +

s.t. 3 2 5

, 0

The Envelope Theorem forConstrained Optimization

dzd

f g dx

df g

dy

df gx x y y

**

**

**

dz

df g

**

Use the first-order conditions on (4):

(4)

Note that the right hand side of equation (5) is the derivative

of the Lagrangean with respect to , i.e.,

(5)

dz

dL x y

*( *, *, *, )

Add (3) to the right hand side of (1):

kuhn-tucker theory slides

Documents