07.math.appendixy
x
x and y axes and the budget constraint
is the feasible set. Choose the highest
iso-utility curve (indifference curve)
holds with equality.
linear. The constraints do not have to
hold with equality. That is, some of the
constraints are not-binding (e.g., the
line going north-east). The solution will
be at one of the points of intersection.
y
x
if the point of highest utility is interior
to the feasible set. This would be the
case in the drawing to the left if the
feasible set were expanded to the right
so that it covered the innermost circle.
141
Closed Convex Sets Almost all optimization problems make extensive use of convex sets. This statement holds true
for maximization via calculus and linear programming as well as non-linear optimization
problems. Today I will show that closed convex sets and hyperplanes are the bases behind the
optimization techniques that we will cover. These concepts are meant to give insight into the
more mechanical methods that we employ and to show that we are fundamentally using the
identical method in all that we have covered in previous courses. The proofs regarding convex
sets are thus not meant for memorization but to provide basic understanding.
We first start off with a number of definitions: Convex Set: Geometric definition--a set is convex if and only if a (straight) line connecting any two points in
the set is also in the set.
Convex sets:
Non Convex sets:
Convex Set: Algebraic definition--Let x,y be two vectors in N space in the set S. E.g.,
x = (x1, x2) y = (y1, y2) . S is convex if and only if Px + [1−P] y ! S for all P such that 0 ≤ P ≤ 1.
142
2
x = x , x 1 2
Note that a convex set is not the same thing as a convex function.
Boundary point: Intuitively boundary points are points, which are on the edge of a set. More
formally, a point is a boundary point if every neighborhood (ball) around the point contains
points not in the set and in the set. They may be members of the set; e.g., in the set 0 ≤ x ≤ 1, the
numbers 0 and 1 are boundary points and members of the set. In the set 0 < x < 1, the numbers 0
and 1 are boundary points and not members of the set.
Closed set: a closed set is a set that includes all its boundary points. The following is a closed
set: 0 ≤ x ≤ 1. In economics we almost always consider closed convex sets. The reason can be
illustrated by considering the opposite. Maximize the amount of gold (G) you get if G < 1.
There is no maximum!
Preference set: The set of all points, which are indifferent or preferred to a given point.
Economics majors are acquainted with indifference curves. A preference set is the set of all
points on the indifference curve plus the all of the points strictly preferred to the indifference
curve. In the typical drawing of an indifference curve the preference set would include the
indifference curve plus all of the points upwards and to the right of the indifference curve.
We can view the ordinary two-dimensional graph of indifference curves as a two-dimensional
topographical map showing height in terms of contour lines. Here height is utility and the
contour lines are indifference curves. Consider the following diagram. There is a mountain
143
(representing utility) sticking out from the paper, the higher the mountain the higher the utility.
Assume that it is an ice cream cone cut in half lengthwise, filled with ice cream and put upside
down on the paper. The cone part is the indifference curve formed by a horizontal slice parallel
to the paper; the cone plus the ice cream represents all points such that utility is either equal to or
greater. We assume the set of such points is convex.
y
x
Indifference curves
Theorem 1. Sufficient conditions for a local optimum to be a global optimum. If the feasible set F is a closed and convex set and if the objective function is a continuous
function on that set and the set of points (such that the function is indifferent or preferred) create a convex set, then the local optimum is a global optimum.
This is a very important result. When these conditions hold, we know that that myopic
optimization will end up at a global optimum. If either the feasible set or the preference set is not
convex, then the local optimum need not be a global optimum.
Feasible set is not convex. 2nd peak is a local optimal but not a global optimum
(See following page)
144
145
Preference set is not convex. Right-most tangent is not a global optimal.
146
We next turn our attention toward hyperplanes. Hyperplane: a line in N-space dividing the space in two.
In two-space a hyperplane is a line. E.g., y = a + bx or a = y - bx or in more general
notation, A = c1x1 + c2x2. In three-space we have a plane: y = A + BX + CZ, or in more general
notation, …a = c1x1 + c2x2 + c3x3. In four or more space we have a hyperplane. Note that y ≥ a
+ Bx or a ≥ c1x1 + c2x2 creates a convex halfspace.
HYPERPANES CREATE CONVEX HALFSPACES
x 1
Supporting hyperplane has one or more points in common with a (closed) convex set but no
interior points. The set lies to one side of the hyperplane.
SUPPORTING HYPERPLANES
The following diagrams illustrate other cases of supporting hyperplanes.
Theorem 2. Given z a boundary point of a closed convex set, there is at least one
supporting hyperplane at z. Theorem 3: If 2 convex sets intersect but without interior points, there is a supporting
hyperplane that separates them.
Neoclassical optimization:
hyperplane separating the two convex sets
Neoclassical optimization (usually only consider edge) Max U (x, y) + (g(x) - 5) ¬
In the following figure, the shaded area is the feasible set and the curved line is the indifference
curve, which characterizes the preference set. The straight line is the hyperplane. We usually
give it a less technical term – the budget set or price line. Given these prices, the point of
intersection maximizes utility.
149
There are other results that are of use. The following two theorems state that the intersection of
closed convex sets is closed and convex.
Theorem 4. The intersection of two convex sets is convex.
Proof: If x ! (S ∩ T) and y ! (S ∩ T), then Z = Px + [1 - P] y ! S because S convex Z = Px + [1 - P] y ! T because T convex Therefore Z ! (S ∩ T)
Theorem 5. The intersection of two closed sets is closed.
150
Donald Wittman APPENDIX A2 Concave and Quasiconcave Functions A. CONCAVE FUNCTIONS Concave functions
x
f(x)
Three definitions of concavity: (1) Concave Function: The line connecting any two points of the function lies on the function or
below.
(2) Concave Function: Algebraic definition: Pf x( ) + 1 ! P[ ] f y( ) " f Px + 1 ! P[ ]y( ). Note that x
and y may be vectors in n-space
(3) Concave Function: Tangency definition: The tangent is always outside or on the function. The sum of two concave functions is also concave.
151
In order to determine whether a function is concave, we generally would prefer a method that did
not rely on drawing a picture of the function. Hence, we have the second derivative test.
If ! ! f x( ) = d2 f dx 2
" 0 , for all x, then the function is concave. This is a necessary and sufficient
condition for concavity.
Example A: f(x) = -x4 is concave since f''(x) = -12x2! 0 for all x.
Example B: f(x) = - x3 is not concave since f''(x) = - 6x and - 6x > 0 for x < 0.
Example C: x ≥ 0 and f(x) = − x3 is concave since f''(x) = − 6x ≤ 0 for all x ≥ 0.
Notice that the test for concavity has a great similarity to the test for a maximum. To test for a
maximum, you find out if the second derivative is less than zero when the first derivative equals
zero. To speak imprecisely, the test for a maximum discovers whether the function is locally
concave. In contrast, the test for concavity requires the second derivative to be non-positive
everywhere.
! f x( ) =12 " 6x = 0 x = 2
! ! f x( ) = "6 < 0 Therefore at x = 2, f(x) is a maximum and not a minimum.
tangent
tangent above function (locally) where f '(x) = 0. Going down hill.
The analogous test for concavity: ! ! f x( ) = "6 < 0 . Therefore f is concave.
152
An Important Property of Concavity: If f(x) is concave, then the set of x such that f(x) ≥ k is convex for all k.
f(x)
153
B. MULTI-DIMENSIONAL CONCAVE FUNCTIONS Next, we would like to expand our intuition by visualizing concave functions and convex sets in
two dimensions. If f x1, x2( ) = z , then f x1, x2( ) is concave if any vertical plane looks like the
single dimension picture (see the slice in the right hand side of the following picture).
f(x ,x )1 2
slice
What happens if we take a horizontal slice? We then we have an iso-function. If f is utility, iso-
utility is an indifference curve. Looking at x1, x2 :
x 2
x 1
from the figure to the above and left.
Notice that the set of x1, x2 such that
f x1, x2( ) ! k is a solid convex space.
If f is concave, then the set x1, x2 such that f x1, x2( ) ! k is convex for all k. The reverse relationship need not be true (a convex set does not imply a concave function). Hessian of 2nd order derivatives test—sufficient conditions. Let f i be the partial of f with respect to the partial of i. f11 f12 f21 f 22
= H for all x1, x2 H1 < 0 H2 > 0
! f concave where H1 = f1 1 and H2 = H
Note well this is only a sufficient condition for concavity. The necessary and sufficient
conditions have loose inequalities and consider permutations (this will be covered later).
154
Convex Function: The line connecting any
two points on a convex function does not lie
below the function.
f(x)
x
Convex Function, tangency definition: the function never lies below any tangent to the function. Convex function (a third definition): A function f is convex if and only if -f is concave. Hessian test: ! ! f x( ) " 0 for all x is a necessary and sufficient condition for f being convex. Note that the set of all x such that
f x( ) ! K is convex.
f(x) k
Students often get mixed up between concave and convex functions and convex sets. Think of a
concave function as creating a cave. This will help distinguish a concave function from a convex
function. While there is a convex set there is no such thing as a concave set. The set of x such
that f(x) ≥ k is convex when f(x) is a concave function; the set of x such that f(x) ≤ k is convex
when f(x) is a convex function.
155
D. QUASICONCAVITY Intuitive concept for one dimension: the function has only one peak. Quasiconcavity: Algebraic definition:
f px + 1! P[ ]y( ) " f x( ) and/or f (y)
f(x)
f x( ) ! k
f(x)
Any horizontal cut creates a convex set of indifferent or higher points. Another lecture covers the hessian test for quasiconcavity.
156
Note well that the sum of two quasiconcave functions need not be quasi-concave. E.g., try x3
and -x2. For x ≥ 0 both are quasiconcave.
x3 -x2
Adding them together:
x 3 ! x 2 = 0 at x= 0 0 − 0 = 0 at x=1/2 1/8 −1/4 = −1/8 at x =1 1 − 1 = 0
A quasiconcave function has to have one peak no matter what way you slice it. These two boxes below look like there is only one peak, but if you go from A to B you will drop
down to C.
157
Donald Wittman APPENDIX A3 Semidefinite determinants Students are acquainted with second order conditions. In the single variable case, if the first
order conditions are equal to zero (f'(x) = 0), then f'' < 0 is sufficient for a local maximum.
These results can be generalized to the n-variable case by looking at the Hessian matrix of
second derivatives when the firsts order conditions are satisfied. Now this approach to
maximization is very limited. It only tests for local maxima; there may be a much higher peak
elsewhere. Furthermore, it provides only a sufficient condition. For example, -x4 is a maximum
at x = 0; yet its second derivative is −12x2 = 0 at x = 0. Therefore the sufficiency test is not
applicable.
In programming we are interested in global optima. We do not look at just a point (where f' = 0),
but the function as a whole. We test whether f is a concave function. It is if f'' ≤ 0 for all relevant
values of x (say for x ≥ 0). If there is more than one variable, we then deal with the Hessian of
second derivatives (this will be explained later). Thus global optimization in non-linear
programming uses related ideas to the local optimization analysis (which in crude but incorrect
terms finds local concavity) typically found in courses in intermediate economics, but there are
three important differences: (1) Conditions on the second derivative of f apply to all values of x
not just where the first order conditions are satisfied. (2) Inequalities need not be strict (zero
values are allowed). (3) Permutations of the Hessian of second order partials are considered.
We will start with strict concavity and strict convexity, which are easier to handle. We know
from the single variable case that if f'' < 0, then f(x) is a strictly concave function. The two-
variable test is a bit more complicated as one has to make sure that the function is concave not
only in one direction but in any direction. To start with, suppose that f is a function of three
variables, f(x1, x2, x3). Letting fi stand for the partial of f with respect to xi, we have the following
matrix of second derivatives:
!
"
# # #
$
%
& & & .
Note that the matrix is symmetric as fij = fji by Young’s theorem. We will now look at the
determinants (denoted by ||) formed from this matrix (denoted by []).
If the following holds, then f is strictly concave:
f11 < 0, f11 f12
f21 f22 f23
f31 f32 f33
f11 f21 f31
f12 f22 f32
= f11 f22 f33 + f12 f23 f31 + f13 f21 f32 ! f31 f22 f13 ! f32 f23 f11 ! f33 f21 f12
If the determinants derived from the matrix of second derivatives of f are negative definite for all relevant values of x, then f is strictly concave. This holds for any number of variables,
not just 3.
Why not only if? f(x) = –x4 is strictly concave, but its second derivative = 0 at x = 0.
The matrix is known as being positive definite if the determinants of the successive principal
minors are all positive for all relevant values of x.
159
If the determinants derived from the matrix of second derivatives of f are positive definite
for all relevant values of x, then f is strictly convex.
Things get a bit more complicated when the above inequalities are no longer strict.
Let
H k be a k-by-k determinant formed by any permutation from the set of N variables along the
diagonal of H, an N-by-N matrix. Note that the bar is to remind us that we are permuting.
H =
!
"
# # # #
$
%
& & & &
H 2 = f11 f12
f12 f11
= f11 f22 - f21 f12
As can be seen, the second expression is redundant. This is for two reasons: 1) the second
determinant is equivalent to the first because we have interchanged a row and then a column,
which does not alter the value of the determinant. 2) In the second place, we will only be
considering second partials so that the matrices are symmetric. Therefore this alteration will
have no effect.
!
"
# # #
$
%
& & &
H 2 = f11 f12
f23 f22
Note that the second determinant is redundant of the first, the fourth redundant of the third and
the sixth redundant of the fifth.
Finally,
H 3 = |H|.
Note that the diagonals in the sub-determinants are always fii from the original diagonal and that
the off diagonal elements are matched to the diagonal. The same holds true for a bordered
hessian, which we will get to later (the borders remain borders, diagonals remain diagonals and
other elements match).
are inappropriate.
The matrix is negative semidefinite if for all relevant values of x, all of the permutations of the
determinants of the successive principal minors alternate in sign starting with a minus. That is, if
for all the permutations,
H 1 ! 0;H 2 " 0;H 3 ! 0, then H is a negative semidefinite matrix.
A matrix of second derivatives of f is negative semidefinite if and only if f is concave. If for all the permutations
H 1 ! 0;H 2 ! 0;H 3 ! 0, then H is a positive semidefinite matrix.
161
A matrix of second derivatives of f is positive semidefinite if and only if f is convex. Just for fun (if you call this fun), I have written this out for the 4-by-4 case, where aij = fij.
If H = a11 a12
a21 a22 a13 a14
H 2 = a11 a12
H3 = a11 a12 a13 a21 a22 a23 a31 a32 a33
and a11 a12 a14 a21 a22 a24 a41 a42 a44
and a11 a13 a14 a31 a33 a34 a41 a43 a44
and a22 a23 a24 a32 a33 a34 a42 a43 a44
H 4 =|H | A matrix is positive semidefinite if for all values the derived determinants H1 ! 0, H2 ! 0, H3 ! 0 A matrix is negative semidefinite if for all values the derived determinants H1 ! 0, H2 " 0, H3 ! 0
162
We next consider the tests for quasiconcavity and quasiconvexity. Again, we will use the
concepts of negative (or positive) definite series of determinants and negative (or positive)
semidefinite series of determinants, but this time the determinants will be bordered with the first
derivatives and therefore the signs get reversed.
It is easy to get confused about signs. Here is the way that I remember. For concavity, we have a
negative semidefinite expansion 1!1" 0;2 ! 2 # 0;3! 3" 0; etc. In testing for quasiconcavity, we
reverse signs because we have a border: hence, 1!1" 0;2 ! 2 # 0;3! 3" 0; etc. As we will see,
with a border, the following will always be true1!1= 0;2 ! 2 " 0. So, we will only have to pay
attention starting with the 3! 3determinant. For convexity, we have a positive semidefinite
expansion 1!1" 0;2 ! 2 " 0;3! 3" 0; etc. In testing for quasiconvexity, we again reverse signs
because we have a border: 1!1" 0;2 ! 2 " 0;3! 3" 0; etc. Again it is true that with a border, the
following will always be true1!1= 0;2 ! 2 " 0. So, we will only have to pay attention starting
with the 3! 3determinant.
We start with the two-variable case. Notice right away that the border is composed of first
derivatives. Notice also that if we consider successive determinants, the first determinant will
always equal 0 and the second will always be negative or 0. So we start with the third. If the 3x3
determinant is strictly positive (and the 4x4 determinant is strictly negative when we have 3
variables, and so forth), we have a negative definite symmetric matrix.
0 f1 f2 f1 f11 f12 f2 f21 f22
!
"
# # #
$
%
& & &
If the bordered hessian of f is negative definite (starting with a positive for the 1x1), then f is strictly quasiconcave.
If the bordered hessian of −f is negative definite (starting with a positive for the 1x1), then f
is strictly quasiconvex.
163
Note that employing the minus sign in front of f, means the value of the determinant is opposite
of the value of the determinant for f when there are an odd number of columns and the same
value of the determinant as for f when there are an even number of columns. Hence if the
expansion is always negative, then f is strictly quasiconvex.
When the value of one or more determinants is 0, we have to go back to permutations and semi-
definiteness, with loose inequalities.
The bordered hessian of f is negative semidefinite (starting with a positive for the 1x1), if and only if f is quasiconcave.
The bordered hessian of −f is negative semidefinite (starting with a positive for the 1x1), if
and only if f is quasiconvex.
Example 1. f(x,y) = x + y2 where x, y ≥ 0. fx = 1; fy = 2y
fxx = 0; fyy = 2; fxy = 0
H = 0 0 0 2
H1 = 0, 2 ! 0; H2 = 0 Hence, we have a positive semidefinite determinant and f is convex. This example illustrates
why we need to do permutations. If we did not do the second permutation, we could have said
that f is negative semidefinite, but this would be incorrect.
164
Example 2. f(x,y) = x2 + y2 where x, y ≥ 0. fx = 2x; fy = 2y
fxx = 2; fyy = 2; fxy = 0
H = 2 0 0 2
H1 = 2 > 0; H2 = 4 > 0 Therefore we have a positive definite matrix and a strictly convex function. We know that a convex function is quasiconvex, as well. But it is useful to demonstrate this by
looking at the bordered hessian (this is also a good way to check about the appropriate signing of
a bordered hessian if you forget). So let us look at the bordered hessian. We start with the 3x3
border as the 2x2 determinants are never positive when we have a bordered hessian and the 1x1
is always 0.
H = 0 f1 f2 f1 f11 f12 f2 f21 f22
= 0 2x 2y 2x 2 0 2y 0 2
0 2x 2y
2x 2 0
= !8y2 ! 8x2 " 0 . So it is quasiconvex.
I have actually given you a slightly different algorithm. I said that if you multiply f by minus 1,
and the –f function is quasiconcave, then f is convex. So let us multiply by –1.
H = 0 !2x !2y
0 !2x !2y
!2x !2 0
= 8y2 + 8x2 ! 0. So –f is quasiconcave and therefore f is quasiconvex.
165
Example 3. f(x,y) = x +2y ! 3xy where x, y ≥ 1 fx = 1 ! 3y; fy = 2 ! 3x
fxx = 0; fyy = 0; fxy = !3
H = 0 !3 !3 0
H1 = 0,H1 = 0; H2 = !9 < 0
Therefore f is neither concave nor convex. Note that we have two values for H1 as we have to
deal with each permutations, and that we have to deal with permutations because we have zeros
and semidefiniteness. This time we need to go to the bordered hessian because we could not
establish concavity or convexity.
H = 0 1 ! 3y 2 ! 3x
1 ! 3y 0 !3 2 ! 3x !3 0
0 1 ! 3y 2 ! 3x
1 ! 3y 0 !3
= !6(1 ! 3y)(2 ! 3x) < 0 because x, y ≥ 1. So, quasiconconvex. Note well that if
x, y ≥ 0 , f is neither quasiconcave nor quasiconvex because y < 1/3 and x > 2/3 would generate a
positive number while y > 1/3 and x > 2/3 would generate a negative number. Concavity and
convexity are global conditions not local.
Example 4: f(x,y) = x3y x ≥ 0, y ≥ 0 fx = 3x2y fxx = 6xy fy = x3 fxy = 3x2 fyy = 0
6xy 3x2
3x2 0
166
Therefore f is neither concave or convex. Let us see if the function is quasiconcave or quasiconvex by looking at the bordered hessian.
0 3x2y x3
3x2y 6xy 3x2
x3 3x2 0
0
3x2y
x3
3x2y
6xy
3x2 = 9x7y + 9x7y - 6x7y = 12x7y ≥ 0 for x, y ≥ 0 Therefore negative semidefinite and therefore quasiconcave. Note that when we have a bordered hessian, we do not pay attention to
H 1 as it is always 0 (the
border remains the same) and
H 2 as all 2x2 are negative (here too, remember that the border
remains with the 0 in the upper left-hand corner). Note that in the case we were saved from doing
permutations as we only had the 3x3 matrix to worry about.
Note that a function can be both concave and quasiconvex or both convex and quasiconcave. A
straight line is both concave and convex (as well as quasiconcave and quasiconvex).
Example 5
Suppose that f = −x2 − y2 − z2 and that x, y, and z are non-negative.
We know that −x2 is concave and that the same holds true for the other two variables. We also
know that the sum of concave functions is concave and that a concave function is quasiconcave.
Nevertheless, let us test directly whether f is quasiconcave as this exercise will help to cement
our understanding.
167
H =
=
0 !2x !2y !2z !2x !2 0 0 !2y 0 !2 0 !2z 0 0 !2
H2 = 0 fx fx fxx
and 0 fy fy fyy
and 0 fz fz fzz
It is always the case that these are ! 0.
H 3 =
= 0 !2x !2y
, 0 !2x !2z
, 0 !2z !2y !2z !2 0 !2y 0 !2
These equal 8x2 +8y2 ! 0; 8x2 +8z2 ! 0; and 8z2 +8y2 ! 0, respectively. Notice once again that if I switch 1 row and 1 column, the value of the determinant remains the
same. In the last case, the z and y rows and z and y columns were switched.
H 4 = H =
=
0 !2x !2y !2z !2x !2 0 0 !2y 0 !2 0 !2z 0 0 !2
Expanding by co-factors (by going down the last column) and repeating the first two rows for
easy multiplication, we get,
H = 2z !2x !2 0 !2y 0 !2 !2z 0 0
!2x !2y !2z
!2 0 0
0 !2x !2y
!2x !2 0
168
So, the bordered hessian is negative semidefinite and therefore the function is quasiconcave.
Note that because of the border, we have positive, negative, positive, negative starting with the
first 1 by 1 in contrast to the test for concavity where negative semidefinite means having
negative, positive, negative, positive without any border.
Example 6
Suppose that f = x2 + y2 + z2 and that x, y, and z are non-negative.
We know that x2 is convex and that the same holds true for the other two variables. We also
know that the sum of convex functions is convex and that a convex function is quasiconvex.
Nevertheless let us test directly whether f is quasiconvex as this exercise will further help to
cement our understanding.
H =
=
0 2x 2y 2z 2x 2 0 0 2y 0 2 0 2z 0 0 2
H2 = 0 fx fx fxx
and 0 fy fy fyy
and 0 fz fz fzz
It is always the case that these are ! 0.
H 3 =
0 fx fy fx fxx fxy fy fyx fyy
= 0 2x 2y 2x 2 0 2y 0 2
, 0 2x 2z 2x 2 0 2z 0 2
, 0 2z 2y 2z 2 0 2y 0 2
These equal −8x2 −8y2 ! 0,−8x2 −8z2 ! 0, and −8z2 −8y2 ! 0, respectively.
169
H 4 = H =
=
0 2x 2y 2z 2x 2 0 0 2y 0 2 0 2z 0 0 2
Expanding by co-factors (by going down the last column) and repeating the first two rows for
easy multiplication, we get,
H = !2z 2x 2 0 2y 0 2 2z 0 0
2x 2y 2z
2 0 0
+ 2 0 2x 2y 2x 2 0 2y 0 2
0 2x 2y
2x 2 0
= !16z2 !16y2 !16x2 " 0
Notice that that each expansion is now negative. We do not have a commonly used term for an
all-negative expansion. We use positive semidefinite for an all-positive expansion, and negative
semidefinite for an alternating negative and positive expansion (when there is no border) and an
alternating positive and negative expansion (when there is a border). That is why the easiest way
to remember all the rules is to test for convexity and quasiconvexity by multiplying the function
by minus one and testing for concavity or quasiconcavity.
HOMEWORK: Find out if concave, convex, quasiconcave, quasiconvex or none of the above for x, y ≥ 0. Your
homework will be completed faster if you remember that a sum of concave functions is concave
(but remember that a sum of quasiconcave functions need not be quasiconcave).
1) (x + y)2 + x 2) x3 + y1/2 3) x1/3 ! 3xy
170
4) x1/2 + y1/3 + 3(xy)1/2 5) 2 log(x+1) + (1/3)log(y+1), where log means natural log.
171
Nonlinear programming is a generalization of linear programming, unconstrained maximization
and constrained maximization.
Maximize ! = f x1, x2 ,x3 . . .xN( )
subject to
. . . . gM x1, x2, . . .xN( ) ! RM xi " 0
Max ! = x1 100 " x1 2[ ] " x2 300 " x2
3[ ] + x1x2 = f x1,x2( )
g2 x1, x2( ) = x1 2 + 3x2 ! 5
x1, x2 ! 0
Remember that g j x1, x2 . . .( ) is the jth function of the variables x1, x2 . . . we could have
g x1 . . .( ), h x . . .( ), k x . . .( ) but this would be notationally difficult.
If f and g j are linear functions then we have a linear program. If we have no constraints but f is
nonlinear then we have optimization using calculus. If we have the constraints holding with strict
equality, we have optimization with equality constraints.
172
2) Graphical Representation of Nonlinear programming.
Nonlinear programming does not have the graphical restrictions found in standard economic
undergraduate texts. This can be illustrated via several examples.
Minimize C = x1 ! 4( )2 + x2 ! 4( )2 subject to
2x1 + 3x2 ! 6
!3x1 ! 2x2 " !12
x1, x2 ! 0
1
2
3
4
5
6
0
2!!!!!!!!!!!!!!2
The feasible set is shaded in.
A) Optimal solution as seen here need not be located at an extreme point.
B) Only one constraint exactly fulfilled even though two constraints (including non-
negativity there are four constraints) and two unknowns.
173
Minimize C = x1 ! 4( )2 + x2 ! 4( )2 subject to
x1 + x2 ! 5
6 5 4 3 2 1
1 2 3 4 5 6
Again the feasible set is shaded in. In this example, the optimal solution does not lie on the
boundary and thus none of the constraints are exactly fulfilled. So can't narrow down to
boundary points. Here the constrained and unconstrained optimums are identical.
174
Our third example displays a problem—non convexity – that we will try to avoid. In this
case, we have a non-convex constraint space (we may also have a non-convex preference set
which would also cause problems) We have illustrated this before, but now we do so in a
different way..
2x1 + 3x2 ! 12
x1, x2 ! 0
6 5 4 3 2 1
1 2 3 4 5 6
The straight lines are the indifference curves. The further to the right, the higher the
indifference curve. We have a local maximum at 2, 3 but not a global maximum.
175
3) Kuhn Tucker Necessary Conditions
In calculus, the FOC or necessary conditions for an interior maximum were f i x1, x2( ) = 0
for all i. We now consider the necessary conditions for a maximum in nonlinear programming.
These are known as the Kuhn-Tucker necessary conditions. We start off with a very simple
problem—a maximization problem with non-negativity constraints. This will enable us to see
the logic behind the Kuhn-Tucker conditions.
Max f x( )
0 (C)
where f x( ) is a one variable differentiable function. Let ! f "
df dx
. If a local maximum occurs in an interior point (Figure A), the first order condition
is ! f x( ) = 0 . This is the same necessary condition as the free extremum. Even in the second
example (Figure B), where we have a local extremum at a boundary point, ! f x( ) = 0 . As a third
possibility, we may have a local maximum at a boundary even though ! f 0( ) < 0 .
Thus the necessary conditions for a maximum are:
A. ! f x( ) = 0 and x > 0 B. ! f x( ) = 0 and x = 0 C. ! f x( ) < 0 and x = 0
For (C) we are at the lower boundary and we cannot decrease x more and make f (x) larger.
These conditions can be stated more elegantly as follows:
! f x( ) ≤ 0 x ≥ 0 and x ! f x( ) = 0 non-negativity says at least one constraint must equal zero
176
For a minimum the necessary conditions are as follows: ! f x( ) ≥ 0 x ≥ 0 and x ! f x( ) = 0
More generally, when ! = f x1, x2 . . . xN( ) = f x( ) , x is a vector, and xi ! 0 , then the K-T
necessary conditions for a maximum are:
fi x( ) = !f !xi
Finally, we consider the most general case with inequality constraints:
Max ! = f x( ) = f x1, x2 , . . . xN( )
subject to
g2 x( ) ! R2
gM x( ) ! RM
This can be rewritten as a Lagrange multiplier
"Max" W x1, x2 , . . . xN ,!1,!2 , . . .!M ( ) = f x1, x2, . . . xN( ) + !1 R1 " g1( ) + !2 R2 " g2( ) . . . !M RM " gM( )
Max is in quotes. Since we are finding a saddle-point not a maximum.
Necessary condition—K-Tucker necessary conditions
!W !xi
• xi = 0.
!W !"i
• "i = 0. .
177
Note that the Lagrangian method has converted the constraints into first order conditions. Also
note that the KT complementary slack requirements !W
i!" # i" = 0 means that we have only
added zero's to the objective function.
A natural question is to ask how the non non-negativity constraints work. A simple example in
the homework should provide insight.
178
4. Objective Function Requirements A. f x1, x2, . . . xn( ) is either (i) concave or
(ii) quasiconcave and there exists no point x* = x1 * , . . . , xn
*( ) such that f i x *( ) = 0 for all i.
The rationale behind these assumptions: Say you have a concave objective function like the
following diagram:
f(x)
x
Then everything is OK. When you max you end up at a global max. The same holds true if
function looked like this.
But, if the objective function looks like the following,
then one could find what looks like a "max" at the inflection point using the 2nd derivative test.
So, to avoid this problem we must rule out complete local satiation.
179
4) Arrow Enthoven Constraint Qualifications
In order to avoid certain pathologies the following assumptions concerning the constraints must
be true. These are known as the Arrow Enthoven Constraint Qualifications:
a) Every constraint functiong j x( ) in the maximization problem is twice differentiable and
quasiconvex in (an open set containing) the non negative orthant.
b) There exists a point x0 in the non negative orthant such that all the constraints are non-
binding (although satisfied). That is, the feasible set has an interior point.
c) One of the following is true:
i. g j x( ) is convex, or
ii. There exists no point x* in the feasible set where gj i x*( ) = 0 for all i and j.
Discussion of points (a), (b) and (c):
(a) We know from earlier lectures that, for a maximization problem, quasiconvex constraints
create convex feasible sets (that is, if f(x) is quasiconvex, the set of all x such that f(x) ! k is
convex). From the opposite perspective, we have seen diagrams illustrating how non-convex
feasible sets may yield non-optimal solutions even when the indifference curve is tangent to the
feasible set. So requirement (a) is ensuring convex feasible sets. In a homework exercise, we will
engage in a mathematical example showing that the KT conditions are not sufficient for a
maximum when quasiconvexity of the constraints is violated. Fortunately, we already know how
to test for quasiconvexity.
(b) In a homework exercise we will see via a simple example the problems that arise when (b) is
violated.
180
(c) This is getting rid of constraint inflection points where the derivative is 0 (and is thus parallel
to getting rid of objective function inflection points). If i holds, that is, the function is convex,
then we do not have to worry about inflection points. If ii holds, then we do not have to worry
either even though we only have quasi-convexity because the condition rules out the possibility
of inflection points.
Example: Testing to see whether c is satisfied.
There is one constraint, LOG(x1 +1)! LOG(x2 +1) " 7 . Then g1 = 1/ (x1 +1) and g2 =
−1/(x1 +1); g11 = !1/ (x1 +1) 2;g12 = 0;g21 = 0;g22 = 1/ (x2 +1)
2. Looking at the hessian of second
derivatives, we can immediately see that the hessian !1/ (x1+1)
2 0
is not positive
definite. So, the constraint is not convex. Hence, c.i is not satisfied.
We next test for quasiconvexity.
0 1/ (x1+1) !1/ (x2+1)
1 / (x1+1) !1/ (x1+1) 2 0
!1/ (x2+1) 0 1 / (x2+1) 2
0 1 / (x1+1) !1/ (x2+1)
1 / (x1+1)
2 !1/ (x2+1) 2(x1+1)
2 = 0
So the constraint is quasiconvex (as well as being quasiconcave).
(c.ii) is satisfied as there are no set of values where both g1 and g2 = 0. (Indeed there are no non-
negative values of x1 and x2 where either g1 or g2 is equal to 0.) Hence the AECQ conditions are
satisfied.
181
Max ! = f x( ) x = x1, x2 ,x3, . . . xn
ST
xi ! 0 i =1, . . . , N
If the following is true then x* gives a global maximum.
(1) The objective function, f, is differentiable and quasiconcave in an open set containing the
non-negative orthant.
(2) Each constraint, g j , is differentiable and quasiconvex in an open set containing the non-
negative orthant.
(3) x* satisfies the Kuhn-Tucker necessary conditions.
(4) The set g j satisfies the AECQ conditions (this is a redundant because an element of (3) is
that the AECQ or another set of constraint qualification hold):
a) g j is either convex
or quasiconvex but there exists no point x* such that gj i x*( ) = 0 for all i, j,
b) There is an interior point to the constraint set.
(5) The second derivative of f exists and
a) f x( ) is concave
b) or quasiconcave and there exists no point x* such that f i x *( ) = 0 for all i (rules out
the local satiation).
182
A word about necessary and sufficient conditions: Sufficient means that if the conditions
hold you will have a maximum; necessary means that if the conditions do not hold, you will not
have a maximum (equivalently, only if the conditions hold is a maximum possible).
HOMEWORK
A. In the following examples, first establish (1) whether the constraint qualifications are met and
(2) whether the conditions on the objective function are met. Then find the KT necessary
conditions and solve for all values. If (1) or (2) are not satisfied show that the KT necessary
conditions can be met, yet the objective is not achieved or that it is impossible for the K-T
conditions to be met. In all cases, x ≥ 0. Also solve graphically.
Note that a monotonic function of one variable is quasiconcave and quasiconvex. In some of
these problems, certain conditions are violated. Therefore the K-T conditions are no longer
necessary and sufficient. Therefore it is possible for the KT conditions to be satisfied and yet the
point not be a maximum.
1. Max f x( ) = x subject to x ≤ 10 (Note that λ is the value of relaxing the constraint and
that it is at least as large as the marginal value of x.)
2. Max f x( ) = x2 subject to x3 ≤ 8
3. Max
f x( ) = x .2 subject to x3 ≤ 8 and x ≤ 10
4. Max f x( ) = x + y subject to x ! 2( )2 + y " 4 and x ! 2( )2 ! y " !4 .
In this case, the AECQ conditions cannot be satisfied.
183
HOMEWORK 2
NOTE: other homework assignments may be substituted for this and the following
homework assignments.
1. Check if objective function satisfies either:
A. Concavity or
3. Write out Lagriangian
5. Provide an economic interpretation of Kuhn-Tucker conditions. E.g. marginal
something equals marginal something else.
6. If possible determine which KT equations hold with equality
7. Perform requested comparative statics based on equalities in 6.
A. A Firm maximizes Profit = [A - BQ(K, L)]Q(K, L) -wL - Ki
Let Q(K, L) be the production function. Q has a hessian of second order derivitives that is
negative definite. A > 2BQ. A, B > 0. QL, QK > 0. QKL = 0.
Answer questions 1-5.
Also show that Profits are homogeneous of degree 0 in A, B i and w. Show that the Profit
function is convex.
Find the effect of an increase in A on K; the effect of an increase of w on K, the effect of an
increase in A on Profits; and the effect of an increase in i on profits.
184
B. Peak load Pricing C.E.G.
Competitive Firm P1, P2 > 0 Prices in the two periods given.
Q1 — supply of good in period 1 Q1 > 0 Q2 — supply of good in period 2 Q2 > 0 Assume that Q2 > Q1
variable cost = C = AQ12 + BQ1Q2 + DQ22A, B, D > 0
4AD > B2
i > 0 interest rate which is the cost of capital K ≥ 0
Max ∏ = P1Q1 + P2Q2 – AQ12 – BQ1Q2 – DQ22 – Ki Present value of price
Subject to
.01 Q1 ≤ K supply of good uses 1% of capacity
.01 Q2 ≤ K Find the effect of a change in P1 on Q2 Note that Q1 and Q2 are not functions of K and L, but primitives (variables). The firm is choosing Q1, Q2 and K subject to constraints.
C. A Shipping Problem
A monopolist ships from point A to B and back. We assume that the demand curve is
higher for going out than for coming back.
X0 = Number of paid trips out XB = Number of paid trips back
X0 ≥ XB Treat this as a constraint (that is, a ship cannot return if it does not go out).
We assume that X0 > XB and that AB > 2BXB
P0 = A0 – BX0 A0, AB , B ≥ 0 Pb= AB – BXB
185
HOMEWORK 3
(1) In order for WAIBPC to be a profit function, what restrictions are there on A and B and C.
W = Wage, I = Interest Rate, P = Price of Good.
(2) Suppose Q(K, L) = 1 3K
1 3L and the firm is competitive in input and ouput markets.
A) Derive Profit maximizing relations.
B) What are second order conditions?
C) Find the slope of the isoquant.
D) Show that ∏ is homogeneous of degree 1.
(3) A monopolist has the following demand curve:
P = A – BQ A, B > 0
CQ2 is the cost curve C > 0
A) Find FOC, SOC.
B) Find the effect of an increase in C on Q.
C) Find the effect of an increase in C on ∏.
D) Write out the profit function, ∏*, explicitly. Remember that a profit function is a
function of the exogenous variables only.
The cost of a round trip is C(Xo)2 where C is a constant.
Find the effect of an increase of AB on X0.
186
Starting with a Cobb Douglas production function Q = 1 3K
1 3L , find L*, K* and ∏*. Show that
∏* satisfies the properties of a profit function then derive the cost function.
This is a lot of mathematical manipulation, but you need to do it to see how everything fits
together.
187
HOMEWORK 5
1. Show that Q B,W + Bi i+2B3 (wi) 1/2[ ] is a cost function.
2. Starting with a Cobb-Douglas technology (with decreasing returns to scale).
a. Derive the first order conditions for profit maximization. (3)
b. Show that the second order conditions are satisfied. (4)
c. Determine the elasticity of substitution. (5)
3. Suppose that Q(K,L) is concave with a negative definite Hessian of second order partials.
a. Find the first order conditions for the minimizing cost subject to an output constraint. (3)
b. Find the effect of a change in W on cost. (3)
4. Again assume Q(K,L) has a negative defnite Hessian of second partials.
a. Find the first order conditions of a perfectly competitive profit maximizing firm (3) and
show the relationship between marginal products and factor prices when L,K>0. (1)
b. Show that profits are homogeneous of degree O in input and output prices. (3)
c. Show that if there are constant returns to scale, there are zero profits. (4)
d. Find the effect of a change in i on profits. (3)
5. Suppose that Q(K,L) = K+L.
a. Derive the isoquant and draw it. (3)
b. Derive the minimal cost function (4) and derive, explain, and draw the isocost curve. (4)