Chapter 4The Maximum Principle: General Inequality Constraints

4.1 Pure State Variable Inequality Constraints: Indirect MethodIt is common to require state variable to remainnonnegative, i.e., i.e., xi(t) 0, i=1,2,,n. Constraints exhibiting (4.1) arecalled pure state variable inequality constraints. The general form is

With (4.1): at any point where a component xi(t)>0, thecorresponding constraint xi(t) 0 is not binding and canbe ignored.

In any interval where xi(t)=0,we must have so that xi does not become negative.The control must be constrained to satisfymaking fi 0, as a constraint of the mixed type (3.3) overthe interval. We can add the constraint

We associate multipliers i with (4.3) whenever (4.3) must be imposed, i.e., whenever xi(t)=0. A convenient way to do this is to impose an either or conditioni xi=0. This will make i=0 whenever xi >0.

We apply the maximum principle in (3.11) with additional necessary conditions satisfied by

and the modified transversality condition

where is a constant vector satisfying We can now form the Lagrangian

where H is as defined in (3.7) and =(1, 2 ,,n ).

Since the constraints are adjoined indirectly (in this case via their first time derivative) to form the Lagrangian, the method is called the indirect adjoining approach. If on the other hand the Lagrangian L is formed by adjoining directly the constraints (4.1), i.e.,

where is a multiplier associated with (4.1), then the method is referred to as the direct adjoining approach.

Remark 4.1 The first two conditions in (4.5) are complementary slackness conditions on the multiplier. The last condition is difficult to motivate. The direct maximum principle multiplier is related to as The complementary slackness conditions forthe direct multiplier are 0 and x*=0. Since 0, it follows thatExample 4.1 Consider the problem:

Solution. The Hamiltonian is

which implies the optimal control to be When x=0, we impose , in order to insure that (4.10) holds. Therefore, the optimal control on thestate constraint boundary is

Now we form the Lagrangian

where 1,2, and satisfy the complementary slackness conditions

Furthermore, the optimal trajectory must satisfy

From the Lagrangian we also get

Let us first try (2)= =0. Then the solution for is thesame as in Example 2.2, namely,

Since (t)-1 on [0,1] and x(0)=1>0, the original optimal control given by (4.11) is u*(t)=-1. Substituting this into (4.8) we get x(t)=1-t, which is positive for t0 for 0t

Figure 4.1: State and Adjoint Trajectories in Example 4.1

With this control, we can solve (4.8) beginning withx(1)=0, and obtain x(t)=0 for 1t2. Since (t) 0 in thesame interval, we see that u*(t)=0 satisfies (4.12)throughout this interval.To complete the solution, wecalculate the Lagrange multipliers.since u*=0, we have1=2=0 throughout 1t2. Then from (4.16) we obtain=-=2-t0 which, with x=0, satisfies (4.15). Thiscompletes the solution.

Remark 4.2 In instances where the initial state or the final state or both are on the constraint boundary, themaximum principle may degenerate I.e., there is no nontrivial solution of the necessary conditions,i.e., (t)0, t[0,T], where T is the terminal time.

4.1.1 Jump ConditionsThere may be a piecewise continuous (t) satisfying a jump condition

at a time at which the state trajectory hits its boundary value zero.Example 4.2 Consider Example 4.1 with T=3 and the terminal state constraint Clearly, the optimal control u* will be the one that keeps x as small as possible, subject to the state constraint (4.1) and the boundary condition x(0)=x(3)=1. Thus,

We only compute the adjoint function and multipliers that satisfy the optimality conditions. These are

4.2 A Maximum Principle: Indirect MethodMixed constraints as in Chapter 3 and the pure stateinequality constraints

where h: En x E1 Ep. By the definition of function h,(4.26) represents a set of p constraints hi(x,t)0,i=1,2,,p.It is noted that the constraint hi0 is calleda constraint of rth order if the rth time derivative ofhi is the first time a term in control u appears in the expression by putting f(x,u,t) for after each differentiation.

In case of first order constraints, h1(x,u,t) is as follows:

With respect to the ith constraint hi(x,t)0, a subInterval (1,2) [0,T] with 1< 2 is called an interiorinterval if hi(x(t),t)>0 for all t(1,2). If the optimaltrajectory satisfies hi(x(t),t)=0 for forsome i, then is called a boundary interval. An instant is called an entry time if there is an interior interval ending at t= and a boundary intervalstarting at . Correspondingly, is called an exit time if a boundary interval ends and an interior intervalstarts at .

If the trajectory just touches the boundary at time ,i.e., and if the trajectory is inthe interior just before and just after ,then is calleda contact time. Entry, exit and contact times are called junction times.

Entry11ExitContact1

Full rank condition on any boundary interval isas follows:

where for

and

To formulate the maximum principle for the problemwith mixed constraints as well as first-order pure stateconstraints, we form the Lagrangian as

H is defined in (3.7), u satisfies the complementary slackness conditions stated in (3.9), and Eq satisfiesthe conditions

The maximum principle sates that the necessary conditions for u* to be an optimal control are that thereexists multipliers ,,,,,and , and the jump parameters , which satisfy (4.29) that follows.

Note that the jump conditions on the adjoint variables in (4.29) generalize the jump condition on H in (4.29)requires that the Hamiltonian should be continuous at if ht = 0.

Example 4.3 Consider the following problem with the discount rate 0:

Solution. As one can see from Figure 4.2 and 4.3, the optimal solution is:Figure 4.2: Feasible State Space and Optimal State Trajectoryfor Example 4.3

Figure 4.3: Adjoint Trajectory for Example 4.3

Note that at the entry time t=1 to the state constraint (4.34), the control u* and, therefore, h1*=u*+2(t-2) is discontinuous, i.e., the entry is non-tangential. on the other hand, u* and h1* are continuous at t=2 so that the exit is tangential. With u* and x* thus obtained, we must obtain , 1 , 2, , and so that the necessary optimality conditions (4.29) holds, i.e.,

From (4.41), we obtain (1-)=e- . This with (4.38) gives:

and

which, along with u* and x*, satisfy (4.29).Note, furthermore, that is continuous at the exittime t=2. At the entry time so that (4.30) also holds.

4.3 Current-Value Maximum Principle: Indirect methodWith the Hamiltonian H as defined in (3.33), we can write The Lagrangian

We can now state the current-value form of the maximum principle as given in (4.42)

4.4 Sufficiency ConditionsThe sufficiency results can be stated in the indirect adjoining framework. In order to do so, let us define theHamiltonian H and the Lagrangian Ld in the direct method as

where d, d, and d are multipliers in the direct formulation, corresponding to , , and in the directformulation.

Theorem 4.1 Let satisfy the necessaryconditions in (4.29) and let If is concave in (x,u) at each t [0,T], S in(3.2) is concave in x,g in (3.3) is quasiconcave in (x,u)h in (4.26) and a in (3.4) are quasiconcave in x, and bin (3.5) is linear in x, then (x*,u*) is optimal.It can be shown that:

Theorem 4.2 Theorem 4.1 remains valid if the concavity of in (x,u) at each t is replaced by the concavity of the maximized HamiltonianTheorem 4.1 and Theorem 4.2 are written for finite horizon problems and remains valid if the transversality conditions on the adjoint variables (4.29) is replaced by the following limiting transversality conditionin x, where

Example 4.1 (Continued) First we obtain the directadjoint variable

It is easy to see that

is linear and hence concave in (x,u) at each t[0,2].

Functionsand

are linear and hence quasiconcave in (x,u) and x, respectively. Functions S 0, a 0 and b 0 satisfy The conditions of Theorem 4.1 trivially.