optimalctrl sp

8/9/2019 OptimalCtrl SP

1/6

Optimal control of an oscillator system

J.C.C. Henriques

2015/02/06

1 Optimal control

1.1 Problem statement

Let x(t ) the state of the system with input u(t ) that satisfies the state equation [1]

ẋ = F (x, u) , (1)

with

x (0) = x0 (2)

u∈ (3)

t ∈ [0, T ] (4)

with T fixed. The set shows the constraints applied to u.

The goal is to determine u, defined in [0, T ] which maximizes the cost functional J defined as

J (u) = Ψ (xT ) +

T 0

(u, x) dt (5)

where

xT = x (T ) . (6)

Ψ (xT ) is the cost associated with the terminal state xT . The Lagrangian (u, x) is the function to

be maximized in [0, T ].

1.2 Pontriagyn maximum principle

Along optimal path for (x, u,λ(t )) it is verified the following necessary conditions for maxi-

mizing J

ẋ = F (x, u) ,

with

x (0) = x0

u∈


2/6

t ∈ [0, T ] (7)

and

− λ̇ = λT ∇F (x, u) +∇ (x, u) (8)

subjected to the terminal conditionλ (T ) =∇Ψ (T ) . (9)

The vector λ(t ) is designated by co-state and Eq. ( 8) by adjoint equation.

For each t , the Hamiltonian defined by

(x, u,λ) = λT F (x, u) + (x, u) (10)

is maximum for the optimal input u. If the maximum is for an u in the interior of then∂

∂ u i= 0. (11)

In the case that the optimum is in the boundary of then Eq. ( 11) does not apply.

2 Oscillator system

Let as consider a simple oscillator system comprising a spring, k , a energy extraction damper,c ,

and control damper, G, with damping as function of, u , see Fig. 1. The equation that describes the

motion of this system is

m ¨ z + c ̇ z + g u ˙ z + kz = f cos(Ωt ) (12)

where the dot denotes time derivative, m is the block mass, f and Ω are the modulus are angular

frequency of the external force. The initial conditions are z (0) = z0 and ˙ z(0) = v0.

The extracted energy, e , can be computed using the instantaneous power of the damper c which

is given by

ė = c ˙ z2. (13)

The initial condition is e (0) = 0.

The system of equations ( 12) and ( 13) can be written as a first-order system in vectorial form as

ẋ = F (x, u) , (14)

defining

x =

v z e T

= x1 x2 x3

T , (15)

2


3/6

Figure 1: Oscillating system with spring, k , energy extraction damper,c , and control damper, g .

and

F (x, u) =

F 1

F 2

F 3

=

( f cos(Ωt )− c x1− g ux1− k x2)/m

x1

c x21

, (16)

with initial conditions

x0 =

v0 z0 0T

. (17)

We want to determine u (t ) for 0≤ t ≤ T such that

J (u) = e (T ), (18)

is maximum subjected to the constraint 0 ≤ u ≤ 1.

In the present case Ψ (xT ) = e (T ) and (x, u) = 0, see Eq. ( 5). Using this, the adjoint equationis given by

− λ̇ = λT ∇F (x, u) . (19)

The gradient of F is given by

∇F =

∂ F 1∂ x1

∂ F 1∂ x2

∂ F 1∂ x3

∂ F 2∂ x1

∂ F 2∂ x2

∂ F 2∂ x3

∂ F 3∂ x1

∂ F 3∂ x2

∂ F 3∂ x3

=

− (c + g u)/m −k/m 0

1 0 0

2c x1 0 0

, (20)

resulting in

λ̇1

λ̇2

λ̇3

=−

−λ1 (c + g u)/m + λ2 + 2λ3c x1

−λ1k/m

0

. (21)

The final condition of λ is λ(T ) =∇Ψ (T ), givingλ1(T )

λ2(T )λ3(T )

=

∂ Ψ

∂ x1∂ Ψ ∂ x2∂ Ψ ∂ x3

x = xT

=

0

01

. (22)

3


4/6

For each t , the Hamiltonian

(x, u,λ) = λT F =λ1 λ2 λ3

( f cos(Ωt )− c x1− g ux1− k x2)/m

x1

c x21

= λ1 ( f cos(Ωt )− c x1− g ux1− k x2)/m +λ2 x1 +λ3c x

21

(23)

is maximum for the optimal input u. The conditions to maximize are

u(t ) =

1, if (−λ1(t ) x1(t ))≥ 0

0, otherwise(24)

The usual solution of this problem is:

1. Set u (t ) = 0.

2. Compute ( 14) using the ( 17).

3. Compute ( 21) backward (from T to 0) to impose the “final” condition ( 22).

4. Determine u (t ) using ( 24).

5. If not converged go to step 2.

A Demonstration of the Pontriagyn maximum principle

Let us define the functional as

= J −

T 0

λT [ẋ−F (x, u)] dt . (25)

The vector λ is introduced so is optimal and satisfies the system of ordinary differential equa-

tions ( 1) for all t ∈ [0, T ]. In other words,

T

0

λT [ẋ−F (x, u)] dt , (26)

is a constraint that is zero for the optimal solution1. The vector λ is called co-state. Introducing

Eq. ( 5) in Eq. ( 28) we get

= Ψ (xT ) +

T 0

(u, x) dt −

T 0

λT [ẋ−F (x, u)] dt

= Ψ (xT ) +

T

0

(u, x) +λT F (x, u)dt −

T

0

λT ẋ dt .

(27)

1Similar to a Lagrange multiplier.

4


5/6

Defining the Hamiltonian function

(x, u,λ) = λT F (x, u) + (x, u) , (28)

we get

= Ψ (xT ) +

T

0

(u, x,λ) dt −

T

0

λT ẋ dt . (29)

Let us assume that uopt is the function which maximizes then functional J then a small pertur-

bation δu results in a decrease of . The system will follow another path x +δx and the variation

of the objective function is negative

δ = (x +δx, v)− x, uopt

< 0, (30)

where v = uopt +δu. Using Eq. ( 29) we get

δ = Ψ (xT +δxT )−Ψ (xT ) +

T 0

[ (v, x +δx,λ)− (u, x,λ)] dt −

T 0

λT δẋ dt . (31)

Integrating by parts the last term of Eq. ( 31)

T 0

λT δẋ dt = λT (T )δx (T )−λT (0)δx (0)−

T 0

λ̇T δx dt (32)

Since the optimal control does not change x (0) we have δx (0). As a result,

δ =Ψ (xT +δxT )−Ψ (xT )−λT (T )δx (T )

+

T 0

[ (v, x +δx,λ)− (u, x,λ)] dt +

T 0

λ̇T δx dt .

(33)

Performing a first order Taylor series expansion of the terms on δx we get

Ψ (xT + δxT )≈Ψ (xT ) +∇Ψ (xT )δxT , (34)

and

(v, x +δx,λ)≈ (v, x,λ) +∇ (v, x,λ)δx (35)

Replacing both expansions in Eq. ( 33)

δ =∇Ψ (xT )−λ

T (T )δx (T ) +

T 0

∇ (u, x,λ) + λ̇

T δx dt

+

T 0

[ (v, x,λ)− (u, x,λ)] dt .

(36)

We can choose

λ̇T

=−∇ (u, x,λ) , (37)

5


6/6

satisfying

λT (T ) =∇Ψ (xT ) , (38)

to assure that the first term of the right-hand-side of Eq. ( 36) is zero. This results in

δ = T

0[ (v, x,λ)− (u, x,λ)] dt . (39)

The states x and co-states λ are computed for uopt and independently of v. If uopt is the optimum

then

uopt, x,λ

≥ (v, x,λ) (40)

∀v∈ . This needs to be proved because we compute uopt by maximizing (u, x,λ).

Suppose that there is an time instant, t 1 where a function w satisfies

(w (t 1) , x (t 1) ,λ (t 1)) > uopt (t 1) , x (t 1) ,λ (t 1)

(41)

Since is a continuous function, there is an interval [ t 1− ε, t 1 + ε] where this inequality holds.

Let w = uopt except in this interval. Using this choice, the variation is

δ =

t 1+εt 1−ε

(w (t ) , x (t ) ,λ (t ))−

uopt (t ) , x (t ) ,λ (t )

dt > 0 (42)

However, this contradicts the hypothesis that uopt is the optimal control.

References

[1] J. L. M. Lemos, Lectures on Optimal Control, Instituto Superior Técnico, 2012.

6

optimalctrl sp

Documents