linear optimal control - uniroma1.itiacoviel/materiale/optimalcontrol-lecture7.pdfd. liberzon,...

Optimal Control

Lecture

Prof. Daniela Iacoviello

Department of Computer, Control, and Management Engineering Antonio Ruberti

Sapienza University of Rome

10/11/2015 Controllo nei sistemi biologici

Lecture 1

Pagina 2

Prof. Daniela Iacoviello

Department of computer, control and management

Engineering Antonio Ruberti

Office: A219 Via Ariosto 25

http://www.dis.uniroma1.it/

Prof.Daniela Iacoviello- Optimal Control

References B.D.O.Anderson, J.B.Moore, Optimal control, Prentice Hall, 1989 C.Bruni, G. Di Pillo, Metodi Variazionali per il controllo ottimo, Masson , 1993 L. Evans, An introduction to mathematical optimal control theory, 1983 H.Kwakernaak , R.Sivan, Linear Optimal Control Systems, Wiley Interscience, 1972 Prof.Daniela Iacoviello- Optimal Control

References

D. E. Kirk, "Optimal Control Theory: An Introduction, New

York, NY: Dover, 2004 D. Liberzon, "Calculus of Variations and Optimal Control

Theory: A Concise Introduction", Princeton University

Press, 2011 How, Jonathan, Principles of optimal control, Spring

2008. (MIT OpenCourseWare: Massachusetts Institute of

Technology). License: Creative Commons BY-NC-SA.

Prof.Daniela Iacoviello- Optimal Control

The linear time-optimal control

Let us consider the problem of optimal control

of a linear system

with fixed initial and final state,

constraints on the control variables and

with cost index equal to the length of the

time interval Prof.Daniela Iacoviello- Optimal Control

Problem: Consider the linear dynamical system:

With and the constraint

Matrices A and B have entries with elements of class

Cn-2 and Cn-1 respectively.

The initial instant ti is fixed and also:

)()()()()( tutBtxtAtx pn RtuRtx )(,)(

Rtpjtu j ,,...,2,1,1)(

0)(,)( Txxtx i

i

Prof.Daniela Iacoviello- Optimal Control

The aim is to determine the final instant

the control and the state

that minimize the cost index:

)(RCu oo

RT o

)(1 RCxo

i

T

t

tTdtTJ

i

)(

Prof.Daniela Iacoviello- Optimal Control

Theorem: Necessary conditions for

to be an optimal solution are that there exists a

constant and an n-dimensional

function , not simultaneously

null and such that:

Possible discontinuities in can appear only in

the points in which has a discontinuity.

*** ,, Tux

0*

0 TtC i ,

1*

pjRBuB

A

j

pTT

T

,...,2,1,1:***

**

*

*uProf.Daniela Iacoviello- Optimal Control

Moreover the associated Hamiltonian is a

continuous function with respect to t and it

results:

Proof: The Hamiltonian associated to the problem is:

Applying the minimum principle the theorem is proved.

0*

* TH

BuAxtuxH TT 00 ,,,,

Prof.Daniela Iacoviello- Optimal Control

Notation:

Let us indicate with the j-th column of the matrix B . The strong controllability corresponds to the condition:

where :

)(tb j

i

n

jjjj

ttpj

tbtbtbtG

,,...,2,1

0)()()(det)(det )()2()1(

nktbtAtbtb

tbtb

k

j

k

j

k

j

jj

,...,3,2)()()()(

)()(

)1()1()(

)1(

Prof.Daniela Iacoviello- Optimal Control

Remark:

Strong controllability corresponds to the controllability in any instant ti, in any time interval and by any component of the control vector

Prof.Daniela Iacoviello- Optimal Control

Remark:

In the NON-steady state case the above condition is sufficient for strong controllability.

In the steady case it is necessary and sufficient and may be written in the usual way:

Prof.Daniela Iacoviello- Optimal Control

,,...,2,10det 1 pjbAAbb j

n

jj

The Pontryagin principle- remind

Problem 3: Consider the dynamical system:

with:

tuxfx ,,

niURCx

ff

RUtuRtx

n

i

pn

,...,2,1,,

)(,)(

0

Prof.Daniela Iacoviello- Optimal Control

Assume fixed the initial control instant and the

initial state

while for final values assume:

where is a function of dimension

of C1 class.

Define the performance index :

with

i

i xtx )(

T

ti

duxLTuxJ ),(),(,,

niRURCt

L

x

LL n

i

,...,2,1,,, 0

0),( TTx 1 nf

Prof.Daniela Iacoviello- Optimal Control

Determine the value

the control and the state

that satisfy the dynamical

system, the constrain t on the control, the

initial and final conditions and minimize the

cost index .

,itT

)(0 RCuo

)(1 RCxo

Prof.Daniela Iacoviello- Optimal Control

Theorem 3- remind:

Consider an admissible solution

such that

IF it is a minimum there exist a constant

and an n-dimensional vector

not simultaneously null such that :

*** ,, Tux

00

*1* ,TtC i

fTTx

rank

*

),(

Prof.Daniela Iacoviello- Optimal Control

RkkdH

Hx

HT

t

T

,,

**

**

*

U

ttutxHttxH

,)(,),(),()(,,),( **

0

****

0

*

Moreover there exists a vector such that:

T

T

T

TH

TxT

**

*

**

)()(

fR

Prof.Daniela Iacoviello- Optimal Control

SINGULAR SOLUTIONS

Definition: Let be an optimal solution of the above problem, and the corresponding multipliers.

The solution is singular if there exists a subinterval

in which the Hamiltonian

Is independent from at least one component of in

Prof.Daniela Iacoviello- Optimal Control

ooo Tux ,,oo 0

''','',' tttt

tttxH ooo ),(,,),( 0

'',' tt

SINGULAR SOLUTIONS

Theorem:

Assume the Hamiltonian of the form

Let be an extremum and the corresponding multipliers such that

Is dependent on any compenent of in any subinterval of

A necessary condition for to be a singular extremum is that there exists a subinterval

such that:

Prof.Daniela Iacoviello- Optimal Control

*** ,, Tux **

0

),,,,(),,,(),,,(),,,,( 002010 tuxNtxHtxHtuxH

),,,,( *

0** txN

],[ Tti

*** ,, Tux

'''],,[]'','[ ttTttt i

]'','[,0),,,( 02 ttttxH

Characterization of the optimal solution

Theorem: Let us consider the minimal time optimal problem .

If the strong controllability condition is satisfied and if an optimal solution exists

it is non singular.

Moreover every component of the optimal control is a piecewise constant assuming only the extreme values and the number of discontinuity instant is limited.

1

Prof.Daniela Iacoviello- Optimal Control

Proof: the non singularity of the solution is proved by contradiction arguments.

If the solution were singular, from the necessary condition of the previous theorem:

There would exist a value

and a subinterval such that:

pjR

BuB

j

p

TT

,...,2,1,1:

***

pj ,...,2,1

],[]'','[ o

i Tttt

]'','[,0)()( ttttbt j

oT Prof.Daniela Iacoviello- Optimal Control

Deriving successively this expression we obtain:

On the other hand must be different from zero in every instant of the interval , otherwise, from the necessary condition

it should be null in the whole interval and in particular .

From the necessary condition:

it would result that also which is absurd.

Therefore, from the condition:

]'','[,0)()( ttttGt j

oT

)(to

** TA0)( To

0*

* TH

00 o

]'','[,0)()( ttttGt j

oT Prof.Daniela Iacoviello- Optimal Control

it results that which contradicts the hypothesis of strong controllability . From the non singularity of the optimal solution it results that the quantity can be null only on isolated points . Therefore the necessary condition implies

]'','[0)(det ttttG j

)()( tbt j

oT

pjRBuB j

pTT ,...,2,1,1:***

],[,...,2,1

,)()()(

o

i

j

oTo

j

Tttpj

tbtsigntu

Prof.Daniela Iacoviello- Optimal Control

To demonstrate that the number of discontinuity instants is finite, we can proceed by contradiction argument that would

Easily imply

which contradicts the hypothesis of strong controllability

]'','[0)(det ttttG j

Prof.Daniela Iacoviello- Optimal Control

A control function that assumes only

the limit values is called bang-bang

control and

the relative instants of discontinuity

are called commutation instants

Prof.Daniela Iacoviello- Optimal Control

Remark

This result holds also to the case in which the control is in the space of measurable function defined on the space of real number R, M(R).

Prof.Daniela Iacoviello- Optimal Control

Theorem (UNIQUENESS):

If the hypothesis of strong controllability is satisfied, if an optimal solution exists it is unique.

Proof: the theorem is proved by contradiction arguments.

Prof.Daniela Iacoviello- Optimal Control

Existence of the optimal solution

Theorem: if the condition of strong controllability is satisfied and an admissible solution exists, then there exists a unique optimal solution nonsingular and the control is bang-bang.

Proof: the existence theorem is proved on the basis of the following results.

Prof.Daniela Iacoviello- Optimal Control

The following result holds:

Theorem: Let assume that the control functions belong to the space M(R).

If an admissible solution exists then the optimal solution exists.

Proof: let us consider the non trivial case in which the number of admissible solution is infinite and let us indicate with the minimum extremum of the instants T corresponding to the admissible solutions

oT

Prof.Daniela Iacoviello- Optimal Control

It is possible to built a sequence of admissible solutions

such that

Let us indicate with the transition matrix of the linear system. It results:

o

kkkk TTTux ,,,o

kk

TT

lim

,t

0)()(,lim

)()(,,lim

,,lim)()(lim

dttutBtT

dttutBtTtT

xtTtTTxTx

k

T

T

kk

k

T

t

o

kk

i

i

o

ikk

o

kkkk

k

o

o

i

Prof.Daniela Iacoviello- Optimal Control

Since it results:

Let us indicate by the function

truncated on the interval . This

function is limited and L2 . The sequence of

functions belongs to the space of

measurable function M , L2 and such that

,....2,1,0)( kTx kk

0)(lim

o

kk

Tx

ku ku

o

i Tt ,

ku

o

ij Tttpjtu ,,,...,2,1,1)(

Prof.Daniela Iacoviello- Optimal Control

Therefore this sequence admits a subsequence

weakly converging to a function .

This implies that for any measurable L2 function it results:

If we indicate by the evolution of the state corresponding to the input from the initial condition we have:

ku Muo

h

o

i

o

i

T

t

oT

T

t

K

T

kdttuthdttuth )()()()(lim

oxou

Prof.Daniela Iacoviello- Optimal Control

Therefore, taking into account the expression

, we deduce :

An admissible measurable control L2

exists able to transfer the initial

state to the origin.

The optimal solution is

)()()(,,

)()(,lim,)(lim

oo

T

t

ooi

i

o

T

t

k

o

k

i

i

oo

kk

TxdttutBtTxtT

dttutBtTxtTTx

o

i

o

i

0)(lim

o

kk

Tx 0)( oo Tx

ooo Tux ,,Prof.Daniela Iacoviello- Optimal Control

Therefore it results in:

If the strong controllability condition is satisfied and if an optimal control in the space of measurable function exists then this solution is unique, non singular and the control is bang bang type

Remark

The existence of the optimal solution is guaranteed only for the couples

for which the admissible solution exists

ii xt ,

Prof.Daniela Iacoviello- Optimal Control

If the strong controllability

condition is satisfied and if an

optimal control in the space of

measurable function exists

then this solution is unique,

non singular and the control

is bang bang type

If the strong controllability

condition is satisfied and if an

admissible solution exists then

this solution is unique,

non singular and the control is

bang bang type

Necessary condition for with u in M(R) to be an optimal solution; its characterization; unicity results

*** ,, Tux

Summary

Prof.Daniela Iacoviello- Optimal Control

Minimum time problem for steady state system

Let us consider the minimum time problem in case of steady state system.

In this case it is possible to deduce results about the number of commutation points

Prof.Daniela Iacoviello- Optimal Control

Problem: Consider the linear dynamical system:

With and the constraint

Matrices A and B have entries with elements of class

Cn-2 and Cn-1 respectively.

The initial instant ti is fixed and also:

)()()( tButAxtx pn RtuRtx )(,)(

Rtpjtu j ,,...,2,1,1)(

0)(,)( Txxtx i

i

Prof.Daniela Iacoviello- Optimal Control

The aim is to determine the final instant

the control and the state

that minimize the cost index:

)(RCu oo

RT o

)(1 RCxo

i

T

t

tTdtTJ

i

)(

Prof.Daniela Iacoviello- Optimal Control

Theorem:

Let us consider the above time minimum problem and

assume that the control function belong to the space

of measurable function M(R); if the system is

controllable there exists a neighbor Ω of the origin

such that for any there exists an optimal

solution.

ix

Prof.Daniela Iacoviello- Optimal Control

Proof:

We will show that it is possible to determine a neighbor

Ω of the origin such that for any there exists

an admissible solution. On the basis of the previous

results this implies the existence of the optimal

solution.

Let us fix any T>0. The existence of a control able to

transfer the initial state to the origin at the

instant T corresponds to the possibility of solving :

ix

ix

Prof.Daniela Iacoviello- Optimal Control

with respect to the control function u.

Let assume for the control the expression:

and substitute it in the preceeding equation:

0)(

T

t

TAitTA

i

i dBuexe

p

i

AT RTteBuT

,,)(

T

t

ATAiAt

i

Ti deBBexe

Gramian matrix

Prof.Daniela Iacoviello- Optimal Control

From the controllability hypothesis, the Gramian

matrix is not singular, therefore:

a measurable admissible control exists if the

initial state is sufficiently near the origin.

Tt

xedeBBeeBu

i

iAt

T

t

ATAAT i

i

TT

,)(

1

Prof.Daniela Iacoviello- Optimal Control

Remark

From results obtained in the non steady state case it results that an optimal non singular solution exists and the control is bang-bang type.

Prof.Daniela Iacoviello- Optimal Control

Theorem: Let us consider the above time

minimum problem in the steady state case

and assume that the control function belongs

to the space of measurable function M(R).

If the system is controllable and the eigenvalues

of matrix A have negative real part there

exists an optimal solution whatever the initial

state is.

Prof.Daniela Iacoviello- Optimal Control

Proof: Whatever the initial state is chosen, an

admissible solution can be defined as follows:

• Let Ω be a neighbor of the origin constituted by the

original state for which an admissible solution exists;

• Let Ω’ be a closed subset in Ω

• From the asymptotic stability of the system it is

possible to reach Ω’ in a finite time with null input,

from any initial state

Prof.Daniela Iacoviello- Optimal Control

Remark

From results obtained in the non steady state case it results that an optimal non singular solution exists and the control is bang-bang type.

Question: how many commutation instants are possible in the optimal time control problem?

Prof.Daniela Iacoviello- Optimal Control

Theorem: If the strong controllability condition is satisfied and if all the eigenvalues of the matrix A are real non positive, then the number of commutation instants for any components of control is less or equal than

n-1, whatever the inititial state is.

Proof: from the previous results, we have:

o

i

j

oTo

j

Tttpj

btsigntu

,,,...,2,1

,)()(

Prof.Daniela Iacoviello- Optimal Control

where the costate in the steady state case is:

Denoting by and the eigenvalues of A and their molteplicity, the r-th component of

can be expressed as follows:

Where prs is a polynomial function of degree less than ms.

0

i

ttA iT

et )(0 )(

k km

0

nretpt

k

s

tt

rsris ,...,2,1,)()(

1

)(0

Prof.Daniela Iacoviello- Optimal Control

Substituting this expression in the bang bang control we obtain:

o

i

k

s

tt

js

k

s

tt

n

r

rsjr

o

j

Tttpj

etPsign

etpbsigntu

is

is

,;,...,2,1

,)(

)()(

1

)(

1

)(

1

Polynomial function of degree less than ms

Prof.Daniela Iacoviello- Optimal Control

The argument of the sign function has at most

m1 + m12+ …. + mk-1 =n-1

real solutions.

Therefore the control has at most n-1 roots.

o

ju

Prof.Daniela Iacoviello- Optimal Control

References

B.D.O.Anderson, J.B.Moore, Optimal control, Prentice Hall, 1989 L. Evans, An introduction to mathematical optimal control theory, 1983 H.Kwakernaak , R.Sivan, Linear Optimal Control Systems, Wiley Interscience, 1972

Prof.Daniela Iacoviello- Optimal Control