linear quadratic gausssian control design with loop transfer recovery [lqg…€¦ ·  ·...

Linear Quadratic Gausssian Control Design withLoop Transfer Recovery ∗

Leonid Freidovich

Department of MathematicsMichigan State University

MI 48824, USA

e-mail:leonid@math.msu.edu

http://www.math.msu.edu/˜leonid/

December 4, 2003

∗Presentation for ME-891, Fall 2003

1

Outline

1

Outline

(a) Linear quadratic regulation [LQR]

1

Outline

(a) Linear quadratic regulation [LQR]

(b) H2 design [LQG]

1

Outline

(a) Linear quadratic regulation [LQR]

(b) H2 design [LQG]

(c) Idea of loop transfer recovery [LTR]

1

Outline

(a) Linear quadratic regulation [LQR]

(b) H2 design [LQG]

(c) Idea of loop transfer recovery [LTR]

(d) Technical implementations of LQR/LTR

1

Outline

(a) Linear quadratic regulation [LQR]

(b) H2 design [LQG]

(c) Idea of loop transfer recovery [LTR]

(d) Technical implementations of LQR/LTR

(e) References

[LQR/LQG/LTR]

2

Linear Quadratic Regulation [LQR] ←x = Ax + B2u,x(0) = x0.

(1)

• (A,B2) is controllable.

2

Linear Quadratic Regulation [LQR] ←x = Ax + B2u,x(0) = x0.

(1)

• (A,B2) is controllable.

Goal: Design state feedback controller

u = K(s)x, [ K(s) is a proper real rational transfer function] (2)

depending on A and B2 such that for arbitrary x0 :

1. the closed-loop system (1), (2) is internally stable,

2. the solution x(t) and the control signal u(t) satisfy certainspecifications.

[LQR/LQG/LTR]

3

Let us choose C1, D12 and define performance index:

Jlqr = ‖z‖22 = ‖C1x + D12u‖22∆=

∞∫0

[C1x(t) + D12u(t)]∗[C1x(t) + D12u(t)] dt (3)

3

Let us choose C1, D12 and define performance index:

Jlqr = ‖z‖22 = ‖C1x + D12u‖22∆=

∞∫0

[C1x(t) + D12u(t)]∗[C1x(t) + D12u(t)] dt (3)

If C∗1D12 = 0 then Jlqr = ‖C1x‖22 + ‖D12u‖22.

3

Let us choose C1, D12 and define performance index:

Jlqr = ‖z‖22 = ‖C1x + D12u‖22∆=

∞∫0

[C1x(t) + D12u(t)]∗[C1x(t) + D12u(t)] dt (3)

If C∗1D12 = 0 then Jlqr = ‖C1x‖22 + ‖D12u‖22.

LQR problem: Design state feedback controller (2) depending on A, B2,C1, and D12 such that for arbitrary x0 :

1. the closed-loop system (1), (2) is internally stable,

2. the performance index (3) is the smallest.

[LQR/LQG/LTR]

4

The solution exists if

• D∗12D12 > 0,

• (C1, A) is detectable;

•[

A− jωI B2

C1 D12

]has full column rank for all ω.

[LQR/LQG/LTR]

5

Properties of the solution:

1. optimal controller K(s) = Klqr is the constant gain:

Klqr = −(D∗12D12)−1(B∗2X + D∗

12C1),

where X ≥ 0 is the stabilizing solution of the Riccati equation:

A∗X + XA−XB2(D∗12D12)−1B∗2X + C∗1 [I −D12(D∗

12D12)−1D∗12]C1 = 0,

with A = A−B2(D∗12D12)−1D∗

12C1,

5

Properties of the solution:

1. optimal controller K(s) = Klqr is the constant gain:

Klqr = −(D∗12D12)−1(B∗2X + D∗

12C1),

where X ≥ 0 is the stabilizing solution of the Riccati equation:

A∗X + XA−XB2(D∗12D12)−1B∗2X + C∗1 [I −D12(D∗

12D12)−1D∗12]C1 = 0,

with A = A−B2(D∗12D12)−1D∗

12C1,

2. A + B2Klqr is Hurwitz and limt→∞

x(t) = 0,

5

Properties of the solution:

1. optimal controller K(s) = Klqr is the constant gain:

Klqr = −(D∗12D12)−1(B∗2X + D∗

12C1),

where X ≥ 0 is the stabilizing solution of the Riccati equation:

A∗X + XA−XB2(D∗12D12)−1B∗2X + C∗1 [I −D12(D∗

12D12)−1D∗12]C1 = 0,

with A = A−B2(D∗12D12)−1D∗

12C1,

2. A + B2Klqr is Hurwitz and limt→∞

x(t) = 0,

3. guaranteed 6 dB (= 20 log 2) gain margin and 60o phase margin in bothdirections.

[LQR/LQG/LTR]

6

Linear Quadratic Gaussian [LQG or H2 ] problem ←Model:

x = Ax + B2u,y = C2x + D21u,x(0) = x0.

(4)

Assumptions:

• (A,B2) is controllable,

• (C2, A) is detectable.

[LQR/LQG/LTR]

7

Goal: Design output feedback controller

u = K(s)y, [ K(s) is a proper real rational transfer function] (5)

depending on A, B2, C2, and D21 such that for arbitrary x0 :

1. the closed-loop system (6), (5) is internally stable,

2. the solution x(t) and the control signal u(t) satisfy certainspecifications.

7

Goal: Design output feedback controller

u = K(s)y, [ K(s) is a proper real rational transfer function] (5)

depending on A, B2, C2, and D21 such that for arbitrary x0 :

1. the closed-loop system (6), (5) is internally stable,

2. the solution x(t) and the control signal u(t) satisfy certainspecifications.

Let us choose C1, D12, as before.

7

Goal: Design output feedback controller

u = K(s)y, [ K(s) is a proper real rational transfer function] (5)

depending on A, B2, C2, and D21 such that for arbitrary x0 :

1. the closed-loop system (6), (5) is internally stable,

2. the solution x(t) and the control signal u(t) satisfy certainspecifications.

Let us choose C1, D12, as before. It could be shown that if D12 6= 0,then the output controller minimizing Jlqr = ‖z‖2 = ‖C1x + D12u‖22 cannotbe proper (i.e. solution does not exist).

[LQR/LQG/LTR]

8

Considerx = Ax + B1w + B2u,y = C2x + D21u,x(0) = x0.

(6)

8

Considerx = Ax + B1w + B2u,y = C2x + D21u,x(0) = x0.

(6)

Define performance index

Jlqg = ‖Tw→z‖22 = max‖w‖2≤1

{‖z‖2∞‖w‖22

}= max‖w‖2≤1

{‖C1x + D12u‖2∞

‖w‖22

}. (7)

8

Considerx = Ax + B1w + B2u,y = C2x + D21u,x(0) = x0.

(6)

Define performance index

Jlqg = ‖Tw→z‖22 = max‖w‖2≤1

{‖z‖2∞‖w‖22

}= max‖w‖2≤1

{‖C1x + D12u‖2∞

‖w‖22

}. (7)

LQG problem: Design output feedback controller (5) depending on A, B1,B2, C1, C2, D12, and D21 such that for arbitrary x0 :

1. the closed-loop system (6), (5) is internally stable,

2. the performance index (7) is the smallest.[LQR/LQG/LTR]

9

The solution exists if

• D∗12D12 > 0,

•[

A− jωI B2

C1 D12

]has full column rank for all ω.

• D21D∗21 > 0 ,

•[

A− jωI B1

C2 D21

]has full row rank for all ω.

[LQR/LQG/LTR]

10

Properties of the solution:

1. optimal K(s) is the observer based controller:

u = Klqgx,˙x = Ax + B2u + Llqg(C2x + D21u− y),x(0) = x0,

10

Properties of the solution:

1. optimal K(s) is the observer based controller:

u = Klqgx,˙x = Ax + B2u + Llqg(C2x + D21u− y),x(0) = x0,

whereKlqg = Klqr

andLlqg = −(Y C∗2 + B1D

∗21)(D21D

∗21)−1,

where Y ≥ 0 is the stabilizing solution of the Riccati equation:

A∗Y + Y A− Y C∗2(D21D∗21)−1C2Y + B1[I −D∗

21(D21D∗21)−1D21]B∗1 = 0,

with A = A−B1D∗21(D21D

∗21)−1C2,

[LQR/LQG/LTR]

11

2. observer cannot be too fast since transient with peaking is not ‘compatible’with small values of ‖C1x + D12u‖∞,

11

2. observer cannot be too fast since transient with peaking is not ‘compatible’with small values of ‖C1x + D12u‖∞,

3. if w(t) ≡ 0 then limt→∞

x(t) = 0,

11

2. observer cannot be too fast since transient with peaking is not ‘compatible’with small values of ‖C1x + D12u‖∞,

3. if w(t) ≡ 0 then limt→∞

x(t) = 0,

4. in general, neither gain nor phase margins are guaranteed (have to checkrobustness for each particular design).

[LQR/LQG/LTR]

12

LQG with loop transfer recovery ←Consider the observer based controller for the system (4) with D21 = 0

x = Ax + B2u,y = C2x,x(0) = x0,

12

LQG with loop transfer recovery ←Consider the observer based controller for the system (4) with D21 = 0

x = Ax + B2u,y = C2x,x(0) = x0,

u = Klqrx,˙x = Ax + B2u + L(C2x− y),x(0) = x0.

(8)

Assumptions:

• (A,B2) is controllable,

• (C2, A) is detectable,

• Klqr is the optimal gain from the LQR problem.

[LQR/LQG/LTR]

13

Goal: Design the observer gain L so that the closed-loop system (8)recovers internal stability and some of the robustness properties (gain andphase margins) of the LQR design.

13

Goal: Design the observer gain L so that the closed-loop system (8)recovers internal stability and some of the robustness properties (gain andphase margins) of the LQR design.

Compare

-����

- x = Ax + B2u

�Klqr

6

u

u

w x

Figure 1: u = {Lt(s)}u

u = Klqr (sI −A)−1B2u︸ ︷︷ ︸x=Ax+B2u

def= {Lt(s)}u

[LQR/LQG/LTR]

14

and

-����

-

r�

x = Ax + B2u -

��Klqr

C2

observer

6

u

u x

w x y

Figure 2: u = {Lo(s)}u

u = Klqr [−(sI −A− LC2 −B2Klqr)−1LC2]︸ ︷︷ ︸observer: ˙x=Ax+B2[Klqrx]+LC2(x−x)

(sI −A)−1B2u︸ ︷︷ ︸x=Ax+B2u

def= {Lo(s)}u

14

and

-����

-

r�

x = Ax + B2u -

��Klqr

C2

observer

6

u

u x

w x y

Figure 2: u = {Lo(s)}u

u = Klqr [−(sI −A− LC2 −B2Klqr)−1LC2]︸ ︷︷ ︸observer: ˙x=Ax+B2[Klqrx]+LC2(x−x)

(sI −A)−1B2u︸ ︷︷ ︸x=Ax+B2u

def= {Lo(s)}u

We want Lo(s) ≡ Lt(s), but Lo(s) is biproper and Lt(s) is strictly proper.[LQR/LQG/LTR]

15

LQG/LTR problem: Design the observer gain L so that

1. the closed-loop system (8) is internally stable,

2. for all ω ∈ [0, ωmax] :Lo(jω) ≈ Lt(jω),

where ωmax � 1,target (under state feedback) open-loop transfer function is

Lt(s) = Klqr(sI −A)−1B2,

achieved (under output feedback) open-loop transfer function is

Lo(s) = KlqrTx→x(s)(sI −A)−1B2,

and ‘excited’ observer transfer function is

Tx→x(s) = [−(sI −A− LC2 −B2Klqr)−1LC2].[LQR/LQG/LTR]

16

Doyle – Stein formula ←It could be shown that if

−L[I − C2(jωI −A)−1L]−1 = B2[C2(jωI −A)−1B2]−1

thenLo(jω) = Lt(jω).

16

Doyle – Stein formula ←It could be shown that if

−L[I − C2(jωI −A)−1L]−1 = B2[C2(jωI −A)−1B2]−1

thenLo(jω) = Lt(jω).

Supposelimε→0{εL(ε)} = −B2W,

where W is not singular and ε� 1.

16

Doyle – Stein formula ←It could be shown that if

−L[I − C2(jωI −A)−1L]−1 = B2[C2(jωI −A)−1B2]−1

thenLo(jω) = Lt(jω).

Supposelimε→0{εL(ε)} = −B2W,

where W is not singular and ε� 1. Then

limε→0{B2W [εI + C2(jωI −A)−1B2W ]−1} = B2[C2(jωI −A)−1B2]−1

and hencelimε→0{Lo(jω)} = Lt(jω).

[LQR/LQG/LTR]

17

Remarks on(−L(ε)[I − C2(jωI −A)−1L(ε)]−1 ≈ B2[C2(jωI −A)−1B2]−1

)• In general, the choice L(ε) = −B2W/ε does not guarantee internal

stability of the closed-loop system.

17

Remarks on(−L(ε)[I − C2(jωI −A)−1L(ε)]−1 ≈ B2[C2(jωI −A)−1B2]−1

)• In general, the choice L(ε) = −B2W/ε does not guarantee internal

stability of the closed-loop system.

• Since

−L(ε)[I −C2(jωI −A)−1L(ε)]−1 = −(jωI −A)(jωI − [A + L(ε)C2])−1L(ε),

17

Remarks on(−L(ε)[I − C2(jωI −A)−1L(ε)]−1 ≈ B2[C2(jωI −A)−1B2]−1

)• In general, the choice L(ε) = −B2W/ε does not guarantee internal

stability of the closed-loop system.

• Since

−L(ε)[I −C2(jωI −A)−1L(ε)]−1 = −(jωI −A)(jωI − [A + L(ε)C2])−1L(ε),

if εL(ε) = −B2W + O(ε), then some of the eigenvalues of [A + L(ε)C2](i.e. poles of the observer error dynamics) approach the zeros of[C2(jωI −A)−1B2] (i.e. zeros of the plant) and the others approach infinityas ε→ 0.

17

Remarks on(−L(ε)[I − C2(jωI −A)−1L(ε)]−1 ≈ B2[C2(jωI −A)−1B2]−1

)• In general, the choice L(ε) = −B2W/ε does not guarantee internal

stability of the closed-loop system.

• Since

−L(ε)[I −C2(jωI −A)−1L(ε)]−1 = −(jωI −A)(jωI − [A + L(ε)C2])−1L(ε),

if εL(ε) = −B2W + O(ε), then some of the eigenvalues of [A + L(ε)C2](i.e. poles of the observer error dynamics) approach the zeros of[C2(jωI −A)−1B2] (i.e. zeros of the plant) and the others approach infinityas ε→ 0.

• If the plant is nonminimum-phase then A + L(ε)C2 is not Hurwitz forsmall values of ε.

17

Remarks on(−L(ε)[I − C2(jωI −A)−1L(ε)]−1 ≈ B2[C2(jωI −A)−1B2]−1

)• In general, the choice L(ε) = −B2W/ε does not guarantee internal

stability of the closed-loop system.

• Since

−L(ε)[I −C2(jωI −A)−1L(ε)]−1 = −(jωI −A)(jωI − [A + L(ε)C2])−1L(ε),

if εL(ε) = −B2W + O(ε), then some of the eigenvalues of [A + L(ε)C2](i.e. poles of the observer error dynamics) approach the zeros of[C2(jωI −A)−1B2] (i.e. zeros of the plant) and the others approach infinityas ε→ 0.

• If the plant is nonminimum-phase then A + L(ε)C2 is not Hurwitz forsmall values of ε.

• Some of the observer’s modes are fast. Therefore peaking phenomenonmust occur.

[LQR/LQG/LTR]

18

LQR design of L(ε) ←Consider LQR problem for the system

˙x = Ax + B2u,x(0) = x0,

where A = A∗ and B2 = C∗2 .

18

LQR design of L(ε) ←Consider LQR problem for the system

˙x = Ax + B2u,x(0) = x0,

where A = A∗ and B2 = C∗2 .

Take C1(ε) = B∗2/ε (cheap control)

18

LQR design of L(ε) ←Consider LQR problem for the system

˙x = Ax + B2u,x(0) = x0,

where A = A∗ and B2 = C∗2 .

Take C1(ε) = B∗2/ε (cheap control) and D12 independent of ε so thatD∗

12D12 = R > 0 and C∗1D12 = 0.

18

LQR design of L(ε) ←Consider LQR problem for the system

˙x = Ax + B2u,x(0) = x0,

where A = A∗ and B2 = C∗2 .

Take C1(ε) = B∗2/ε (cheap control) and D12 independent of ε so thatD∗

12D12 = R > 0 and C∗1D12 = 0.

Suppose Klqr(ε) is the solution gain of the problem above for each ε > 0.

TakeL(ε) =

[Klqr(ε)

]∗.

[LQR/LQG/LTR]

19

Then [A + B2Klqr(ε)

]∗ = A + L(ε)C2

is Hurwitz

19

Then [A + B2Klqr(ε)

]∗ = A + L(ε)C2

is Hurwitz andL(ε) = −X(ε)C∗2R−1,

where X(ε) is the stabilizing solution of the Riccati equation

AX(ε) + X(ε)A∗ − X(ε)C∗2R−1C2X(ε) + B2B∗2/ε2 = 0.

19

Then [A + B2Klqr(ε)

]∗ = A + L(ε)C2

is Hurwitz andL(ε) = −X(ε)C∗2R−1,

where X(ε) is the stabilizing solution of the Riccati equation

AX(ε) + X(ε)A∗ − X(ε)C∗2R−1C2X(ε) + B2B∗2/ε2 = 0.

It could be shown that if the plant is of minimum phase (and left invertible)then lim

ε→0{ε2[AX(ε) + X(ε)A∗]} = 0 and

limε→0{εL(ε)} = lim

ε→0{−εX(ε)C∗2R−1} = −B2R

−1/2 = −B2W

[LQR/LQG/LTR]

20

Alternative designs of L(ε) ←

For the ‘broken’ closed-loop system

x = Ax + B2u,y = C2x + D21u,x(0) = x0,

20

Alternative designs of L(ε) ←

For the ‘broken’ closed-loop system

x = Ax + B2u,y = C2x + D21u,x(0) = x0,

u = Klqrx,˙x = Ax + B2u + L(C2x + D21u− y),x(0) = x0.

we have: Lt(s) = Klqr(sI −A)−1B2 and

Lo(s) = Klqr[−(sI−A−LC2−B2Klqr+LD21Klqr)−1]L[C2(sI−A)−1B2+D21].[LQR/LQG/LTR]

21

It could be shown that

Lt(s)− Lo(s) = M(s)[I + M(s)]−1(I −KlqrB2),

where M(s) = −Klqr([sI −A]−1 − LC2)−1(B2 + LD21).

Therefore it is reasonable to search for L that minimizes ‖M(s)‖2 or‖M(s)‖∞.

Also, a pole-placement technique can be used to design the observer gain aswell. However, it is not trivial and the system need to be transformed into thespecial coordinate basis.

[LQR/LQG/LTR]

22

References

[1] H. Kwakernaak and R. Sivan, Linear optimal control systems, N.Y.:Wiley-Interscience, 1972.

[2] J.C. Doyle, “Guaranteed margins for LQG regulators,” IEEE Trans. onAC, Vol. 23, pp. 756–757, 1978.

[3] J.C. Doyle and G. Stein, “Robustness with observers,” IEEE Trans. onAC, Vol. 24, pp. 607–611, 1979.

[4] J.C. Doyle and G. Stein, “Multivariable feedback design: concepts for aclassical/Modern synthesis,” IEEE Trans. on AC, Vol. 26, pp. 4–16, 1981.

[5] G. Stein and M. Athans, “The LQG/LTR procedure for multivariablecontrol design,” IEEE Trans. on AC, Vol. 32, pp. 105–114, 1987.

[LQR/LQG/LTR]

23

[6] J.C. Doyle, K. Glover, P.P. Khargonekar, and B.A. Francis, “State-spacesolution to standard H2 and H∞ control problems,” IEEE Trans. onAC, Vol. 34, pp. 831–847, 1989.

[7] A. Saberi, B.M. Chen, and P. Sanuti, Loop transfer recovery: analysisand design, London: Springer-Verlag, 1993.

[8] K. Zhou and J.C. Doyle, Essentials of robust control, N.J.: Prentice-Hall,1998.

[9] J.S. Freudenberg, Robustness with observers, Lecture notes, Universityof Michigan, 1999.

[10] C. Gokcek, ME-891: Robust and Optimal Control, Lecture notes,Michigan State University, 2003.

[11] B.M. Chen, Loop transfer recovery, Lecture notes, 2003.←

[LQR/LQG/LTR]