![Page 1: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/1.jpg)
1
ME233 Advanced Control II
Lecture 10
Infinite-horizon LQRPART I
(ME232 Class Notes pp. 135-137)
![Page 2: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/2.jpg)
2
LTI Optimal regulators (review)
• State space description of a discrete time LTI
• Find optimal control
• That drives the state to the origin
![Page 3: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/3.jpg)
3
Finite Horizon LQ optimal regulator
(review)LTI system:
We want to find the optimal control sequence:
which minimizes the cost functional:
![Page 4: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/4.jpg)
4
LQ Cost Functional (review)
• total number of steps—“horizon”
• penalizes the final state
deviation from the origin
• penalizes the transient state deviation
from the origin and the control effort
symmetric
![Page 5: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/5.jpg)
5
Finite-horizon LQR solution (review)
Where P(k) is computed backwards in time using the
discrete Riccati difference equation :
![Page 6: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/6.jpg)
Properties of Matrix P(k) (review)
P(k) satisfies:
1)
2)
6
(symmetric)
(positive semi-definite)
![Page 7: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/7.jpg)
7
Example – Double Integrator
ZOHu(t)u(k) x(k)Tx(t)
v(k)T
1
s
1
s
v(t)
Double integrator with ZOH and sampling time T =1:
position
velocity
![Page 8: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/8.jpg)
Choose:
8
Example – Double Integrator
LQR cost:
only penalize
position x1and control u
![Page 9: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/9.jpg)
9
Example – Double Integrator (DI)
Compute P(k) for an arbitrary and N.
Computing backwards:
![Page 10: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/10.jpg)
10
0 2 4 6 8 100
1
2
3
4
5
6
7
8
9
k
P(1
,1),
P(1
,2),
P(2
,2)
Example – DI Finite Horizon Case 1
• N = 10 , R = 10,
P11(k)
P22(k)
P12(k)
![Page 11: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/11.jpg)
11
0 5 10 15 20 25 300
1
2
3
4
5
6
7
8
9
k
P(1
,1),
P(1
,2),
P(2
,2)
Example – DI Finite Horizon Case 2
• N = 30 , R = 10,
P11(k)
P22(k)
P12(k)
![Page 12: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/12.jpg)
12
0 5 10 15 20 25 300
1
2
3
4
5
6
7
8
9
k
P(1
,1),
P(1
,2),
P(2
,2)
Example – DI Finite Horizon Case 3
• N = 30 , R = 10,
P11(k)
P22(k)
P12(k)
![Page 13: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/13.jpg)
13
Example – DI Finite Horizon
Observation:
In all cases, regardless of the choice of
when the horizon, N, is sufficiently large
the backwards computation of the Riccati Eq.
always converges to the same solution:
![Page 14: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/14.jpg)
14
Infinite-Horizon LQ regulator
LTI system:
LQR that minimizes the cost:
• We now consider the limiting behavior when
![Page 15: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/15.jpg)
Optimal control:
LTI system:
15
Infinite Horizon (IH) LQ regulator
Consider the limiting behavior when
Riccati equation:
![Page 16: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/16.jpg)
1) When does there exist a BOUNDED limiting solution
to the Riccati Eq.
for all choices of ?
16
Infinite Horizon LQ regulator question 1
Consider the limiting behavior when
![Page 17: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/17.jpg)
2) When does there exist a UNIQUE limiting solution
to the Riccati Eq.
regardless of the choice of ?
17
Infinite Horizon LQ regulator question 2
Consider the limiting behavior when
![Page 18: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/18.jpg)
3) When does the limiting solution
to the Riccati Eq.
yield an asymptotically stable closed loop system?
18
Infinite Horizon LQ regulator question 3
Consider the limiting behavior when
is Schur
(all eigenvalues inside unit circle)
![Page 19: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/19.jpg)
19
LQ regulator Cost
Define the square root of , i.e.
Define the matrices C and D such that
![Page 20: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/20.jpg)
20
LQ regulator Cost
• Define the fictitious output p(k) such that
![Page 21: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/21.jpg)
21
Infinite Horizon LQ optimal regulator
LTI system:
Find optimal control which minimizes the cost functional:
![Page 22: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/22.jpg)
Stabilizability Assumption
We are only interested in the case where the
closed-loop dynamics are asymptotically stable
If (A,B) is not stabilizable, then there does not
exist a control scheme that results is
asymptotically stable closed-loop dynamics
For the infinite horizon optimal LQR
problem, we always assume that (A,B) is
stabilizable
22
![Page 23: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/23.jpg)
23
Theorem 1 : Existence of a bounded P∞
Let be stabilizable (uncontrollable modes are asymptotically stable)
Then, for , as
the “backwards” solution of the Riccati Eq.
converges to a BOUNDED limiting solution
that satisfies the algebraic Riccati equation (DARE):
![Page 24: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/24.jpg)
24
Theorem 1 : Notes
• Theorem-1 only guarantees the existence of
a bounded solution to the algebraic
Riccati Equation
• The solution may not be unique, i.e. different
final conditions may result in
different limiting solutions P∞ or may not
even yield a limiting solution!
![Page 25: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/25.jpg)
25
Theorem 2 : Existence and uniqueness of a
positive definite asymptotic stabilizing solution
If (A,B) is stabilizable and the state-space
realization C(zI - A)-1B + D has no transmission
zeros, then
1) There exists a unique, bounded
solution to the DARE
2) The closed-loop plant
is asymptotically stable
![Page 26: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/26.jpg)
26
Theorem 3 : Existence of a stabilizing solution
If (A,B) is stabilizable and the state-space
realization C(zI - A)-1B + D has no transmission
zeros satisfying , then
1) There exists a unique, bounded
solution to the DARE
2) The closed-loop plant
is asymptotically stable
![Page 27: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/27.jpg)
Theorem 4 : A different approach
The discrete algebraic Riccati equation (DARE) has a
solution for which is Schur if and only if
(A,B) is stabilizable and the state-space realization
has no transmission zeros on the unit circle.
Moreover, is the optimal control
policy that achieves asymptotic stability
27
![Page 28: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/28.jpg)
Special case: S=0
It turns out that the transmission zeros of
correspond to the unobservable modes of
(This will be assigned as a homework problem)
In Theorems 2 and 3, the transmission zeros
condition becomes an observability/detectability
condition
28
![Page 29: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/29.jpg)
29
Theorem 2 : Existence and uniqueness of a positive
definite asymptotic stabilizing solution, S = 0
If (A,B) is stabilizable and (C,A) is observable, then
1) There exists a unique, bounded
solution to the DARE
2) The closed-loop plant
is asymptotically stable
![Page 30: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/30.jpg)
30
If (A,B) is stabilizable and (C,A) is detectable, then
1) There exists a unique, bounded
solution to the DARE
2) The closed-loop plant
is asymptotically stable
Theorem 3 : Existence of a stabilizing solution, S = 0
![Page 31: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/31.jpg)
Theorem 4 : A different approach, S = 0
The discrete algebraic Riccati equation (DARE) has a
solution for which is Schur if and only if
(A,B) is stabilizable and (C,A) has no unobservable
modes on the unit circle.
Moreover, is the optimal control
policy that achieves asymptotic stability
31
![Page 32: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/32.jpg)
32
Notes, S=0
When (A,B) stabilizable and (C,A) observable or
detectable, the infinite-horizon cost (N →∞)
becomes
• The closed-loop plant is asymptotically stable
• Solution of the DARE is unique, independent of P(N)
![Page 33: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/33.jpg)
33
Explanation: why is stabilizability needed
not stabilizable
there are unstable uncontrollable modes
there might be some initial conditions such that
since the optimal cost is given by
![Page 34: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/34.jpg)
34
Explanation: why is detectability is needed, S=0
not detectable
there are unstable unobservable modes
these modes do not affect the optimal cost
no need to stabilize these modes
![Page 35: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/35.jpg)
35
Explanation: why is observability needed
The DARE can be written in the Joseph stabilized
form:
(closed-loop matrix)
define
![Page 36: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/36.jpg)
36
Explanation: why is observability needed
Joseph stabilized ARE:
Looks like a Lyapunov equation and, in fact, it
is the Lyapunov equation for the observability
Grammian of the pair
![Page 37: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/37.jpg)
37
Explanation: why is observability needed
Joseph stabilized ARE:
It can be shown that:
observable observable
asymptotically
stable (Schur)
stabilizable and
![Page 38: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/38.jpg)
38
Example – Double Integrator
LQR
![Page 39: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/39.jpg)
39
Example – Double Integrator
Penalize position in the infinite horizon cost functional:
Observable Controllable
![Page 40: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/40.jpg)
40
Example - Steady State Solution
• The steady state solution of the DARE:
• Use matlab function dare
• Get steady state answer:
![Page 41: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/41.jpg)
41
Example - Infinite Horizon LQ Regulator
• The control law is given by:
• Closed-loop poles are the eigenvalues of
• Use matlab command:
>> abs(eig(Ac))
is Schur
ans =
0.6736
0.6736
Answer
![Page 42: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/42.jpg)
42
Summary
• Convergence of LQR as horizon
– stabilizable
– detectable
• Infinite-horizon LQR
• Unique, positive definite solution of algebraic
Riccati equation
• Closed-loop system is asymptotically stable
![Page 43: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/43.jpg)
43
Additional Material (you are not
responsible for this)• Solutions of Infinite Horizon LQR using the
Hamiltonian Matrix
– (see ME232 class notes by M. Tomizuka)
• Strong and stabilizing solutions of the discrete time
algebraic Riccati equation (DARE)
• Some additional results on the convergence of the
asymptotic convergence of the discrete time Riccati
equation (DRE)
![Page 44: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/44.jpg)
44
Infinite Horizon LQ optimal regulator
Consider the nth order discrete time LTI system:
We want to find the optimal control which minimizes the
cost functional :
Assume:
• is controllable or asymptotically stabilizable
• is observable or asymptotically detectable
![Page 45: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/45.jpg)
45
Infinite Horizon LQR Solution:
Discrete time Algebraic Riccati (DARE) equation:
![Page 46: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/46.jpg)
46
Solution of the DARE
DARE:
1) Assume that A is nonsingular and define the 2n x 2n
Backwards Hamiltonian matrix:
2) Compute its first n eigenvalues ( ):
![Page 47: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/47.jpg)
47
Solution of the DARE
• The first n eigenvalues of H are the eigenvalues of
and are all inside the unit circle,
(I.e. asymptotically stable)
where
• The remaining eigenvalues of H satisfy:
![Page 48: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/48.jpg)
48
Solution of the DARE
4) Define the n x n matrices:
3) For each unstable eigenvalue of H
(outside the unit circle), compute its associated eigenvector :
![Page 49: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/49.jpg)
49
Solution of the ARE5) Finally, P is computed as follows:
• Matlab command dare: (Discrete ARE)
![Page 50: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/50.jpg)
Strong Solution of the DARE
A solution of the DARE
is said to be a strong solution
if the corresponding closed loop matrix Ac
has all its eigenvalues on or inside the unit circle.
50
![Page 51: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/51.jpg)
Stabilizing Solution of the DARE
A strong solution of the DARE
is said to be stabilizing
if the corresponding closed loop matrix Ac
is Schur, i.e. it has all its eigenvalues inside the
unit circle.
51
![Page 52: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/52.jpg)
Theorem – Solutions to the DARE
Provided that (A,B) is stabilizable, then
i. the strong solution of the DARE exists and
is unique.
ii. if (C,A) is detectable, the strong solution is
the only nonnegative definite solution of the
DARE.
iii. if (C,A) is has no unobservable modes on
the unit circle, then the strong solution
coincides with the stabilizing solution.
iv. if (C,A) has an unobservable mode on the
unit circle, then there is no stabilizing
solution.
52
![Page 53: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/53.jpg)
Theorem – Solution to the DARE
Provided that (A,B) is stabilizable, then
v. if (C,A) has an unobservable mode inside
or on the unit circle, then the strong solution
is not positive definite.
vi. if (C,A) has an unobservable mode outside
the unit circle, then as well as the the
strong solution, there is at least one
nonnegative definite solution of the DARE
53
S. W. Chan, G.C. Goodwin and K.S. Sin, “Convergence
properties of the Riccati difference equation in optimal filtering
of nonstabilizable systems, “IEEE Trans. of Automatic Control
AC-29 (1984) pp 110-118.
![Page 54: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/54.jpg)
Theorems - convergence of the DRE54
Consider the “backwards” solution of the discrete time
Riccati Equation
1) Subject to
i. (A,B) is stabilizable and (C,A) is detectable,
ii.
then, as P(k) converges exponentially to a
unique stabilizing solution P∞ of the DARE
![Page 55: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/55.jpg)
Theorems - convergence of the DRE55
Consider the “backwards” solution of the discrete time
Riccati Equation
2) Subject to
i. (A,B) is stabilizable
ii. (C,A) is has no unobservable modes on the unit circle
iii.
then, as P(k) converges exponentially to a
unique stabilizing solution P∞ of the DARE
![Page 56: ME233 Advanced Control II Lecture 10 Infinite-horizon LQR · • Solutions of Infinite Horizon LQR using the Hamiltonian Matrix –(see ME232 class notes by M. Tomizuka) • Strong](https://reader030.vdocuments.net/reader030/viewer/2022040115/5e7ed3db83ebf315b716aa2a/html5/thumbnails/56.jpg)
Theorems - convergence of the DRE56
Consider the “backwards” solution of the discrete time
Riccati Equation
3) Subject to
i. (A,B) is controllable
ii. or
then, as P(k) converges to a unique strong
solution P∞ of the DARE
S. W. Chan, G.C. Goodwin and K.S. Sin, “Convergence
properties of the Riccati difference equation in optimal filtering
of nonstabilizable systems, “IEEE Trans. of Automatic Control
AC-29 (1984) pp 110-118.