singular value decomposition analysis introduction€¦ · feedback performance specifications in...

Singular Value Decomposition Analysis

376_069 Multivariable feedback control V3 1 of 36


Introduction • Introduce a linear algebra tool: singular

values of a matrix • Motivation Why do we need singular values in

MIMO control designs? • Definition and properties of singular values • Singular value decomposition (SVD)

provides directional information



SISO sinusoidal steady-state (I) u(s) g(s) y(s) • Assume: g(s) strictly stable • Sinusoidal steady-state u(t) = uejωt ⇒ y(t) = yejωt y = g(jω)u • Note g(jω) is a complex scalar g(jω) = |g(jω)| ejφ(ω) Magnitude: |g(jω)| = g*(jω)g(jω) Phase: φ(ω) = Im( (j ))arctan Re( (j ))

gg

ωω

⎧ ⎫⎪ ⎪⎨ ⎬⎪ ⎪⎩ ⎭

Above plotted in Bode plot defines frequency response of SISO plant g(s) • |g(jω)| defines plant gain at frequency ω



SISO sinusoidal steady-state (II)

• Input with complex amplitude u(t) = uejωt ⇒ u = |u|ejψ u(t) = |u|ej(ωt + ψ) • Interpretation Re{u(t)} = |u|cos(ωt + ψ) Im{u(t)} = |u|sin(ωt + ψ) • Complex steady-state output y(t) = y ejωt y = g(jω) u

= |g(jω)| ejφ(ω) |u|ejψ = |g(jω)| |u| ej(φ(ω) + ψ) |y| = |g(jω)| |u| Re{y(t)} = |y|cos(ωt + ψ + φ(ω)) Im{y(t)} = |y|sin(ωt + ψ + φ(ω))



SISO Bode plot information

• Provides graphical "summary" of plant gain

at different frequencies • Concepts of "small" and "large" gain are

clear |g(jω)| >> 1 ⇒ large gain |g(jω)| << 1 ⇒ small gain



MIMO sinusoidal steady-state u(s) g(s) y(s) • Assume: G(s) strictly stable

• Sinusoidal inputs generate at steady-state

sinusoidal outputs

• Sinusoidal steady-state u(t) = uejωt ;u ∈ C m ⇒ y(t) = yejωt ;y ∈ C p y = G(jω)u • G(jω) : p x m complex matrix • Need notion of size of G(jω) vs. frequency

want visualize MIMO gain on Bode plot



Issues • Deal with complex vectors and complex

matrices.

• How to quantify "large" and "small" • Impact of directions y(s) = G(s) u(s) u(t) = uejωt ;u ∈ C m ⇒ y(t) = yejωt ;y ∈ C p y = G(jω)u Direction and size of u and Plant properties at frequency ω yield Direction and size of y • Singular value decomposition (SVD) provides

the "tool" for analysis



Notes on complex vectors • x is a complex vector; x ∈ C n

x = ⎝⎜⎜⎛

⎠⎟⎟⎞x1

x2 :

xn

xi = ai + jbi ; i = 1, 2, ..., n • Magnitude x = ||x||2 xH = [x*1 x*2 ... x*n] ||x||2 = xHx • Example x = 1

2 3jj

⎛ ⎞⎜ ⎟⎜ ⎟⎜ ⎟⎝ ⎠

+−

;xH = [1 - j 2 + j3]

xHx = (1 - j)(1 + j) + (2 + j3)(2 - j3) = 15 ⇒ ||x||2 = 15



Notes on complex matrices • A is a n x m matrix with complex-valued

elements: aik = αik + jβik • Notation AH : complex-conjugate transpose of A AH is a m x n matrix • Note: AHA m x m matrix AAH n x n matrix • Fact: AAH and AHA have real non-negative

eigenvalues • Example A =

1 12j

j j

⎛ ⎞⎜ ⎟⎜ ⎟⎜ ⎟⎝ ⎠

+− +

⇒ AH =11 2

jj j

⎛ ⎞⎜ ⎟⎜ ⎟⎜ ⎟⎝ ⎠− −

det(λΙ - AHA) = λ2 - 9λ + 5 ;λ1 = 8.41 ;λ2 = 0.59 det(λΙ - AHA) = det(λΙ - AAH) • In this example λi[AHA] = λi[AAH] > 0



Definition of singular values • A is a n x m complex matrix • Suppose : rank(A) = k • Notation σi(A): singular value of A • Definition: The strictly positive square roots of the non-zero eigenvalues of AHA ( and AAH equivalently), are the singular values of A σi(A) = λi[AHA] = λi[AAH] > 0 i = 1, 2, ..., k



The singular value decomposition

• A is a n x m complex matrix: rank(A) = k • σ1 ≥ σ2 ≥ ... ≥ σk ≥ 0 : are singular values of A

Σ =

0 ... 0 010 ... 0 02... ... ... ... ...0 0 ... 0

0 0 ... 0 0k

σ

σ

σ

⎛ ⎞⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎝ ⎠

Σ n x m real matrix • SVD A = U Σ VH A = UH Σ V U : n x n unitary matrix (UH = U-1) V : m x m unitary matrix (VH = V-1) • Column vectors of U and V are orthonormal



More on SVD • SVD A = U Σ VH A = UH Σ V • U : n x n unitary matrix (UH = U-1) U = [u1 u2 ... un] ;uH

iuj = δij ui : Left singular vectors of A (is right eigenvector of AAH associated with λi[AAH] ) • V : m x m unitary matrix (VH = V-1) V = [v1 v2 ... vm] ;vH

ivj = δij vi : Right singular vectors of A (is right eigenvector of AHA associated with λi[AHA] )



Geometric interpretation • A complex n x n matrix • A-1 exists ⇒ λi(A) ≠ 0 • Consider linear transformation y = Ax ;x , y ∈ Rn x A y • Euclidean norm ||x||2 = x'x ;||y||2 = y'y • Spectral norm of matrix A

A

2=max

x≠0

Ax2

x2

= 2

2x 1max Ax

=

• Singular value relations

σ max ( A) = max

x2=1

Ax2= A

2

σ min ( A) = min

x2=1

Ax2= 1

A−1

2



Graphical visualization • Real case : y = Ax ; y, x ∈ Rn, A ∈ Rnxn; n=2

• SVD A = U Σ VH

Σ = ⎝⎜⎜⎛

⎠⎟⎟⎞σ1 = σmax 0

0 σ2=σmin

U = [u1 u2] V = [v1 v2] Vector OD = v1 ; Vector OD" = u1 Vector OE = v2 ; Vector OE" = u2 Length of vector OD' = σmax = σ1

Length of vector OE' = σmin = σ2



MIMO frequency response u(t) = uejωt G(s) y(t) = yejωt • Restrict u to unit (complex) sphere, i.e. ||u||2 = 1 • ui(t) is complex sinusoid, e.g.

ui(t) = |ui|ejψiejωt Re{ui(t)} = |ui|cos(ωt + ψi) • Output response y(s) = G(s)u(s) • Singular values σmax(G(jω)) =

22u 1

max G(j )uω=

= ||ymax(ω)||2

σmin(G(jω)) =

22u 1

min G(j )uω=

= ||ymin(ω)||2 • Maximum and minimum singular values of

G(jω) define max and min amplification of unit sinusoidal input at frequency ω



Bode plot visualization



Discussion • The concept of singular values will be heavily

exploited in analysis and design of MIMO feedback systems

• Correct interpretation of singular value plot

hinges on units of physical variables (scaling) • Singular value results assume "roundness"

(convexity) of input signal space u(t) = uejωt G(s) y(t) = yejωt Input Space Output Space

Feedback Performance Specifications in the Frequency Domain


Feedback Performance Specifications in the Frequency

Domain Introduction • Use singular values to establish nature of

mimo performance specs in frequency domain • Performance attributes

- command following - disturbance rejection - insensitivity to sensor noise

• Stability-robustness to be addressed later



Fundamental relations

• True tracking error: e(s) = r(s) - y(s) • Loop TFM: L(s) L(s) = G(s)K(s) • Sensitivity TFM: S(s) S(s) = [Ι + L(s)]-1 • Closed-loop TFM: T(s) T(s) = [Ι + L(s)]-1L(s) • Performance equation

e(s) = S(s)[r(s) - d(s)] + T(s)n(s)

Constraint: T(s) + S(s) = Ι



Command following (I)

• Sinusoidal command yields sinusoidal error r(t) = rejωt ⇒ e(t) = eejωt • Relation: e = S(jω)r ||e||2 ≤ σmax[S(jω)]||r||2 • Ωr range of frequencies where command input

has energy • Prescription for good command following make σmax[S(jω)] << 1 for all ω ∈ Ωr



Command following (II) • Interpretation for unit command sinusoid r(t) = rejωt ; ||r||2 = 1 • Worst error at frequency ω e(t) = eejωt ||e||2 = σmax[S(jω)] Attained when r points along right singular vector associated with σmax • Best error at frequency ω ||e||2 = σmin[S(jω)] Attained when r points along right singular vector associated with σmin • In general σmin[S(jω)] ≤ ||e||2 ≤ σmax[S(jω)]



Command following (III) • Objective: express prescription for good

command following in terms of loop TFM L(s) = G(s)K(s) • Singular value facts σmax[A-1] =

[ ]min

1Aσ

σmin[A] - 1 ≤ σmin[Ι + A] ≤ σmin[A] + 1

• Recall: Need σmax[S(jω)] << 1 ;ω ∈ Ωr But S(jω) = [Ι + L(jω)]-1

⇒ σmax[S(jω)] =

1σ min I +L(jω )⎡⎣ ⎤⎦

<< 1

⇒ σmin[Ι + L(jω)] >> 1 ;ω ∈ Ωr But σmin[Ι + L(jω)] < σmin[L(jω)] + 1 ⇒ Need σmin[L(jω)] >> 1 ;ω ∈ Ωr • For good command following make σmin[G(s)K(s)] >> 1 for all ω ∈ Ωr



Disturbance rejection (I)

• Sinusoidal disturbance yields sinusoidal error d(t) = dejωt ⇒ e(t) = eejωt • Relation e = S(jω)d ||e||2 ≤ σmax[S(jω)]||d||2 • Ωd range of frequencies where disturbance

inputs has energy • Prescription for good disturbance rejection make σmax[S(jω)] << 1 for all ω ∈ Ωd or make σmin[G(s)K(s)] >> 1 for all ω ∈ Ωd



Disturbance rejection (II) • Interpretation for unit disturbance sinusoid d(t) = dejωt ; ||d||2 = 1 • Worst error at frequency ω ||e||2 = σmax[S(jω)] Attained when d points along right singular vector associated with σmax • Best error at frequency ω ||e||2 = σmin[S(jω)] Attained when d points along right singular vector associated with σmin • In general σmin[S(jω)] ≤ ||e||2 ≤ σmax[S(jω)]



Quantitative relations • Loop TFM: L(jω) = G(jω)K(jω) • Sensitivity TFM: S(jω) = [Ι + L(jω)]-1 • Closed-loop TFM: T(jω) = [Ι + L(jω)]-1 L(jω) • Ωp = Ωr ∪ Ωd • Key relations Let 0 < δ << 1 If σmax[S(jω)] ≤ δ ≤ 1 ; all ω ∈ Ωp Then 1 <<

�

1−δδ

≤ σmin[L(jω)] ; all ω ∈ Ωp and 1 - δ ≤ σmin[T(jω)] ≤ σmax[T(jω)] ≤ 1 + δ ; all ω ∈ Ωp ⇒ T(jω) ≈ Ι - Proofs: not in this course



Comment • Good command following and good

disturbance rejection served by similar requirements

ω ∈ Ωp = Ωr ∪ Ωd • Large loop gain σmin[L(jω)] >> 1 • Small sensitivity σmax[S(jω)] << 1 • Flat closed-loop response σmin[T(jω)] ≈ σmax[T(jω)] ≈ 1



Insensitivity to sensor noise

• Sinusoidal noise yields sinusoidal error n(t) = nejωt ⇒ e(t) = eejωt • Relation e = T(jω)n ||e||2 ≤ σmax[T(jω)]||n||2 • Ωn range of frequencies where noise has

significant energy • Prescription for good sensor noise rejection

make σmax[T(jω)]<<1 for all ω ∈ Ωn



Conflict with performance • Let 0 < γ << 1 • Suppose that σmax[T(jω)] ≤ γ for all ω ∈ Ωn Then

σmin[L(jω)] ≤ σmax[L(jω)] ≤ γ1-γ

≈ γ ∀ ω ∈ Ωn

⇒ low loop gain for all ω ∈ Ωn and 1 ≈1 - γ ≤ σmin[S(jω)] ≤ σmax[S(jω)] ⇒ large sensitivity for all ω ∈ Ωn • Bad command following and disturbance

rejection in frequency range ω ∈ Ωn • Consequence of constraint S(s) + T(s) = Ι



Design implications • Need wide frequency separation between sets

Ωp = Ωr ∪ Ωd and Ωn • Cannot do good command following and

disturbance rejection with noisy sensors that make low frequency errors (drift, bias, etc.)

• Stability-robustness to unmodelled high-

frequency dynamics, far-away nonminimum phase zeros, and neglected small time-delays impact design in the same way as region Ωn

Directional Information in Singular Value Plots



Introduction • Plots of min and max singular values vs.

Frequency provide valuable insight into frequency domain properties of mimo systems

• Singular value decomposition provides

directional information left singular vectors right singular vectors • Need to understand and exploit directional

information



SVD and linear equations • y = Ax x A y • SVD A = U Σ VH ⇒ y = U Σ VH x • Suppose : x = vi (right singular vector) ⇒ y = U Σ VH vi Note (since vH

i vj = δij)

VHvi =

0:1:0

⎛ ⎞⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎝ ⎠

(1 in row i) ; ΣVH vi =

0:

:0

iσ

⎛ ⎞⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎜ ⎟⎝ ⎠

(σi in row i)

then yi = σi ui (ui left singular vector) • Visualization



SVD directional information (I) • Maximum singular value σ1(A) = σmax(A) - Associated max right singular vector vmax = v1 ; ||vmax||2 = 1 - Associated max left singular vector umax = u1 ; ||umax||2 = 1 • Max amplification direction Let y = Ax If x = vmax Then y = σmaxumax ||y||2 = σmax (A) ;max amplification • Visualization

Vmax = x Umax

y



SVD directional information (II) • Minimum singular value: Rank(A)= m σm(A) = σmin(A) - Associated min right singular vector vmin = vm ; ||vmin||2 = 1 - Associated min left singular vector umin = um ; ||umin||2 = 1 • Min amplification direction Let y = Ax If x = vmin Then y = σminumin ||y||2 = σmin (A) ;min amplification • Visualization

Vmin = x Umin

y



Utilizing SVD directional information

• System:

y(s) = G(s)u(s) ;G(s) m x m matrix • Pick ω, calculate G(jω), do SVD G(jω) = U(jω) Σ(jω) VH(jω) • Maximum direction analysis - Find σmax(ω), vmax(ω) , umax(ω)

- Write [vmax(ω)]i = |ai|ejψi

- Write [umax(ω)]i = |bi|ejφi - Apply input ui(t) = |ai| sin(ωt + ψi) with u(t) = (u1(t); …;ui(t);…um(t)) then yi(t) = σmax|bi| sin(ωt + φi) • Minimum directional analysis similar • Complex directions of right and left singular

vectors correspond to sinusoidal vectors with relative phase-shift among their elements



DC gain matrix analysis • SVD-based direction analysis easiest at DC

because plant is real (ω = 0) • Plant model (strictly stable) dx(t)/dt = Ax(t) + Bu(t) y(t) = Cx(t) G(s) = C(sΙ - A)-1B • Plant DC gain matrix s = jω = 0 G(0) = - CA-1B • At steady-state u(t) = u = real constant vector ⇒ y(t→∞) = y = real constant vector y = G(0)u or y = - CA-1Bu



SVD at DC • Steady-state analysis y = G(0)u G(0) = - CA-1B = real • SVD at G(0) G(0) = UΣVH (U, Σ, V = real) • Max amplification direction If u = vmax ; ||u||2 = 1 Then y = σmaxumax • Min amplification direction If u = vmin ; ||u||2 = 1 Then y = σminumin • Above provides valuable insight upon MIMO

plant characteristics at DC

singular value decomposition analysis introduction€¦ · feedback performance specifications in...

Documents