lec 04 linear algebra

Linear Algebra topicsL. Lanari

Control Systems

Wednesday, October 8, 2014

Lanari: CS - Linear Algebra 2

Matrices

M =

m11 m12 m1qm21 m22 m2q...

... ...

mp1 mp,q1 mpq

M = {mij : i = 1, . . . , p & j = 1, . . . , q}

MT =mij : m

ij = mji

1 1

v : n 1vT : 1 n

M : p p

M : p q (p = q)

transpose

vector (column)

row vector

scalar

square matrix

rectangular matrix

Terminologyand notation



Matrices

m11 m12 m1n0 m22

. . . m2n

0. . .

. . ....

0 0 mnn

M1 0 00 M2 00 0 M3

M11 M120 M22

M = diag{Mi}, i = 1, . . . , k

m1 0 0

0 m2. . . 0

0. . .

. . . 00 0 mn

M = diag{mi}, i = 1, . . . , pdiagonal matrix

block diagonal matrix

upper triangular matrix

upper block triangular matrix

lower triangular matrix

strictly lower triangular matrix

lower block triangular matrix

strictly lower block triangular matrix



Matrices

determinant of a diagonal matrix = product of the diagonal terms

det

M11 M120 M22

= det(M11) det(M22)

det

M1 00 M2

= det(M1) det(M2) special case

special case

useful for the eigenvalue computation

a11 a1n...

. . ....

an1 ann

1 0...

. . ....

0 n

=

1 0...

. . ....

0 n

a11 a1n...

. . ....

an1 ann

a111 a1nn

.... . .

...an11 annn

=

a111 a1n1

.... . .

...an1n annn

gives



Matrices

det(M) = 0

A1

1= A

(AB)1 = B1A1

M = diag{mi} = M1 = diag{1

mi}

non-singular(invertible)square matrix

A1A = AA1 = I

A0 = I

detA= det

AT



generic transformationu

AuA

u

u

Au

Auor

particular directionssuch that Au is parallel to u

Au = u

scalar(scaling factor)

Eigenvalues & Eigenvectors



Aui = iui (iI A)ui = 0

det(iI A) = 0

pA() = det(I A) = n + an1n1 + an2n2 + + a1+ a0

i

pA() = det(I A) = 0

eigenvalue

for non-trivial solution

characteristic polynomial (order n = dim(A))

eigenvalue solution of


(scaling)

i

eigenvector ui

ui = 0



pA() = det(I A) = n + an1n1 + an2n2 + + a1+ a0

ii R

i C i

ma(i)

(i,i )

generic eigenvalue

polynomial with real coefficients.

algebraicmultiplicity


i Ri C

ui

uii

real components

ui complex components

pA() = 0

also is a solution pairs

therefore

if then

ieigenvalue pA() = 0solution of with multiplicity

The set of the n solutions of is defined as the spectrum of A: (A)




special cases

m11 m12 m1n0 m22

. . . m2n

0. . .

. . ....

0 0 mnn

m1 0 0

0 m2. . . 0

0. . .

. . . 00 0 mn

diagonalmatrix

triangularmatrix

eigenvalues = {mi}

eigenvalues = {mii}

elements on themain diagonal



A TAT1

Aui = iui uivTi A = v

Ti i = iv

Ti vTi

A(ui) = Aui = iui = i(ui)


det(T ) = 0

Tsame eigenvalues as A

right eigenvector

left eigenvector

eigenvalues are invariant under similarity transformations

eigenvector associated to i is not unique

all belong to the same linear subspace

(proof)

vTi uj = ijwith

similarity transformations




i eigenvalue one or more linearly independent eigenvectors

A1 =

1 00 1

, u11 =

10

, u12 =

01

A2 =

1 0 1

, with = 0, only u1 =

10

ma(i) > 1

pA() = ( 1)2

Vi = {u Rn|Au = iu}

Linear subspace (eigenspace) associated to an eigenvalue i

dim(Vi) = mg(i)

mg(i) = dim [Ker(A iI)] = n rank(A iI)

geometric multiplicity

with

A1 and A2 same eigenvalues with same m.a.



1 mg(i) ma(i) n


A1 =

1 00 1

, (A1 1I) =

0 00 0

mg(1) = 2 = ma(1)

A2 =

1 0 1

, (A2 1I) =

0 0 0

mg(1) = 1 < ma(1)

useful property

45

u1

u2

u3

P =

0.5 0 0.50 1 00.5 0 0.5

u1 =

010

u2 =

101

u3 =

101

1 = 2 = 1

3 = 0

projection matrix

ma(1) = 2 = mg(1)



Diagonalization

Def. An (n x n) matrix A is said to be diagonalizable if there exists an invertible (n x n) matrix T such that TAT -1 is a diagonal matrix

Th. An (n x n) matrix A is diagonalizable if and only if it has n independent eigenvectors

Since the eigenvalues are invariant under similarity transformations

if A diagonalizable

TAT1 = = diag{i}, i = 1, . . . , n

eigenvalues of A



Diagonalization

Aui = iui, i = 1, . . . , n

Au1 u2 un

=u1 u2 un

1 0 00 2 0

. . .0 0 n

We need to find T

i ui n linearly independent(by hyp.)

non-singular

in matrix form

AU = U

U =u1 u2 un



AU = U

AT1 = T1 A = T1T = TAT1

Diagonalization

Distinct eigenvalues (real and/or complex) A diagonalizable==

T1 = Ubeing non-singular, we can define T such thatU

therefore the diagonalizing similarity transformation is T s.t.

T1 = U =u1 u2 un

A: (n x n) is diagonalizable if and only if

mg(i) = ma(i) for every i



Diagonalization

Complex eigenvalues?i = i + ji

ui = uai + jubi

eigenvalue

eigenvector

diagonalization

or real block 2 x 2

complexelements

realelements

(i,i )

T1 =ui ui

Di = TAT1 =

i 00 i

T1 =uai ubi

Mi = TAT1 =

i ii i

2 choices

real system representation for complex eigenvalues



r = diag{1, . . . ,r}

Diagonalization

Simultaneous presence of real and complex eigenvalues

If A diagonalizable, there exists a non-singular matrix R such that

real eigenvalues

complex eigenvalues

RAR1 = diag {r,Mr+1,Mr+3, . . . ,Mq1}

Mi = TAT1 =

i ii i



Spectral decomposition or Eigendecomposition

A = U U1

U1 =

vT1vT2...vTn

U1U = I vTi uj = ij , i, j = 1, . . . , n

A =n

i=1

i ui vTi

Hyp: A diagonalizable

U =u1 u2 un

columns (linearly independent)

rows

A = U U1 =1u1 2u2 nun

vT1vT2...vTn

spectral form of A



A =n

i=1

i ui vTi

Spectral decomposition

columnrow

is the projection matrix on the invariant subspace generated by ui

(n x n) Pi = uivTi

A =

2 10 1

u1 =

10

u2 =

11

vT1 =1 1

vT2 =

0 1

P1 =

1 10 0

P2 =

0 10 1

1 = 2

2 = 1

u1

u2

P1 v

P2 v

v

v =

21



Not diagonalizable case

mg(i) < ma(i)

mg(i) nki

Jk =

i 1 0 00 i 1 0...

. . .. . .

. . ....

0 0 i 10 0 i

Rnknk

blocks of dimension

Jordanblock

of dimension nk

Null space has dimension 1Jk iI

1 mg(i) ma(i) nSince in general

if A not diagonalizable then

the knowledge of this dimension is out of scope



Not diagonalizable case

Generalized eigenvector chain of nk generalized eigenvectors

Jordan canonical form (block diagonal)

ma(i) = n =p

i=1

nk

mg(i) = p

example: unique eigenvalue i of matrix A (n x n)

TAT1 = J =

J1

. . .Jp

Jk Rnknk

T :

Jordan block of dim nkwith



Special cases

if ma(i) = 1 then mg(i) = 1

if mg(i) = 1 then only one Jordan block of dimension ma(i)

then only one Jordan block of dimension ma(i)

1 10 1

1 = 1 ma(1) = 2 mg(1) = 1

From the rank-nullity theorem

rank(A - iI)nullity(A - iI)n

if i unique eigenvalue of matrix A and if rank(iI - A) = ma(i) - 1

dim (Rn) = dim (Ker(A iI)) + dim (Im(A iI))

A iI : Rn Rn



Summary

A diagonalizable

A not diagonalizable

= diag{i} T s.t. TAT1 =

mg(i) = ma(i) for all i

T s.t.TAT1 = diag{Jk}

A =n

i=1

i ui vTi

r = diag{1, . . . ,r}

Mi =

i ii i

for real & complex i

alternative choice for complex i

Jk =

i 1 0...

. . .. . .

...0 i 10 0 i

Rnknk

Jordan blocks

mg(i) < ma(i)

spectral form

block diagonal


lec 04 linear algebra

Documents

linear algebra topicsl

linear subspace eigenspace

linear subspaceproofvti

n solutions

diagonal termsdetm11

tsame eigenvalues

eigenvalue idimvi

m11 m12 m1qm21 m22 m2q