5.1Orthogonality

A set of vectors is called an orthogonal set if all pairs of distinct vectors in the set are orthogonal.

An orthonormal set is an orthogonal set of unit vectors.

An orthogonal (orthonormal) basis for a subspace W of Rn

is a basis for W that is an orthogonal (orthonormal) set.

An orthogonal matrix is a square matrix whose columns form an orthonormal set.

Definitions

1) Is the following set of vectors orthogonal? orthonormal?

2) Find an orthogonal basis and an orthonormal basis for the subspace W of Rn

},...,,{ b) 2

11

,142

,21

3 a) 21 neee

Examples

02: W

zyx

zyx

All vectors in an orthogonal set are linearly independent.

Let {v1, v2,…, vk } be an orthogonal basis for a subspace W of Rn

and w be any vector in W. Then the unique

scalars c1 ,c2 , …, ck such that w = c1v1 + c2v2 + …+ ckvk

are given by

Theorems

kivvvw

ii

ii ,...,1for c

Proof: To find ci we take the dot product with vi w vi = (c1v1 + c2v2 + …+ ckvk ) vi

4) Is the following matrix orthogonal?

If it is orthogonal, find its inverse and its transpose.

cossinsincos

010001100

B212141

123 A C

Examples3) The orthogonal basis for the subspace W in previous example is

Pick a vector in W and express it in terms of the vectorsin the basis.

11

1,

011

The following statements are equivalent for a matrix A :1) A is orthogonal 2) A-1 = AT

3) ||Av|| = ||v|| for every v in Rn

4) Av1∙ Av2 = v1∙ v2 for every v1 ,v2 in Rn

Theorems on Orthogonal Matrix

Let A be an orthogonal matrix. Then1) its rows form an orthonormal set. 2) A-1 is also orthogonal. 3) |det(A)| = 14) |λ| = 1 where λ is an eigenvalue of A5) If A and B are orthogonal matrices, then so is AB

5.2Orthogonal Complements

and Orthogonal Projections

Recall: A normal vector n to a plane is orthogonal to every vector in that plane. If the plane passes through the origin, then it is a subspace W of R3 .

Also, span(n) is also a subspace of R3 Note that every vector in span(n) is orthogonal to

every vector in subspace W . Then span(n) is called orthogonal complement of W.

A vector v is said to be orthogonal to a subspace W of Rn if it is orthogonal to all vectors in W.

The set of all vectors that are orthogonal to W is called the orthogonal complement of W, denoted W ┴ . That is

Orthogonal Complements

W} 0 :R{W wwvv n

Definition:

http://www.math.tamu.edu/~yvorobet/MATH304-2011C/Lect3-02web.pdf

W perp

1) Find the orthogonal complements for W of R3 .

02: c)

110

and 011

vectorsby) spanned (subspace direction withplane b)

321

span a)

zyxzyx

W

W

W

Example

Let W be a subspace of Rn .

1) W ┴ is a subspace of Rn .2) (W ┴)┴ = W3) W ∩ W ┴ = {0}4) If W = span(w1,w2,…,wk), then v is in W ┴ iff v∙wi = 0

for all i =1,…,k.

Theorems

Let A be an m x n matrix. Then(row(A))┴ = null(A) and (col(A))┴ = null(AT)

Proof?

2) Use previous theorem to find the orthogonal complements for W of R3

.

1 0a) plane with direction (subspace spanned by) vectors 1 and 1

0 1

3 12 2

b) subspace spanned by vectors , an0 21 0

4 1

W

W

32

d 62

5

Example

1 2v

v

u v u vproj v vv vv

perp

2 1

w u

w = u u - w

Let u and v be nonzero vectors. w1 is called the vector component of u along v

(or projection of u onto v), and is denoted by projvu w2 is called the vector component of u orthogonal to v

w2 w1

u

v

Orthogonal Projections

Let W be a subspace of Rn with an orthogonal basis {u1, u2,…, uk }, the orthogonal projection of v onto W is defined as:

projW v = proju1 v + proju2 v + … + projuk v

The component of v orthogonal to W is the vectorperpW v = v – projw v

Orthogonal Projections

Let W be a subspace of Rn and v

be any vector in Rn .

Then there are unique vectors w1 in W and w2 in W ┴

such that v = w1 + w2 .

3) Find the orthogonal projection of v = [ 1, -1, 2 ] onto W and the component of v orthogonal to W.

1a) span 2

3

1 -1b) subspace spanned by 1 and 1

0 1

c) : 2 0

W

W

xW y x y z

z

Examples

5.3The Gram-Schmidt Process

And the QR Factorization

Goal: To construct an orthogonal (orthonormal) basis for any subspace of Rn

.We start with any basis {x1, x2,…, xk }, and “orthogonalize” each vector vi in the basis one at a time by finding the component of vi orthogonal to W = span(x1, x2,…, xi-1 ).

The Gram-Schmidt Process

Let {x1, x2,…, xk } be a basis for a subspace W. Then choose the following vectors:

v1 = x1,v2 = x2 – projv1 x2

v3 = x3 – projv1 x3 – projv2 x3

… and so on Then {v1, v2,…, vk } is orthogonal basis for W . We can normalize each vector in the basis to form an

orthonormal basis.

1) Use the following basis to find an orthonormal basis for R2

2) Find an orthogonal basis for R3 that contains the vector

,21

,13

Examples

121

101

, 1

11

,

Note: Since Q is orthogonal, Q-1 = QT and we have R = QT A

The QR Factorization

If A is an m x n matrix with linearly independent columns, then A can be factored as A = QR where R is an invertible upper triangular matrix and Q is an m x n orthogonal matrix. In fact columns of Q form orthonormal basis for Rn

which can be constructed from columns of A by using Gram-Schmidt process.

3) Find a QR factorization for the following matrices.

11-10121-1-1

A

2113

A

Examples

5.4Orthogonal Diagonalization

of Symmetric Matrices

1) Diagonalize the matrix.

6223

A

Example

Recall: A square matrix A is symmetric if AT = A. A square matrix A is diagonalizable if there exists a

matrix P and a diagonal matrix D such that P-1AP = D.

A square matrix A is orthogonally diagonalizable if there exists an orthogonal matrix Q and a diagonal matrix D such that Q-1AQ = D.

Note that Q-1 = QT

Orthogonal Diagonalization

Definition:

1. If A is orthogonally diagonalizable, then A is symmetric.2. If A is a real symmetric matrix, then the eigenvalues of A

are real.3. If A is a symmetric matrix, then any two eigenvectors

corresponding to distinct eigenvalues of A are orthogonal.

Theorems

A square matrix A is orthogonally diagonalizable if and only if it is symmetric.

2) Orthogonally diagonalize the matrix

and write A in terms of matrices Q and D.

011101110

A

Example

If A is orthogonally diagonalizable, and QTAQ = D then A can written as

where qi is the orthonormal column of Q, and λi is the corresponding eigenvalue.

A 222111T

nnnTT qq...qqqq

Theorem

This fact will help us construct the matrix A giveneigenvalues and orthogonal eigenvectors.

3) Find a 2 x 2 matrix that has eigenvalues 2 and 7, withcorresponding eigenvectors

21

v1

2 v 21

Example

5.5Applications

A quadratic form in x and y :

A quadratic form in x,y and z:

Quadratic Forms

2 2 2ax by cz dxy exz fyz

2 2ax by cxy 12

12

T a cc b

x x

1 12 2

1 12 21 12 2

T

a d ed b fe f c

x x

where x is the variable (column) matrix.

A quadratic form in n variables is a function f : Rn

R of the form:

where A is a symmetric n x n matrix and x is in Rn

Quadratic Forms

( ) Tf Ax x x

A is called the matrix associated with f.

2 2

2 2

( , ) 8

( , ) 2 5

z f x y x y xy

z f x y x y

The Principal Axes Theorem

Example: Find a change of variable that transforms theQuadratic into one with no cross-product terms.

Every quadratic form can be diagonalized. In fact,if A is a symmetric n x n matrix and if Q is an orthogonal matrix so that QTAQ = D then the change of variable x = Qy transforms the quadratic form into 2 2 2

1 1 2 2 A T Tn nD y y ... y x x y y

2 2

2 2

( , ) 8

( , ) 2 5

z f x y x y xy

z f x y x y