Matrix Algebra Basics

By Nittaya NoinanKanchanapisekwittayalai phechabun

M.4

Algebra

Matrix Algebra

• Matrix algebra is a means of expressing large numbers of calculations made upon ordered sets of numbers.

• Often referred to as Linear Algebra• Many equations would be completely intractable if

scalar mathematics had to be used. It is also important to note that the scalar algebra is under there somewhere.

Matrix (Basic Definitions)

ij

knk

n

n

A

aa

aa

aa

,,

,,

,,

1

221

111

A

An m × n matrix A is a rectangular array of numbers with m rows and n columns. (Rows are horizontal and columns are vertical.) The numbers m and n are the dimensions of A. The numbers in the matrix are called its entries. The entry in row i and column j is called aij .

4

Matrix

A

a11 ,, a1n

a21 ,, a2n

am1 ,, amn

Aij

A matrix is any doubly subscripted array of elements arranged in rows and columns.

Definitions - Matrix

• A matrix is a set of rows and columns of numbers

• Denoted with a bold Capital letter• All matrices (and vectors) have an order -

that is the number of rows x the number of columns.

• Thus A =

654

321

32654

321

x

Definitions - scalar

• scalar - a number– denoted with regular type as is scalar algebra– [1] or [a]

Definitions - vector

• vector - a single row or column of numbers– denoted with bold small letters– row vector a =

– column vector x =

54321

5

4

3

2

1

x

x

x

x

x

Row Vector

[1 x n] matrix

 

jn aaaaA ,, 2 1

Column Vector

i

m

a

a

a

a

A 2

1

[m x 1] matrix

Special matrices

• There are a number of special matrices– Square– Diagonal– Symmetric– Null– Identity

Square matrix• A square matrix is just what it sounds like, an nxn matrix

• Square matrices are quite useful for describing the properties or interrelationships among a set of things – like a data set.

44434241

34333231

24232221

14131211

aaaa

aaaa

aaaa

aaaa

Square Matrix

B

5 4 7

3 6 1

2 1 3

Same number of rows and columns

Diagonal Matrices

– A diagonal matrix is a square matrix that has values on the diagonal with all off-diagonal entities being zero.

44

33

22

11

000

000

000

000

a

a

a

a

Symmetric Matrix• All of the elements in the upper right portion of

the matrix are identical to those in the lower left.

• For example, the correlation matrix

Identity Matrix

• The identity matrix I is a diagonal matrix where the diagonal elements all equal one. It is used in a fashion analogous to multiplying through by "1" in scalar math.

1000

0100

0010

0001

Null Matrix

• A square matrix where all elements equal zero.

• Not usually ‘used’ so much as sometimes the result of a calculation. – Analogous to “a+b=0”

0000

0000

0000

0000

Types of Matrix

• Identity matrices - I

• Diagonal

1001

1000010000100001

• Symmetric

– Diagonal matrices are (of course) symmetric– Identity matrices are (of course) diagonal

4000010000200001

fecedbcba

The Identity

Identity Matrix

I

1 0 0 0

0 1 0 0

0 0 1 0

0 0 0 1

Square matrix with ones on the diagonal and zeros elsewhere.

Operations with Matrices (Transpose)

TransposeThe transpose, AT , of a matrix A is the matrix obtained from A by writing its rows as columns. If A is an k×n matrix and B = AT then B is the n×k matrix with bij = aji. If AT=A, then A is symmetric.

 

Example:

,CD(CD)

rA(rA)A,)(A

,BAB)(A,BAB)(A

aa

aa

aa

aaa

aaa

TTT

TTTT

TTTTTT

T

Then, matrix.n man be D andmatrix mk a be CLet

scalar. a isr andn k are B andA where

: verifyeasy toit It

2313

2212

2111

232221

131211

The Transpose of a Matrix At

• Taking the transpose is an operation that creates a new matrix based on an existing one.

• The rows of A = the columns of At

• Hold upper left and lower right corners and rotate 180 degrees.

Transpose Matrix

nmmm

n

n

t

aaa

aaa

aaa

A

,,

,, ,

,, ,

2,1

22212

12111

Rows become columns and columns become rows

,,

,,

,,

21

2,2221

1,1211

mnmm

n

n

aaa

aaa

aaa

A

Example of a transpose

654

321,

63

52

41tAA

The Transpose of a Matrix At

• If A = At, then A is symmetric (i.e. correlation matrix) • If A AT = A then At is idempotent

– (and A' = A)

• The transpose of a sum = sum of transposes• The transpose of a product = the product of the

transposes in reverse order

tttt CBACBA )(

Transpose Matrix

Ex 1

41

03

21

A

402

131TA

(32) (23)

Transpose Matrix

Ex 2

(34)

(43)

2572

1310

3414

B

2

5

1

3

3

4711

204TB

Matrix Equality

• Two matrices are equal iff (if and only if) all of their elements are identical

• Note: your data set is a matrix.

Matrix Equality

Ex1. Assume A = B find x , y ,z

41

03

21

A

4

3

2

,

z

y

x

B

Solution. If A = B that mean x = 1y = 0z = -1

Matrix Equality

Ex2. Assume C = D find x , y ,z

Solution. If C = D that mean y = 2 , z = 2 and x + y = 4 thus x + 2 = 4then x = 2

2572

1310

341yx

C

zy

D

57

1310

3414

,

Matrix Operations

• Addition and Subtraction• Multiplication• Transposition• Inversion

Matrix Addition

A new matrix C may be defined as the additive combination of matrices A and B where: C = A + B is defined by: 

Cij Aij Bij

Note: all three matrices are of the same dimension

Addition

A a11 a12

a21 a22

B b11 b12

b21 b22

C a11 b11 a12 b12

a21b21 a 22 b22

If

and

then

If A and B are both m × n matrices then the sum of A and B, denoted A + B, is a matrix obtained by adding corresponding elements of A and B.

310

221A

412

403B

2BA

If A and B are both m × n matrices then the sum of A and B, denoted A + B, is a matrix obtained by adding corresponding elements of A and B.

add these

310

221A

412

403B

22BA

add these

310

221A

412

403B

622BA

add these

310

221A

412

403B

2

622BA

add these

310

221A

412

403B

02

622BA

add these

310

221A

412

403B

102

622BA

add these

Matrix Addition Example

A B 3 4

5 6

1 2

3 4

4 6

8 10

C

ABBA

CBACBA )()(

Addition and Subtraction (cont.)

• Where

• Hence

129

107

85

64

64

64

65

43

21

323232

313131

222222

212121

121212

111111

cba

cba

cba

cba

cba

cba

Matrix Subtraction

C = A - BIs defined by

Cij Aij Bij

Note: all three matrices are of the same dimension

Subtraction

A a11 a12

a21 a22

B b11 b12

b21 b22

22222121

12121111

baba

babaC

If

and

then

Addition and Subtraction (cont.)

• Where

• Hence

01

21

43

64

64

64

65

43

21

323232

313131

222222

212121

121212

111111

cba

cba

cba

cba

cba

cba

Operations with Matrices (Scalar Multiple)

Scalar Multiple

If A is a matrix and r is a number (sometimes called a scalar in this context), then the scalar multiple, rA, is obtained by multiplying every entry in A by r. In symbols, (rA)ij = raij . 

Example:

41

0 14 12

28 6

0 7 6

14 32

Scalar Multiplication

• To multiply a scalar times a matrix, simply multiply each element of the matrix by the scalar quantity

2221

1211

2221

1211

22

222

aa

aa

aa

aa

If A is an m × n matrix and s is a scalar, then we let kA denote the matrix obtained by multiplying every element of A by k. This procedure is called scalar multiplication.

k hA kh A

k h A kA hA

k A B kA kB

310

221A

930

663

331303

2323133A

PROPERTIES OF SCALAR MULTIPLICATION

The m × n zero matrix, denoted 0, is the m × n matrix whose elements are all zeros.

00

0)(

0

A

AA

AA

00

00 000

2 × 21 × 3

Operations with Matrices (Product)

ProductIf A has dimensions k × m and B has dimensions m × n, then the productAB is defined, and has dimensions k × n. The entry (AB)ij is obtainedby multiplying row i of A by column j of B, which is done by multiplyingcorresponding entries together and then adding the results i.e.,

B.IB B,matrix mnany

for andA AI A,matrix n many for

100

01 0

00 1

Imatrix Identity

.

Example

....)...( 22112

1

21

nn

mjimjiji

mj

j

j

imii

fDeBfCeA

dDcBdCcA

bDaBbCaA

DC

BA

fe

dc

ba

bababa

b

b

b

aaa

Matrix Multiplication (cont.)

• To multiply a matrix times a matrix, we write • A times B as AB

• This is pre-multiplying B by A, or post-multiplying A by B.

Matrix Multiplication (cont.)

• In order to multiply matrices, they must be conformable (the number of columns in A must equal the number of rows in B.)

• an (mxn) x (nxp) = (mxp)• an (mxn) x (pxn) = cannot be done• a (1xn) x (nx1) = a scalar (1x1)

Matrix Multiplication (cont.)

• The general rule for matrix multiplication is:

PjandMiwherebacN

kkjikij ,...,2,1,,...,2,1

1

Matrix Multiplication

Matrices A and B have these dimensions:

[r x c] and [s x d]

Matrix Multiplication

Matrices A and B can be multiplied if:

[r x c] and [s x d]

c = s

Matrix Multiplication

The resulting matrix will have the dimensions:

[r x c] and [s x d]

r x d

Computation: A x B = C

A a11 a12

a21 a22

B b11 b12 b13

b21 b22 b23

232213212222122121221121

2312131122121211 21121111

babababababa

babababababaC

[2 x 2]

[2 x 3]

[2 x 3]

Computation: A x B = C

A

2 3

1 1

1 0

and B

1 1 1

1 0 2

[3 x 2] [2 x 3]A and B can be multiplied

1 1 1

3 1 2

8 2 5

12*01*1 10*01*1 11*01*1

32*11*1 10*11*1 21*11*1

82*31*2 20*31*2 51*31*2

C

[3 x 3]

Computation: A x B = C

1 1 1

3 1 2

8 2 5

12*01*1 10*01*1 11*01*1

32*11*1 10*11*1 21*11*1

82*31*2 20*31*2 51*31*2

C

A

2 3

1 1

1 0

and B

1 1 1

1 0 2

[3 x 2] [2 x 3]

[3 x 3]

Result is 3 x 3

The multiplication of matrices is easier shown than put into words. You multiply the rows of the first matrix with the columns of the second adding products

140

123A

13

31

42

B

Find AB

First we multiply across the first row and down the first column adding products. We put the answer in the first row, first column of the answer.

23 1223 5311223

140

123A

13

31

42

B

Find AB

We multiplied across first row and down first column so we put the answer in the first row, first column.

5AB

Now we multiply across the first row and down the second column and we’ll put the answer in the first row, second column.

43 3243 7113243

75AB

Now we multiply across the second row and down the first column and we’ll put the answer in the second row, first column.

20 1420 1311420

1

75AB

Now we multiply across the second row and down the second column and we’ll put the answer in the second row, second column.

40 3440 11113440

111

75AB

Notice the sizes of A and B and the size of the product AB.

To multiply matrices A and B look at their dimensions

pnnm MUST BE SAME

SIZE OF PRODUCT

If the number of columns of A does not equal the number of rows of B then the

product AB is undefined.

6

BA

126

BA

2126

BA

3

2126

BA

143

2126

BA

4143

2126

BA

9

4143

2126

BA

109

4143

2126

BA

4109

4143

2126

BA

Now let’s look at the product BA.

13

31

42

B

140

123A

BAAB

2332 can multiply

size of

answer

across first row as we go down first column:

60432

across first row as we go down second column:

124422

across first row as we go down third column:

21412

across second row as we go down first column:

30331

across second row as we go down second column:

144321 41311

across third row as we go down first column:

90133

across third row as we go down second column:

104123

across third row as we go down third coluacross second row as we go down third column:

mn:

41113

Completely different than AB!

Commuter's Beware!

BCACCBA

ACABCBA

CABBCA

PROPERTIES OF MATRIX MULTIPLICATION

BAAB Is it possible for AB = BA ?,yes it is possible.

an n n matrix with ones on the main diagonal and zeros elsewhere

100

010

001

3I

What is AI?

What is IA?

322

510

212

A

100

010

001

3I

A

322

510

212

A

322

510

212

Multiplying a matrix by the identity gives

the matrix back again.

Matrix multiplication is not Commutative

• AB does not necessarily equal BA• (BA may even be an impossible operation)

Yet matrix multiplication is Associative

• A(BC) = (AB)C

Laws of Matrix Algebra

• The matrix addition, subtraction, scalar multiplication and matrix multiplication, have the following properties.

BC. AC B)C AC, (A AB C) A(B

A B B A

A(BC). C, (AB)C B) (A C) (B A

:Laws veDistributi

:Additionfor Law eCommutativ

:Laws eAssociativ

An example - cont

• Since the matrix product is a scalar found by summing the elements of the vector squared.

Determinants• Determinant is a scalar

– Defined for a square matrix– Is the sum of selected products of the elements of the matrix, each product

being multiplied by +1 or -1

11 12 1

21 22 2

1 1

1 2

det( ) ( 1) ( 1)

n

n nn i j i j

ij ij ij ijj i

n n nn

a a a

a a aA a M a M

a a a

• Mij=det(Aij), Aij is the (n-1)×(n-1) submatrix obtained by deleting row i and column j from A.

Determinants

• The determinant of a 3 ×3 matrix is

11 12 1322 23 21 23 21 221 1 1 2 1 3

21 22 23 11 12 1331 3232 33 31 33

31 32 33

( 1) ( 1) ( 1)

a a aa a a a a a

a a a a a aa aa a a a

a a a

Example

1 1 1 2 1 3

1 2 35 6 4 6 4 5

4 5 6 1( 1) 2( 1) 3( 1)8 10 7 10 7 8

7 8 10

50 48 2(40 42) 3(32 35) 3

bcaddc

baA )det(• The determinant of a 2 ×2 matrix A is

• In Matlab: det(A) = det(A)

The Determinant of a Matrix

• The determinant of a matrix A is denoted by |A|.

• Determinants exist only for square matrices.• They are a matrix characteristic, and they are

also difficult to compute

The Determinant for a 2x2 matrix

• If A =

• Then

• That one is easy

21122211 aaaaA

2221

1211

aa

aa

The Determinant for a 3x3 matrix • If A =

• Then

333231

232221

131211

aaa

aaa

aaa

312213322113332112312312322311332211 aaaaaaaaaaaaaaaaaaA

Determinants

• For 4 x 4 and up don't try. For those interested, expansion by minors and cofactors is the preferred method.

• (However the spaghetti method works well! Simply duplicate all but the last column of the matrix next to the original and sum the products of the diagonals along the following pattern.)

Properties of Determinates

• Determinants have several mathematical properties which are useful in matrix manipulations.

– 1 |A|=|A'|.– 2. If a row of A = 0, then |A|= 0.– 3. If every value in a row is multiplied by k, then |A| =

k|A|.– 4. If two rows (or columns) are interchanged the sign,

but not value, of |A| changes.– 5. If two rows are identical, |A| = 0.

Properties of Determinates

– 6. |A| remains unchanged if each element of a row or each element multiplied by a constant, is added to any other row.

– 7. Det of product = product of Det's |AB| = |A| |B|

– 8. Det of a diagonal matrix = product of the diagonal elements

Matrix Division

We have seen that for 2x2 (“two by two”) matrices A and B then AB BA

To divide matrices we need to define what we mean by division!

With numbers or algebra we use b/a to solve ax=b. The equivalent in 2x2 matrices is to solve AX=B where A, B and X are 2x2 matrices.

Inverse Matrix

In numbers, the inverse of 3 is 1/3 = 3-1

In algebra, the inverse of a is 1/a = a-1

In matrices, the inverse of A is A-1

3-1 is defined so that 3 x 3-1 = 3-1 x 3 = 1a-1 is defined so that a x a-1 = a-1 x a = 1A-1 is defined so that A A-1 = A-1 A = I

However, for a square matrix A there is not always an inverse A-1

Inverse Matrix

In matrices, the inverse of A is A-1

A-1 is defined so that A A-1 = A-1 A = I

However, for a square matrix A there is not always an inverse A-1

If A-1 does not exist then the matrix is said to be singular

If A-1 does exist then the matrix is said to be non-singular

Inverse Matrix

In matrices, the inverse of A is A-1

A-1 is defined so that A A-1 = A-1 A = I

A square matrix A has an inverse if, and only if, A is non-singular.

If A-1 does exist the the solution to AX=B is

X = A-1 B

Inverse Matrix

A-1 is defined so that A A-1 = A-1 A = I

If A-1 does exist the the solution to AX=B is

AX = B

Pre-multiply by A-1 A-1AX = A-1B

But A-1A = I so IX = A-1B X = A-1B

Inverse Matrix

AX = B

Pre-multiply by A-1 A-1AX = A-1B

But A-1A = I so IX = A-1B X = A-1B

If the inverse of A is A-1 then the inverse of A-1 is A. This is because if AC = I then CA = I, and also any matrix inverse is unique.

Inverse Matrix

If the inverse of A is A-1 then the inverse of A-1 is A. This is because if AC = I then CA = I, and also any matrix inverse is unique.

What is the inverse of

Then solve for u, v, w, x

30

12B

xw

vu1let B

20

13

6

11B

General Inverse Matrix

dc

baC

bcadD

ac

bd

Dxw

vu

where

1let 1C

1

0

0

1

dxcv

bxav

dwcu

bwau

a

c

cwbcad

Subtract

dawcau

cbcwacu

)(

:

0

Inverse of a Matrix• Definition. If A is a square matrix, i.e., A has dimensions n×n. Matrix A is

nonsingular or invertible if there exists a matrix B such that AB=BA=In. For example.

Common notation for the inverse of a matrix A is A-1

If A is an invertible matrix, then (AT)-1 = (A-1)T

The inverse matrix A-1 is unique when it exists. If A is invertible, A-1 is also invertible A is the inverse matrix of A-1. (A-1)-1=A.

• In Matlab: A-1 = inv(A)

• Matrix division:

A/B = AB-1

10

01

3

2

3

1

3

2

3

23

1

3

1

3

1

3

2

3

1

3

13

1

3

2

21

11

Calculation of Inversion using Determinants

Def: For any n×n matrix A, let Cij denote the (i,j)th cofactor of A, that is, (-1)i+j times the determinant of the submatrix obtained by deleting row i and column j form A, i.e., Cij = (-1)i+j Mij . The n×n matrix whose (i,j)th entry is Cji, the (j,i)th cofactor of A is called the adjoint of A and is written adj A.

thus-1

Thm: Let A be a nonsingular matrix. Then,

1A .

detadj A

A

Calculation of Inversion using Determinants

thus

Example: find the inverse of the matrix

Solve:

2 4 5

0 3 0

1 0 1

A

11 12 13

21 22 23

31 32 33

11 21 31

12 22 32

13 23 33

1

3 0 0 0 0 33, 0, 3,

0 1 1 1 1 0

4 5 2 5 2 44, 3, 4,

0 1 1 1 1 0

4 5 2 5 2 415, 0, 6,

3 0 0 0 0 3

det 9,

3 4 15

0 3 0 .

3 4 6

31

,9

C C C

C C C

C C C

A

C C C

adjA C C C

C C C

So A

4 15

0 3 0 .

3 4 6

Using Determinants to find the inverse of a matrix can be very complicated. Gaussian elimination is more efficient for high dimension matrix.

Calculation of Inversion using Gaussian Elimination

Elementary row operations:

o Interchange two rows of a matrixo Change a row by adding to it a multiple

of another rowo Multiply each element in a row by the

same nonzero number

• To calculate the inverse of matrix A, we apply the elementary row operations on the augmented matrix [A I] and reduce this matrix to the form of [I B]

• The right half of this augmented matrix B is the inverse of A

Calculation of inversion using Gaussian elimination

I is the identity matrix, and use Gaussian elimination to obtain a matrix of the form

The matrix

100 ,,

01 0 ,,

00 1 ,,

][

1

221

111

nnn

n

n

aa

aa

aa

IA

nnn

n

n

aa

aa

aa

,,

,,

,,

1

221

111

A

nnnn

n

n

bbb

bbb

bbb

100

01 0

00 1

21

22221

11211

nnnn

n

n

bbb

bbb

bbb

B

21

22221

11211

is then the matrix inverse of A

Example

The matrix

1 1 1 |1 0 0

[ | ] 12 2 3 | 0 1 0

3 4 1 | 0 0 1

A I

1 1 1

12 2 3

3 4 1

A

is then the matrix inverse of A

1 1 1 | 1 0 0

0 10 15 | 12 1 0

0 0 3.5 | 4.2 0.1 1

3 11 0 0 | 0.4

35 72 3

0 1 0 | 0.635 71 2

0 0 1 | 1.235 7

(ii)+(-12)×(i), (iii)+(-3) ×(i), (iii)+(ii) ×(1/10)

3 10.4

35 72 3

0.635 71 2

1.235 7

10

01

24

13?

Let A be an n n matrix. If there exists a matrix B such that AB = BA = I then we call this matrix the inverse of A and denote it A-1.

2

32

2

11

10

01

24

13

2

32

2

11

Can we find a matrix to multiply the first matrix by to get the identity?

If A has an inverse we say that A is nonsingular. If A-1 does not exist we say A is singular.

To find the inverse of a matrix we put the matrix A, a line and then the identity matrix. We then perform row operations on matrix A to turn it into the identity. We carry the row operations across and the right hand side will turn into the inverse.

To find the inverse of a matrix we put the matrix A, a line and then the identity matrix. We then perform row operations on matrix A to turn it into the identity. We carry the row operations across and the right hand side will turn into the inverse.

72

31A

1210

01312r1+r

2

1072

0131

1210

0131

r2

1210

3701r1 r2

72

31A

12

371A

Check this answer by multiplying. We should get the identity matrix if we’ve found the inverse.

10

011AA

Inversion

We can use A-1 to solve a system of equations

352

13

yx

yx

bxA

To see how, we can re-write a system of equations as matrices.

coefficient matrix

variable matrix

constant matrix

52

31

y

x

3

1

bx 1A

bx 11 AAA

bxA left multiply both sides by the inverse of A

This is just the identity

bx 1 AIbut the identity times a matrix just gives us back the matrix so we have:

This then gives us a formula for finding the variable matrix: Multiply A inverse by the constants.

352

13

yx

yx

52

31A find the

inverse

1052

0131

1210

0131-2r1+r2

1210

0131

-r2

1210

3501r1-3r2

1

4

3

1

12

351bAThis is the answer to the system

xy

The system of linear equations

Systems of Equations in Matrix Form11 1 12 2 13 3 1 1

21 1 22 2 23 3 2 2

1 1 2 2 3 3

n n

n n

k k k kn n k

a x a x a x a x b

a x a x a x a x b

a x a x a x a x b

can be rewritten as the matrix equation Ax=b, where

1 111 1

2 2

1

, , .n

k knn k

x ba a

x bA x b

a ax b

If an n×n matrix A is invertible, then it is nonsingular, and the unique solution to the system of linear equations Ax=b is x=A-1b.

Example: solve the linear system

1

-1

Matrix Inversion

4 1 2 x 4

5 2 1 ; X y ; b 4

1 0 3 z 3

6 -3 -31

A -14 10 66

-2 1 3

x 6 -3 -3 41

y -14 10 6 46

z -2 1 3 3

1 2; y 1 3; z 5 6

AX d

A

X A b

x

4 2 4

5 2 4

3 3

x y z

x y z

x z

b

Matrix Inversion

B 1B BB 1 I

Like a reciprocal in scalar math

Like the number one in scalar math

Linear System of Simultaneous Equations

1st Precinct : x1 x2 6

2nd Pr ecinct : 2x1 x2 9

First precinct: 6 arrests last week equally divided between felonies and misdemeanors.

Second precinct: 9 arrests - there were twice as many felonies as the first precinct.

Solution

9

6 *

1 2

1 1

2

1

x

x

3

3

2

1

x

x

1 2

1 1 Note:

Inverse ofis

1 2

1 1

9

6*

1 2

1 1 *

1 2

1 1*

1 2

1 1

2

1

x

x Premultiply both sides by inverse matrix

3

3 *

1 0

0 1

2

1

x

x A square matrix multiplied by its inverse results in the identity matrix.A 2x2 identity matrix multiplied by the 2x1 matrix results in the original 2x1 matrix.

aijxj bi or Ax bj1

n

x A 1Ax A 1b

n equations in n variables:

unknown values of x can be found using the inverse of matrix A such that

General Form

Good Luck