Gaussian Elimination2

45 �1 2 7

�2 6 9 0�7 5 �3 5

3

5Apply elementary row operations to the augmented matrix

2

64

5 �1 2 7

0 285

495

145

0 185 � 1

5745

3

75 (eqn 2)�✓�25

◆(eqn 1)

(eqn 3)�✓�75

◆(eqn 1)

2

64

5 �1 2 7

0 285

495

145

0 0 � 6510

655

3

75 (eqn 3)�

✓9

14

◆(eqn 2)

Solution : x1 = 3, x2 = 4, x3 = �2

PivotsMultipliers

1

Matrix inverse

3x = 7

We can solve the linear system

by multiplying both sides by the “inverse” of 3 :

(3�1)3x = (3�1)7

x =7

3

If a linear system is non-singular or invertible, we can write the solution in the same manner

Ax = b

x = A�1b

where A�1is the inverse of A.

solution is unique!

solution is unique!2

The identity matrix

2

44 �3 21 0 �52 7 �9

3

5

2

41 0 00 1 00 0 1

3

5 =

2

44 �3 21 0 �52 7 �9

3

5

The identity matrix is the “multiplicative identity”, or the “1” for square matrices.

2

41 0 00 1 00 0 1

3

5

2

44 �3 21 0 �52 7 �9

3

5 =

2

44 �3 21 0 �52 7 �9

3

5Ex.

It has the property that for any n⇥ n matrix A,

AI = IA = A

An n ⇥ n identity matrix I is a diagonal matrix with

all 1s on the diagonal.

3

Matrix inverse

Just as (3

�1)(3) = (3)(3

�1) = 1, a matrix A and its

inverse satisfy

A�1A = AA�1= I

where I is the identity matrix.

2

64

0 1 2

1 0 3

4 �3 8

3

75

2

64

� 92 7 � 3

2

�2 4 �132 �2 � 1

2

3

75 =

A A�1

2

64

1 0 0

0 1 0

0 0 1

3

75

I

2

64

� 92 7 � 3

2

�2 4 �132 �2 � 1

2

3

75

2

64

0 1 2

1 0 3

4 �3 8

3

75 =

AA�1

2

64

1 0 0

0 1 0

0 0 1

3

75

I

The identity times a vector also returns that vector.

4

Matrix inverse

Multiply both sides by A�1to get

A�1Ax = A�1b

Ix = A�1b

Since I times any vector or matrix is always that vector

or matrix, we have

x = A�1b

The solution is unique

If A is invertible, we could solve Ax = b as follows :

5

Example2

64

0 1 2

1 0 3

4 �3 8

3

75

2

64x1

x2

x3

3

75 =

2

64�1

2

11

3

75

Multiply both sides by A�1to get

Check!

2

64x1

x2

x3

3

75 =

2

64

� 92 7 � 3

2

�2 4 �132 �2 � 1

2

3

75

2

64�1

2

11

3

75 =

2

642

�1

0

3

75

A x b

A�1x b

This is the only solution to this system.

6

Some facts about inverses

The inverse exists if and only if elimination produces n (non-zero) pivots.

A matrix cannot have two di↵erent inverses. If you find one inverse, you have

found both the right and left inverse.

A matrix A commutes with its inverse : AA�1= A�1A = I

If A is invertible, then Ax = 0 can only have the zero solution x = A�10 = 0.

Suppose there is a non-zero vector x such that Ax = 0. Then A cannot have an

inverse. Suppose it did have an inverse B such that AB = BA = I. Then

x = B0 = 0

But we said that x is nonzero. Therefore, the inverse B cannot exist. Matrices

which don’t have inverses are called singular or non-invertible. Such matrices are

like the multiplicative ”0” of matrices. (Think of the scalar system 0x = b. We

cannot divide by 0 to solve for x.)

Only square matrices have inverses.

7

Two special cases

The inverse of a 2 ⇥ 2 matrix can easily be written

down.

a bc d

��1

=

1

ad� bc

d �b

�c a

�

The inverse of a diagonal matrix is given by

2

6664

d1 0

d2.

.

.

0 dn

3

7775

�1

=

2

6664

1d1

0

1d2

.

.

.

0

1dn

3

7775

the “determinant”

All off-diagonal entries are zero 8

The inverse of sums and products

Suppose both A and B are invertible. What can we say about the inverse of(A+B)? Or the inverse of AB?

We can’t say anything about the inverse of A + B. Suppose A is invertible. Let

B = �A. Then A+B = 0, which is not invertible.

If A and B are invertible, the inverse of the product AB is given by (AB)

�1=

B�1A�1.

(AB)(AB)

�1= ABB�1A�1

= AIA�1= I

We don’t need to check the other side, since the inverse is unique and so is both

a left and right inverse.

9

Gauss-Jordan Method

2

40 1 2 1 0 01 0 3 0 1 04 �3 8 0 0 1

3

5

(eqn 2)

(eqn 1)

2

41 0 3 0 1 00 1 2 1 0 04 �3 8 0 0 1

3

5

2

41 0 3 0 1 00 1 2 1 0 00 �3 �4 0 �4 1

3

5 (eqn 3)� (4)(eqn 1)

2

41 0 3 0 1 00 1 2 1 0 00 0 2 3 �4 1

3

5 (eqn 3)� (�3)(eqn 2)

A I

If we were solving a linear system, we could stop here at the upper triangular form. But to get the inverse, we continue. upper triangular

The Gauss-Jordon method gives us a way to compute the matrix inverse.

Carry out row operations on A and I simultaneously. The first step is a row

exchange

10

Gauss-Jordan Method

Divide the last row by 2 2

41 0 3 0 1 00 1 2 1 0 00 0 1 3

2 �2 12

3

5

Reverse the elimination process until A is reduced to the identity.

2

64

1 0 0 � 92 7 � 3

2

0 1 0 �2 4 �1

0 0 1 32 �2 1

2

3

75 (eqn 2)� (2)(eqn 3)

(eqn 1)� (3)(eqn 3)

I A�1

The augmented matrix [A I] is row-reduced to [I A�1].

The Gauss-Jordan Method is a method for finding the inverse of a matrix.

get 1s on the diagonal ✓1

2

◆(eqn 3)

11

Example

Find the inverse of A by two di↵erent methods. Verify

the inverse you found.

A =

1 3

2 7

�

Use the formula for the inverse of a 2x2 matrix

A�1 =1

(1)(7)� (3)(2)

7 �3

�2 1

�=

7 �3

�2 1

�

1 3 1 02 7 0 1

�!

1 3 1 00 1 �2 1

�!

1 0 7 �30 1 �2 1

�Use the Gauss-Jordan Method

12

Inverses of elimination matrices

E21

2

64

? ? ?

? ? ?

? ? ?

3

75

2

64

(row 1)

(row 2)� (`21)(row 1)

(row 3)

3

75 =

2

64

(row 1)

(row 2)

(row 3)

3

75

E�121

2

64

1 0 0

`21 1 0

0 0 1

3

75

2

64

(row 1)

(row 2)� (`21)(row 1)

(row 3)

3

75 =

2

64

(row 1)

(row 2)

(row 3)

3

75

(`21)(row 1) + (1) [(row 2)� (`21)(row 1)] = (row 2)

2

4X X XX X XX X X

3

5

2

4X X X0 X XX X X

3

5

2

64

1 0 0

�`21 1 0

0 0 1

3

75

2

64

(row 1)

(row 2)

(row 3)

3

75 =

2

64

(row 1)

(row 2)� (`21)(row 1)

(row 3)

3

75

Find

The inverse of the elimination matrix undoes the effect of elimination.

13

Inverses of elimination matrices

E31

(`31)(row 1) + (1) [(row 3)� (`31)(row 1)] = (row 3)

E�131

2

64

1 0 0

0 1 0

`31 0 1

3

75

2

664

(row 1)

(row 2)

(row 3)� (`31)(row 1)

3

775 =

2

64

(row 1)

(row 2)

(row 3)

3

75

2

64

1 0 0

0 1 0

�`31 0 1

3

75

2

64

(row 1)

(row 2)

(row 3)

3

75 =

2

64

(row 1)

(row 2)

(row 3)� (`31)(row 1)

3

75

2

4X X X0 X XX X X

3

5

2

4X X X0 X X0 X X

3

5

14

Inverses of elimination matrices

2

64

1 0 0

0 1 0

0 �`32 1

3

75

2

64

(row 1)

(row 2)

(row 3)

3

75 =

2

64

(row 1)

(row 2)

(row 3)� (`32)(row 2)

3

75

2

4X X X0 X X0 X X

3

52

4X X X0 X X0 0 X

3

5

upper triangular

E32

E�132

2

64

1 0 0

0 1 0

0 `32 1

3

75

2

64

(row 1)

(row 2)

(row 3)� (`32)(row 2)

3

75 =

2

64

(row 1)

(row 2)

(row 3)

3

75

(`32)(row 2) + (1) [(row 3)� (`32)(row 2)] = (row 3)

15

Inverse of elimination matrices

What is the product

=

2

64

1 0 0

`21 1 0

`31 `32 1

3

75

We can write A as the product of a lower triangular matrix and an upper triangular

matrix.

E32E31E21A = U

or

A = (E�121 E�1

31 E�132 )U = LU

The multipliers used in elimination appear in L.

(E32E31E21)�1 = E�1

21 E�131 E�1

32 ?

This is the LU decomposition of A.

E�121 E�1

31 E�132 =

2

64

1 0 0

`21 1 0

0 0 1

3

75

2

64

1 0 0

0 1 0

`31 0 1

3

75

2

64

1 0 0

0 1 0

0 `32 1

3

75

16

Practice!

A =

2

664

3 �7 �2 2�3 5 1 06 �4 0 �5

�9 5 �5 12

3

775

L =

2

664

1 0 0 0`21 1 0 0`31 `32 1 0`41 `42 `43 1

3

775 U =

2

664

X X X X0 X X X0 0 X X0 0 0 X

3

775

Numbers are all integers!

Check that you get LU = A.

Row reduce the matrix A to get an upper triangular matrix U . Along the way,

record the multipliers `ij you use in a lower triangular matrix L. Check that you

get LU = A.

The e

nd!

17