math10212 linear algebra lecture notes...math10212 linear algebra b lecture 1 linear systems last...

MATH10212 Linear Algebra

Lecture Notes

Last change: 20 May 2019

MATH10212 • Linear Algebra B • Lecture 1 • Linear systems • Last change: 20 May 2019 2

Lecture 1 Introduction

Textbook

Students are strongly advised to ac-quire a copy of the Textbook:

D. C. Lay. Linear Algebra and itsApplications. Pearson, 2006. ISBN0-521-28713-4.

Other editions can be used as well; the

book is easily available on Amazon (thisincludes some very cheap used copies)and in Blackwell’s bookshop on campus.

These lecture notes should betreated only as indication of thecontent of the course; the text-book contains detailed explana-tions, worked out examples, etc.

About Homework

The Undergraduate Student Hand-book 1 says:

As a general rule for each hour in classyou should spend two hours on indepen-dent study. This will include reading lec-ture notes and textbooks, working on ex-ercises and preparing for coursework andexaminations.

In respect of this course, MATH10212Linear Algebra B, this means that stu-dents are expected to spend 8 (eight!)hours a week in private study of LinearAlgebra.

Normally, homework assignments willconsist of some odd numbered exercisesfrom the sections of the Textbook cov-ered in the lectures up to Wednesday inparticular week. The Textbook containsanswers to most odd numbered exercises(but the numbering of exercises mightchange from one edition to another).

Homework N (already posted on thewebpage and BB9 space of the course)should be returned to SupervisionClasses teachers early in Week N, formarking and discussion at supervisionclasses on Tuesday and Wednesday.

Communication

The Course Webpage is

http://www.maths.manchester.ac.

uk/~avb/math10212-Linear-Algebra-B.

html

The Course Webpage page is updatedalmost daily, sometimes several times aday. Refresh it (and files it is linked to)in your browser, otherwise you may missthe changes.

Email: Feel free to write to me withquestions, etc., at the address

[email protected].

uk

but only from your university e-mailaccount.

Emails from Gmail, Hotmail, etc. auto-matically go to spam.

1http://www.maths.manchester.ac.uk/study/undergraduate/information-for-current-students/

undergraduatestudenthandbook/teachingandlearning/timetabledclasses/.


What is Linear Algebra?

It is well-known that the total cost ofa purchase of amounts g1, g2, g3 of somegoods at prices p1, p2, p3, respectively, isan expression

p1g1 + p2g2 + p3g3 =3∑

i=1

pigi.

Expressions of this kind,

a1x1 + · · ·+ anxn

are called linear forms in variablesx1, . . . , xn with coefficients a1, . . . , an.

• Linear Algebra studies the mathe-matics of linear forms.

• Over the course, we shall developincreasingly compact notation foroperations of Linear Algebra. Inparticular, we shall discover that

p1g1 + p2g2 + p3g3

can be very conveniently written as

[p1 p2 p3

] g1g2g3

and then abbreviated

[p1 p2 p3

] g1g2g3

= P TG,

where

P =

p1p2p3

and G =

g1g2g3

,

and T (transposition) turns rowvectors into column vectors, andvice versa:

P T =

p1p2p3

T

=[p1 p2 p3

],

and

[p1 p2 p3

]T=

p1p2p3

.• Physicists use even more short no-

tation and, instead of

p1g1 + p2g2 + p3g3 =3∑

i=1

pigi

write

p1g1 + p2g

2 + p3g3 = pig

i,

omitting the summation sign en-tirely. This particular trick wasinvented by Albert Einstein, of allpeople. I do not use “physics”tricks in my lectures, but amprepared to give a few addi-tional lectures to physics stu-dents.

• Warning: Increasingly compactnotation leads to increasingly com-pact and abstract language.

• Unlike, say, Calculus, Linear Alge-bra focuses more on the develop-ment of a special mathematics lan-guage rather than on procedures.

Systems of simultaneous linear equations

A linear equation in the variablesx1, . . . , xn is an equation that can be

written in the form

a1x1 + a2x2 + · · ·+ anxn = b


where b and the coefficients a1, . . . , anare real numbers. The subscript n canbe any natural number.

A system of simultaneous linearequations is a collection of one or morelinear equations involving the same vari-ables, say x1, . . . , xn. For example,

x1 + x2 = 3

x1 − x2 = 1

We shall abbreviate the words “a sys-tem of simultaneous linear equa-

tions” just to “a linear system”.

A solution of the system is a list(s1, . . . , sn) of numbers that makes eachequation a true identity when the valuess1, . . . , sn are substituted for x1, . . . , xn,respectively. For example, in the systemabove (2, 1) is a solution.

The set of all possible solutions is calledthe solution set of the linear system.

Two linear systems are equivalent if thehave the same solution set.


Lecture 2 Systems of linear equations:

Elementary Operations [Lay 1.1]

A solution of the system is a list(s1, . . . , sn) of numbers that makes eachequation a true identity when the valuess1, . . . , sn are substituted for x1, . . . , xn,respectively. For example, in the systemabove (2, 1) is a solution.

The set of all possible solutions is calledthe solution set of the linear system.

Two linear systems are equivalent if thehave the same solution set.

We shall be use the following elemen-tary operations on systems od simul-taneous liner equations:

Replacement Replace one equation bythe sum of itself and a multiple ofanother equation.

Interchange Interchange two equa-tions.

Scaling Multiply all terms in a equa-tion by a nonzero constant.

Note: The elementary operations are re-versible.

Theorem: Elementary operationspreserve solutions.

If a system of simultaneous linear equa-tions is obtained from another system byelementary operations, then the two sys-tems have the same solution set.

We shall prove later in the course that asystem of linear equations has either

• no solution, or

• exactly one solution, or

• infinitely many solutions,

under the assumption that the coeffi-cients and solutions of the systems arereal or complex numbers.

A system of linear equations is said tobe consistent it if has solutions (eitherone or infinitely many), and a system ininconsistent if it has no solution.

Solving a linear system

The basic strategy is

to replace one system withan equivalent system (that is,with the same solution set)which is easier to solve.

Existence and uniquenessquestions

• Is the system consistent?

• If a solution exist, is it unique?

Equivalence of linear systems

• When are two linear systemsequivalent?

Checking solutions

Given a solution of the system of linearequations, how to check that it is cor-rect?


Row reduction and echelon forms [Lay 1.2]

Matrix notation

It is convenient to write coefficients of alinear system in the form of a matrix, arectangular table. For example, the sys-tem

x1 − 2x2 + 3x3 = 1

x1 + x2 = 2

x2 + x3 = 3

has the matrix of coefficients1 −2 31 1 00 1 1

and the augmented matrix1 −2 3 1

1 1 0 20 1 1 3

;

notice how the coefficients are alignedin columns, and how missing coefficientsare replaced by 0.

The augmented matrix in the exampleabove has 3 rows and 4 columns; we saythat it is a 3×4 matrix. Generally, a ma-trix with m rows and n columns is calledan m× n matrix.

Elementary row operations

Replacement Replace one row by thesum of itself and a multiple of an-other row.

Interchange Interchange two rows.

Scaling Multiply all entries in a row bya nonzero constant.

The two matrices are row equivalent ifthere is a sequence of elementary row op-erations that transforms one matrix intothe other.

Note:

• The row operations are reversible.

• Row equivalence of matrices is anequivalence relation on the set ofmatrices.

Theorem: Row Equivalence.

If the augmented matrices of two linearsystems are row equivalent, then the twosystems have the same solution set.

A nonzero row or column of a matrix isa row or column which contains at leastone nonzero entry.

We can now formulate a theorem (to beproven later).

Theorem: Equivalence of linearsystems.

Two linear systems are equivalent if andonly if the augmented matrix of one ofthem can be obtained from the aug-mented matrix of another system by frowoperations and insertion / deletion ofzero rows.

MATH10212 • Linear Algebra B • Lecture 3 • Solution of Linear Systems • Last change: 20 May 2019 7

Lecture 3 Solution of Linear Systems[Lay 1.2]

A nonzero row or column of a matrix isa row or column which contains at leastone nonzero entry. A leading entry ofa row is the leftmost nonzero entry (in anon-zero row).

Definition. A matrix is in echelonform (or row echelon form) if it hasthe following three properties:

1. All nonzero rows are above any rowof zeroes.

2. Each leading entry of a row is incolumn to the right of the leadingentry of the row above it.

3. All entries in a column below aleading entry are zeroes.

If, in addition, the following twoconditions are satisfied,

4. All leading entries are equal 1.

5. Each leading 1 is the only non-zeroentry in its column

then the matrix is in reduced echelonform.

An echelon matrix is a matrix in ech-elon form.

Any non-zero matrix can be row re-duced (that, transformed by elementaryrow operations) into a matrix in echelonform (but the same matrix can give riseto different echelon forms).

Examples. The following is a schematicpresentation of an echelon matrix:

� ∗ ∗ ∗ ∗0 � ∗ ∗ ∗0 0 0 � ∗

and this is a reduced echelon matrix:

1 0 ∗ 0 ∗0 1 ∗ 0 ∗0 0 0 1 ∗

Theorem 1.2.1: Uniqueness of thereduced echelon form.

Each matrix is row equivalent to one andonly one reduced echelon form.


Lecture 4 Solution of

Linear Systems[Lay 1.2]

Definition. A pivot position in a ma-trix A is a location in A that correspondsto a leading 1 in the reduced echelonform of A. A pivot column is a col-umn of A that contains a pivot position.

Example for solving in the lecture(The Row Reduction Algorithm):

0 2 2 2 21 1 1 1 11 1 1 3 31 1 1 2 2

A pivot is a nonzero number in a pivotposition which is used to create zeroes inthe column below it.

A rule for row reduction:

1. Pick the leftmost non-zero columnand in it the topmost nonzero en-try; it is a pivot.

2. Using scaling, make the pivotequal 1.

3. Using replacement row opera-tions, kill all non-zero entries in thecolumn below the pivot.

4. Mark the row and column contain-ing the pivot as pivoted.

5. Repeat the same with the matrixmade of not pivoted yet rows andcolumns.

6. When this is over, interchangethe rows making sure that the re-sulting matrix is in echelon form.

7. Using replacement row opera-tions, kill all non-zero entries in thecolumn above the pivot entries.

Solution of Linear Systems

When we converted the augmented ma-trix of a linear system into its reducedrow echelon form, we can write out theentire solution set of the system.

Example. Let1 0 −5 10 1 1 40 0 0 0

be the augmented matrix of a a linearsystem; then the system is equivalent to

x1 − 5x3 = 1

x2 + x3 = 4

0 = 0

The variables x1 and x2 correspond topivot columns in the matrix and a recalled basic variables (also leading orpivot variables). The other variable,x3 is a free variable.

Free variables can be assigned arbitraryvalues and then leading variables ex-pressed in terms of free variables:

x1 = 1 + 5x3

x2 = 4− x3x3 is free

Theorem 1.2.2: Existence andUniqueness

A linear system is consistent if and onlyif the rightmost column of the aug-mented matrix is not a pivot column—that is, if and only if an echelon form ofthe augmented matrix has no row of theform [

0 · · · 0 b]

with b nonzero

If a linear system is consistent, then thesolution set contains either


(i) a unique solution, when there areno free variables, or

(ii) infinitely many solutions, whenthere is at least one free variable.

Using row reduction to solve a lin-ear system

1. Write the augmented matrix of thesystem.

2. Use the row reduction algorithmto obtain an equivalent augmented

matrix in echelon form. Decidewhether the system is consistent.

3. if the system is consistent, get thereduced echelon form.

4. Write the system of equations cor-responding to the matrix obtainedin Step 3.

5. Express each basic variable interms of any free variables appear-ing in the equation.

MATH10212 • Linear Algebra B • Lecture 4 • Vector equations • Last change: 20 May 2019 10

Lecture 4 Vector equations [Lay 1.3]

A matrix with only one column is calleda column vector, or simply a vector.

Rn is the set of all column vectors withn entries.

A row vector: a matric with one row.

Two vectors are equal if and only if theyhave

• the same shape,

• the same number of rows,

• and their corresponding entries areequal.

The set of al vectors with n entries isdenoted Rn.

The sum u+v of two vectors u and v inRn is obtained by adding correspondingentries in u and v. For example in R2[

12

]+

[−1−1

]=

[01

].

The scalar multiple cv of a vector vand a real number (“scalar”) c is thevector obtained by multiplying each en-try in v by c. For example in R3,

1.5

10−3

=

1.50−2

.The vector whose entries are all zeroes iscalled the zero vector and denoted 0:

0 =

00...0

.Operations with row vectors are definedin a similar way.

Algebraic properties of Rn

For all u,v,w ∈ Rn and all scalars c andd:

1. u + v = v + u

2. (u + v) + w = u + (v + w)

3. u + 0 = 0 + u = u

4. u + (−u) = −u + u = 0

5. c(u + v) = cu + cv

6. (c+ d)u = cu + du

7. c(du) = (cd)u

8. 1u = u

(Here −u denotes (−1)u.)

Linear combinations

Given vectors v1,v2, . . . ,vp in Rn andscalars c1, c2, . . . , cp, the vector

y = c1v1 + · · · cpvp

is called a linear combina-tion of v1,v2, . . . ,vp with weightsc1, c2, . . . , cp.

Rewriting a linear system asa vector equation

Consider an example: the linear system

x2 + x3 = 2

x1 + x2 + x3 = 3

x1 + x2 − x3 = 2

can be written as equality of two vectors: x2 + x3x1 + x2 + x3x1 + x2 − x3

=

232

which is the same as

x1

011

+ x2

111

+ x3

11−1

=

232

MATH10212 • Linear Algebra B • Lecture 4 • Vector equations • Last change: 20 May 2019 11

Let us write the matrix0 1 1 21 1 1 31 1 −1 2

in a way that calls attention to itscolumns: [

a1 a2 a3 b]

Denote

a1 =

011

, a2 =

111

, a3 =

11−1

and

b =

232

,then the vector equation can be writtenas

x1a1 + x2a2 + x3a3 = b.

Notice that to solve this equation is thesame as

express b as a linear combi-nation of a1, a2, a3, and findall such expressions.

Therefore

solving a linear system is the same asfinding an expression of the vector of theright part of the system as a linear com-bination of columns in its matrix of co-efficients.

A vector equation

x1a1 + x2a2 + · · ·+ xnan = b.

has the same solution set as the linearsystem whose augmented matrix is[

a1 a2 · · · an b]

In particular b can be generated by a lin-ear combination of a1, a2, . . . , an if andonly if there is a solution of the corre-sponding linear system.

Definition. If v1, . . . ,vp are in Rn,then the set of all linear combination ofv1, . . . ,vp is denoted by

Span{v1, . . . ,vp}

and is called the subset of Rn spanned(or generated) by v1, . . . ,vp; or thespan of vectors v1, . . . ,vp.

That is, Span{v1, . . . ,vp} is the collec-tion of all vectors which can be writtenin the form

c1v1 + c2v2 + · · ·+ cpvp

with c1, . . . , cp scalars.

We say that vectors v1, . . . ,vp span Rn

if

Span{v1, . . . ,vp} = Rn

We can reformulate the definition ofspan as follows: iv a1, . . . , ap ∈ Rn, then

Span{a1, . . . , ap} = {b ∈ Rn such that

[a1 · · · ap|b]

is the augmented matrix

of a consistent system

of linear equations},

or, in simpler words,

Span{a1, . . . , ap} = {b ∈ Rn such that

the system of equations

x1a1 + · · ·+ xpap = b

has a solution}.

MATH10212 • Linear Algebra B • Lecture 5 • The matrix equation Ax = b • Last change: 20 May 2019 12

Lecture 5: The matrix equation Ax = b [Lay 1.4]

Definition. If A is an m × n matrix,with columns a1, . . . , an, and if x is inRn, then the product of A and x, de-noted Ax, is the linear combination ofthe columns of A using the correspond-ing entries in x as weights:

Ax =[a1a2 · · · an

] x1...xn

= x1a1 + x2a2 + · · ·+ xnan

Example. The system

x2 + x3 = 2

x1 + x2 + x3 = 3

x1 + x2 − x3 = 2

was written as

x1a1 + x2a2 + x3a3 = b.

where

a1 =

011

, a2 =

111

, a3 =

11−1

and

b =

232

.In the matrix product notation it be-comes 0 1 1

1 1 11 1 −1

x1x2x3

=

232

or

Ax = b

where

A =[a1 a2 a3

], x =

x1x2x3

.

Theorem 1.4.3: If A is an m × n ma-trix, with columns a1, . . . , an, and if x isin Rn, the matrix equation

Ax = b

has the same solution set as the vectorequation

x1a1 + x2a2 + · · ·+ xnan = b

which has the same solution set as thesystem of linear equations whose aug-mented matrix is[

a1 a2 · · · an b].

Existence of solutions. The equationAx = b has a solution if and only if b isa linear combination of columns of A.

Theorem 1.4.4: Let A be an m×n ma-trix. Then the following statements areequivalent.

(a) For each b ∈ Rm, the equationAx = b has a solution.

(b) Each b ∈ Rm is a linear combina-tion of columns of A.

(c) The columns of A span Rm.

(d) A has a pivot position in every row.

Row-vector rule for computing Ax.If the product Ax is defined then the ithentry in Ax is the sum of products ofcorresponding entries from the row i ofA and from the vector x.

Theorem 1.4.5: Properties of thematrix-vector product Ax.

If A is an m× n matrix, u,v ∈ Rn, andc is a scalar, then

(a) A(u + v) = Au + Av;

(b) A(cu) = c(Au).

MATH10212 • Linear Algebra B • Lecture 5 • The matrix equation Ax = b • Last change: 20 May 2019 13

Homogeneous linear systems

A linear system is homogeneous if itcan be written as

Ax = 0.

A homogeneous system always has atleast one solution x = 0 (trivial solu-tion).

Therefore for homogeneous systems animportant question os existence of a

nontrivial solution, that is, a nonzerovector x which satisfies Ax = 0:

The homogeneous system Ax = b hasa nontrivial solution if and only if theequation has at least one free variable.

Example.

x1 + 2x2 − x3 = 0

x1 + 3x3 + x3 = 0

MATH10212 • Linear Algebra B • Lecture 6 • Solution sets of linear equations • Last change: 20 May 2019 14

Lecture 6: Solution sets of linear equations [Lay 1.5]

Nonhomogeneous systems

When a nonhomogeneous system hasmany solutions, the general solution canbe written in parametric vector form aone vector plus an arbitrary linea com-bination of vectors that satisfy the cor-responding homogeneous system.

Example.

x1 + 2x2 − x3 = 0

x1 + 3x3 + x3 = 5

Theorem 1.5.6: Suppose the equation

Ax = b

is consistent for some given b, and pbe a solution. Then the solution set ofAx = b is the set of all vectors of theform

w = p + vh,

where vh is any solution of the homoge-neous equation

Ax = 0.

MATH10212 • Linear Algebra B • Lecture 7 • Linear independence • Last change: 20 May 2019 15

Lecture 7: Linear independence [Lay 1.7]

Definition. An indexed set of vectors

{v1, . . . ,vp }

in Rn is linearly independent if thevector equation

x1v1 + · · ·+ xpvp = 0

has only trivial solution.

The set{v1, . . . ,vp }

in Rn is linearly dependent if there ex-ist weights c1, . . . , cp, not all zero, suchthat

c1v1 + · · ·+ cpvp = 0

Linear independence of matrixcolumns. The matrix equation

Ax = 0

where A is made of columns

A =[a1 · · · an

]can be written as

x1a1 + · · ·+ xnan = 0

Therefore the columns of matrix A arelinearly independent if and only if theequation

Ax = 0

has only the trivial solution.

A set of one vectors {v1} is linearly de-pendent if v1 = 0.

A set of two vectors {v1,v2 } is linearlydependent if at least one of the vectorsis a multiple of the other.

Theorem 1.7.7: Characterisation oflinearly dependent sets. An indexedset

S = {v1, . . . ,vp }

of two or more vectors is linearly depen-dent if and only if at least one of thevectors in S is a linear combination ofthe others.

Theorem 1.7.8: dependence of“big” sets. If a set contains more vec-tors than entries in each vector, then theset is linearly dependent. Thus, any set

{v1, . . . ,vp }

in Rn is linearly dependent if p > n.

MATH10212 • Linear Algebra B • Lecture 8 • Linear transformations • Last change: 20 May 2019 16

Lecture 8: Introduction to linear transformations [1.8]

Transformation. A transformationT from Rn to Rm is a rule that assignsto each vector x in Rn a vector T (x) inRm.

The set Rn is called the domain of T ,the set Rm is the codomain of T .

Matrix transformations. With everym × n matrix A we can associated thetransformation

T : Rn −→ Rm

x 7→ Ax.

In short:T (x) = Ax.

The range of a matrix transforma-tion. The range of T is the set of alllinear combinations of the columns of A.

Indeed, this can be immediately seenfrom the fact that each image T (x) hasthe form

T (x) = Ax = x1a1 + · · ·+ xnan

Definition: Linear transformations.A transformation

T : Rn −→ Rm

is linear if:

• T (u + v) = T (u) + T (v) for allvectors u,v ∈ Rn;

• T (cu) = cT (u) for all vectors uand all scalars c.

Properties of linear transforma-tions. If T is a linear transformationthen

T (0) = 0

and

T (cu + dv) = cT (u) + dT (v).

The identity matrix. An n × n ma-trix with 1’s on the diagonal and 0’s else-where is called the identity matrix In.For example,

I1 =[1], I2 =

[1 00 1

],

I3 =

1 0 00 1 00 0 1

, I4 =

1 0 0 00 1 0 00 0 1 00 0 0 1

.The columns of the identity matrix Inwill be denoted

e1, e2, . . . , en.

For example, in R3

e1 =

100

, e2 =

010

, e3 =

001

.Obsetrve that if x is an arbitrary vectorin Rn,

x =

x1...xn

,then

x =

x1x2x3...xn

= x1

100...0

+ x2

010...0

+ · · ·+ xn

000...1

= x1e1 + x2e2 + · · ·+ xnen.

The identity transformation. It iseasy to check that

Inx = x for all x ∈ Rn.

MATH10212 • Linear Algebra B • Lecture 8 • Linear transformations • Last change: 20 May 2019 17

Therefore the linear transformation as-sociated with the identity matrix is the

identity transformation of Rn:

Rn −→ Rn

x 7→ x

The matrix of a linear transformation [Lay 1.9]

Theorem 1.9.10: The matrix of alinear transformation. Let

T : Rn −→ Rm

be a linear transformation.

Then there exists a unique matrix A suchthat

T (x) = Ax for all x ∈ Rn.

In fact, A is the m×n matrix whose jthcolumn is the vector T (ej) where ej isthe jth column of the identity matrix inRn:

A =[T (e1) · · · T (en)

]

Proof. First we express x in terms ofe1, . . . , en:

x = x1e1 + x2e2 + · · ·+ xnen

and compute, using definition of lineartransformation

T (x) = T (x1e1 + x2e2 + · · ·+ xnen)

= T (x1e1) + · · ·+ T (xnen)

= x1T (e1) + · · ·+ xnT (en)

and then switch to matrix

notation:

=[T (e1) · · · T (en)

] x1...xn

= Ax.

The matrix A is called the standardmatrix for the linear transforma-tion T .

Definition. A transformation

T : Rn −→ Rm

is onto Rm if each b ∈ Rm is the imageof at least one x ∈ Rn.

A transformation T is one-to-one ifeach b ∈ Rm is the image of at mostone x ∈ Rn.

Theorem: One-to-one transforma-tions: a criterion. A linear transfor-mation

T : Rn −→ Rm

is one-to-one if and only if the equation

T (x) = 0

has only the trivial solution.

Theorem: “One-to-one” and“onto” in terms of matrices. Let

T : Rn −→ Rm

be linear transformation and let A be thestandard matrix for T . Then:

• T maps Rn onto Rm if and only ifthe columns of A span Rm.

• T is one-to-one if and only if thecolumns of A are linearly indepen-dent.

MATH10212 • Linear Algebra B • Lecture 9 • Matrix operations: Addition • Last change: 20 May 2019 18

Lecture 9: Matrix operations: Addition [Lay 2.1]

Labeling of matrix entries. Let A bean m× n matrix.

A =[a1 a2 · · · an

]

=

a11 · · · a1j · · · a1n...

......

ai1 · · · aij · · · ain...

......

am1 · · · amj · · · amn

Notice that in aij the first subscript idenotes the row number, the second sub-script j the column number of the entryaij. In particular, the column aj is

aj =

a1j...aij...amj

.

Diagonal matrices, zero matrices.The diagonal entries in A are

a11, a22, a33, . . .

For example, the diagonal entries of thematrix

A =

1 2 34 5 67 8 9

are 1, 5, and 9.

A square matrix is a matrix with equalnumbers of rows and columns.

A diagonal matrix is a square matrixwhose non-diagonal entries are zeroes.

Matrices

[1 00 2

],

1 0 00 0 00 0 2

,π 0 0

0√

2 00 0 3

are all diagonal. The identity matrices

[1],

[1 00 1

],

1 0 00 1 00 0 1

, . . .are diagonal.

Zero matrix. By definition, 0 is a m×nmatrix whose entries are all zero. For ex-ample, matrices

[0 0

],

[0 0 00 0 0

]are zero matrices. Notice that zerosquare matrices, like

[0 00 0

],

0 0 00 0 00 0 0

are diagonal!

Sums. If

A =[a1 a2 · · · an

]and

B =[b1 b2 · · · bn

]are m × n matrices then we define thesum A+B as

A+B =[a1 + b1 a2 + b2 · · · an + bn

]

=

a11 + b11 · · · a1j + b1j · · · a1n + b1n

......

...ai1 + bi1 · · · aij + bij · · · ain + bin

......

...am1 + bm1 · · · amj + bmj · · · amn + bmn

MATH10212 • Linear Algebra B • Lecture 9 • Matrix operations: Addition • Last change: 20 May 2019 19

Scalar multiple. If c is a a scalar thenwe define

cA =[ca1 ca2 · · · can

]

=

ca11 · · · ca1j · · · ca1n

......

...cai1 · · · caij · · · cain

......

...cam1 · · · camj · · · camn

Theorem 2.1.1: Properties of ma-trix addition. Let A, B, and C be ma-

trices of the same size and r and s bescalars.

1. A+B = B + A

2. (A+B) + C = A+ (B + C)

3. A+ 0 = A

4. r(A+B) = rA+ rB

5. (r + s)A = rA+ sA

6. r(sA) = (rs)A.

MATH10212 • Linear Algebra B • Lecture 10 • Matrix multiplication • Last change: 20 May 2019 20

Lecture 10: Matrix multiplication [Lay 2.1]

Composition of linear transforma-tions. Let B be an m × n matrix andA an p × m matrix. They define lineartransformations

T : Rn −→ Rm, x 7→ Bx

and

S : Rm −→ Rp, y 7→ Ay.

Their composition

(S ◦ T )(x) = S(T (x))

is a linear transformation

S ◦ T : Rn −→ Rp.

From the previous lecture, we know thatlinear transformations are given by ma-trices. What is the matrix of S ◦ T?

Multiplication of matrices. To an-swer the above question, we need to com-pute A(Bx) in matrix form.

Write x as

x =

x1...xn

and observe

Bx = x1b1 + · · ·+ xnbn.

Hence

A(Bx) = A(x1b1 + · · ·+ xnbn)

= A(x1b1) + · · ·+ A(xnbn)

= x1A(b1) + · · ·+ xnA(bn)

=[Ab1 Ab2 · · · Abn

]x

Therefore multiplication by the matrix

C =[Ab1 Ab2 · · · Abn

]transforms x into A(Bx). Hence C isthe matrix of the linear transformation

S ◦T and it will be natural to call C theproduct of A and B and denote

C = A ·B

(but the multiplication symbol “·” is fre-quently skipped).

Definition: Matrix multiplication.If A is an p × m matrix and B is anm× n matrix with columns

b1, . . . ,bn

then the product AB is the p×n matrixwhose columns are

Ab1, . . . , Abn :

AB = A[b1 · · · bn

]=

[Ab1 · · · Abn

].

Columns of AB. Each column Abj ofAB is a linear combination of columns ofA with weights taken from the jth col-umn of B:

Abj =[a1 · · · am

] b1j...bmj

= b1ja1 + · · ·+ bmjam

Mnemonic rules

[m× n matrix] · [n× p matrix] =[m× p matrix]

columnj(AB) = A · columnj(B)

rowi(AB) = rowi(A) ·B

Theorem 2.1.2: Properties of ma-trix multiplication. Let A be an m×nmatrix and let B and C be matrices forwhich indicated sums and products aredefined. Then the following identitiesare true:

MATH10212 • Linear Algebra B • Lecture 10 • Matrix multiplication • Last change: 20 May 2019 21

1. A(BC) = (AB)C

2. A(B + C) = AB + AC

3. (B + C)A = BA+ CA

4. r(AB) = (rA)B = A(rB) for anyscalar r

5. ImA = A = AIn

Powers of matrix. As it is usual inalgebra, we define, for a square matrixA,

Ak = A · · ·A (k times)

If A 6= 0 then we set

A0 = I

The transpose of a matrix. Thetranspose AT of an m × n matrix A isthe n×m matrix whose rows are formedfrom corresponding columns of A:

[1 2 34 5 6

]T=

1 42 53 6

Theorem 2.1.3: Properties of trans-pose. Let A and B denote matriceswhose sizes are appropriate for the fol-lowing sums and products. Then wehave:

1. (AT )T = A

2. (A+B)T = AT +BT

3. (rA)T = r(AT ) for any scalar r

4. (AB)T = BTAT

MATH10212 • Linear Algebra B • Lecture 11 • The inverse of a matrix • Last change: 20 May 2019 22

Lecture 11: The inverse

of a matrix [Lay 2.2]

Invertible matrices

An n×n matrix A is invertible if thereis an n× n matrix C such that

CA = I and AC = I

C is called the inverse of A.

The inverse of A, if exists, is unique (!)and is denoted A−1:

A−1A = I and AA−1 = I.

Singular matrices. A non-invertiblematrix is called a singular matrix.

An invertible matrix is nonsingular.

Theorem 2.2.4: Inverse of a 2 × 2matrix. Let

A =

[a bc d

]If ad− bc 6= 0 then A is invertible and[

a bc d

]−1=

1

ad− bc

[d −b−c a

]

The quantity ad − bc is called the de-terminant of A:

detA = ad− bc

Theorem 2.2.5: Solving matrixequations. If A is an invertible n × nmatrix, then for each b ∈ Rn, the equa-tion Ax = b has the unique solution

x = A−1b.

Theorem 2.2.6: Properties of in-vertible matrices.

(a) If A is an invertible matrix, thenA−1 is also invertible and

(A−1)−1 = A

(b) If A and B are n×n invertible ma-trices, then so is AB, and

(AB)−1 = B−1A−1

(c) If A is an invertible matrix, thenso is AT , and

(AT )−1 = (A−1)T

MATH10212 • Linear Algebra B • Lecture 12 • Characterizations of invertible matrices • Last change: 20 May 2019 23

Lecture 12: Characterizations of invertible matrices.

Definition: Elementary matrices.An elementary matrix is a matrix ob-tained by performing a single elementaryrow operation on an identity matrix.

If an elementary row transformation isperformed on an n × n matrix A, theresulting matrix can be written as EA,where the m×m matrix is made by thesame row operations on Im.

Each elementary matrix E is invertible.The inverse of E is the elementary ma-trix of the same type that transforms Eback into I.

Theorem 2.2.7: Invertible matrices.An n × n matrix A is invertible if andonly if A is row equivalent to In, andin this case, any sequence of elementaryrow operations that reduces A to In alsotransforms In into A−1.

Computation of inverses.

• Form the augmented matrix[A I

]and row reduce it.

• If A is row equivalent to I,then

[A I

]is row equivalent to[

I A−1].

• Otherwise A has no inverse.

The Invertible Matrix Theorem2.3.8: For an n × n matrix A, the fol-lowing are equivalent:

(a) A is invertible.

(b) A is row equivalent to In.

(c) A has n pivot positions.

(d) Ax = 0 has only the trivial solu-tion.

(e) The columns of A are linearly in-dependent.

(f) The linear transformation x 7→ Axis one-to-one.

(g) Ax = b has at least one solutionfor each b ∈ Rn.

(h) The columns of A span Rn.

(i) x 7→ Ax maps Rn onto Rn.

(j) There is an n × n matrix C suchthat CA = I.

(k) There is an n × n matrix D suchthat AD = I.

(l) AT is invertible.

One-sided inverse is the inverse. LetA and B be square matrices.

If AB = I then both A and B are in-vertible and

B = A−1 and A = B−1.

Theorem 2.3.9: Invertible lineartransformations. Let

T : Rn −→ Rn

be a linear transformation and A itsstandard matrix.

Then T is invertible if and only if A isan invertible matrix.

In that case, the linear transformation

S(x) = A−1x

is the only transformation satisfying

S(T (x)) = x for all x ∈ Rn

T (S(x)) = x for all x ∈ Rn

MATH10212 • Linear Algebra B • Lecture 13 • Partitioned matrices • Last change: 20 May 2019 24

Lecture 13 Partitioned matrices [Lay 2.4]

Example: Partitioned matrix

A =

1 2 a3 4 bp q z

=

[A11 A12

A21 A22

]

A is a 3× 3 matrix which can be viewedas a 2× 2 partitioned (or block) ma-trix with blocks

A11 =

[1 23 4

], A12 =

[ab

],

A21 =[p q

], A22 =

[z]

Addition of partitioned matrices. Ifmatrices A and B are of the same sizeand partitioned the same way, they canbe added block-by-block.

Similarly, partitioned matrices can bemultiplied by a scalar blockwise.

Multiplication of partitioned ma-trices. If the column partition of Amatches the row partition of B then

AB can be computed by the usual row-column rule, with blocks treated as ma-trix entries.

A =

1 2 a3 4 bp q z

, B =

α βγ δΓ ∆

Example

A =

1 2 a3 4 bp q z

=

[A11 A12

A21 A22

],

B =

α βγ δΓ ∆

=

[B1

B2

]

AB =

[A11 A12

A21 A22

] [B1

B2

]=

[A11B1 + A12B2

A21B1 + A22B2

]

=

[1 23 4

] [α βγ δ

]+

[ab

] [Γ ∆

][p q

] [α βγ δ

]+[z] [

Γ ∆]

Theorem 2.4.10: Column-Row Expansion of AB. If A is m × n and B is n × pthen

AB =[col1(A) col2(A) · · · coln(A)

]

row1(B)row2(B)

...rown(B)

= col1(A)row1(B) + · · ·+ coln(A)rown(B)

Example Partitioned matrices natu-rally arise in analysis of systems of si-multaneous linear equations. Consider a

sysetm of equations:

x1 + 2x2 + 3x3 + 4x4 = 1

x1 + 3x2 + 3x3 + 4x4 = 2

MATH10212 • Linear Algebra B • Lecture 13 • Partitioned matrices • Last change: 20 May 2019 25

In matrix form, it looks as

[1 2 3 41 3 3 4

]x1x2x3x4

=

[12

]

Denote

A =

[1 21 3

]C =

[3 43 4

]Y1 =

[x1x2

]Y2 =

[x3x4

]B =

[12

]then we can rewrite the matrices in-volved in the matrix equation ar parti-tioned matrices:[

A C] [Y1Y2

]= B

and, multiplying matrices as partitionedmatrices,

AY1 + CY2 = B.

Please notice that matrix A is invertible– at this stage it should be an easy men-tal calculation for you. Multiplying theboth parts of the last equation by A−1,we get

Y1 + A−1CY2 = A−1B

andY1 = A−1B − A−1CY2

The entries of

Y2 =

[x3x4

]can be taken for free parameters: x3 = t,x4 = s, and x1 and x2 expressed as[

x1x2

]= A−1B − A−1C

[ts

]

Observe that

A−1 =

[3 −2−1 1

]and

A−1C =

[3 −2−1 1

]·[3 43 4

]=

[3 40 0

]which gives us an answer:[x1x2

]=

[3 −2−1 1

] [12

]−[3 40 0

] [ts

]=

[−11

]−[3t+ 4s

0

][x3x4

]=

[ts

]We may wish to bring it in a more tra-ditional form:x1x2x3x4

=

−1100

+

−3t− 4s

0ts

=

−1100

+ t

−3010

+ s

−4001

Of course, when we solving a singles sys-tem of equations, the formulae of thekind

Y1 = A−1B − A−1CY2

do not save time and effort, but in manyapplications you have to solve millionsof systems of linear equations with thesame matrix of coefficients but differentright-hand parts, and in this situationshortcuts based on compound matricesbecome very useful.

MATH10212 • Linear Algebra B • Lecture 14 • Subspaces • Last change: 20 May 2019 26

Lecture 14: Subspaces of Rn [Lay 2.8]

Subspace. A subspace of Rn is any setH that has the properties

(a) 0 ∈ H.

(b) For each vectors u,v ∈ H, the sumu + v ∈ H.

(c) For each vector u ∈ H and eachscalar c, cu ∈ H.

Rn is a subspace of itself.

{0 } is a subspace, called the zero sub-space.

Span. Span{v1, . . . ,vp } is the sub-space spanned (or generated) byv1, . . . ,vp.

The column space of a matrix A is theset ColA of all linear combinations ofcolumns of A.

The null space of a matrix A is theset NulA of all solutions to the homo-geneous system

Ax = 0

Theorem 2.8.12: The null space.The null space of an m×n matrix A isa subspace of Rn.

Equivalently, the set of all solutions to asystem

Ax = 0

of m homogeneous linear equations in nunknowns is a subspace of Rn.

Basis of a subspace. A basis of a sub-space H of Rn is a linearly independentset in H that spans H.

Theorem 2.8.13: Pivot columns of amatrix A form a basis of ColA.

Dimension and rank [Lay 2.9]

Theorem: Given a basis b1, . . . ,bp in asubspace H of Rn, every vector u ∈ H isuniquely expressed as a linear combina-tion of b1, . . . ,bp.

Solving homogeneous systems.Finding a parametric solution of a sys-tem of homogeneous linear equations

Ax = 0

means to find a basis of NulA.

Coordinate system. Let

B = {b1, . . . ,bp }

be a basis for a subspace H.

For each x ∈ H, the coordinates of xrelative to the basis B are the weights

c1, . . . , cp

such that

x = c1b1 + · · ·+ cpbp.

The vector

[x]B =

c1...cp

is the coordinate vector of x.

Dimension. The dimension of anonzero subspace H, denoted by dimH,is the number of vectors in any basis ofH.

The dimension of the zero subspace {0}is defined to be 0.

Rank of a matrix. The rank of a ma-trix A, denoted by rankA, is the dimen-

MATH10212 • Linear Algebra B • Lecture 14 • Subspaces • Last change: 20 May 2019 27

sion of the column space ColA of A:

rankA = dim ColA

The Rank Theorem 2.9.14: If a ma-trix A has n columns,

rankA+ dim NulA = n.

The Basis Theorem 2.9.15: Let H bea p-dimensional subspace of Rn.

• Any linearly independent set of ex-actly p elements in H is automati-cally a basis for H.

• Also, any set of p elements of Hthat spansH is automatically a ba-sis of H.

For example, this means that the vectors100

,2

30

,4

56

form a basis of R3: there are three ofthem and they are linearly independent(the latter should be obvious to studentsat this stage).

The Invertible Matrix Theorem(continued) Let A be an n× n matrix.Then the following statements are eachequivalent to the statement that A is aninvertible matrix.

(m) The columns of A form a basis ofRn.

(n) ColA = Rn.

(o) dim ColA = n.

(p) rankA = n.

(q) NulA = {0}.

(r) dim NulA = 0.

MATH10212 • Linear Algebra B • Lecture 15 • Introduction to determinants • Last change: 20 May 2019 28

Lecture 15: Introduction to determinants [Lay 3.1]

The determinant detA of a square ma-trix A is a certain number assigned to thematrix; it is defined recursively, thatis, we define first determinants of matri-ces of sizes 1 × 1, 2 × 2, and 3 × 3, andthen supply a formula which expressesdeterminants of n× n matrices in termsof determinants of (n− 1)× (n− 1) ma-trices.

The determinant of a 1 × 1 matrixA =

[a11]

is defined simply as beingequal its only entry a11:

det[a11]

= a11.

The determinant of a 2 × 2 matrixis defined by the formula

det

[a11 a12a21 a22

]= a11a22 − a12a21.

The determinant of a 3× 3 matrix

A =

a11 a12 a13a21 a22 a23a31 a32 a33

is defined by the formula

det

a11 a12 a13a21 a22 a23a31 a32 a33

= a11 ·[a22 a23a32 a33

]− a12 ·

[a21 a23a31 a33

]+ a13 ·

[a21 a22a31 a32

]

Example. The determinant detA of thematrix

A =

2 0 70 1 01 0 4

equals

2 · det

[1 00 4

]+ 7 · det

[0 11 0

]which further simplifies as

2 · 4 + 7 · (−1) = 8− 7 = 1.

Submatrices. By definition, the sub-matrix Aij is obtained from the matrixA by crossing out row i and column j.

For example, if

A =

1 2 34 5 67 8 9

,then

A22 =

[1 37 9

], A31 =

[2 35 6

]

Recursive definition of determi-nant. For n > 3, the determinant ofan n × n matrix A is defined as the ex-pression

detA = a11 detA11 − a12 detA12 + · · ·+ (−1)1+na1n detA1n

=n∑

j=1

(−1)1+ja1j detA1j


which involves the determinants ofsmaller (n−1)×(n−1) submatrices A1j,which, in their turn, can be evaluated bya similar formula wich reduces the cal-cualtions to the (n − 2) × (n − 2) case,and can be repeated all the way down todeterminants of size 2× 2.

Cofactors. The (i, j)-cofactor of A is

Cij = (−1)i+j detAij

Then

detA = a11C11 + a12C12 + · · ·+ a1nC1n

is cofactor expansion across the firstrow of A.

Theorem 3.1.1: Cofactor expan-sion. For any row i, the result of thecofactor expansion across the row i is thesame:

detA = ai1Ci1 + ai2Ci2 + · · ·+ ainCin

The chess board pattern of signs incofactor expansions. The signs

(−1)i+j

which appear in the formula for co-factors, form the easy-to-recognise andeasy-to-remember “chess board” pat-tern:

+ − + · · ·− + −+ − +...

. . .

Theorem 3.1.2: The determinantof a triangular matrix. If A is a tri-angular n × n matrix then detA is theproduct of the diagonal entries of A.

Proof: This proof contains more detailsthan the one given in the textbook. It

is based on induction on n, which, toavoid the use of “general” notation is il-lustrated by a simple example. Let

A =

a11 0 0 0 0a21 a22 0 0 0a31 a32 a33 0 0a41 a42 a43 a44 0a51 a52 a53 a54 a55

.All entries in the first row of A–with pos-sible exception of a11—are zeroes,

a12 = · · · = a15 = 0,

therefore in the formula

detA = a11 detA11 + · · ·+ a55 detA55

all summand with possible exception ofthe first one,

a11 detA11,

are zeroes, and therefore

detA = a11 detA11

= a11 det

a22 0 0 0a32 a33 0 0a42 a43 a44 0a52 a53 a54 a55

.But the smaller matrix A11 is also lowertriangle, and therefore we can concludeby induction that

det

a22 0 0 0a32 a33 0 0a42 a43 a44 0a52 a53 a54 a55

= a22 · · · a55

hence

detA = a11 · (a22 · · · a55)= a11a22 · · · a55

is the product of diagonal entries of A.

The basis of induction is the case n = 2;of course, in that case

det

[a11 0a21 a22

]= a11a22 − 0 · a21= a11a22


is the product of the diagonal entries ofA.

Corollary. The determinant of a diag-onal matrix equals is the product of itsdiagonal elements:

det

d1 0 · · · 00 d2 0...

. . ....

0 · · · dn

= d1d2 · · · dn.

Corollary. The determinant of theidentity matrix equals 1:

det In = 1.

Corollary. The determinant of the zeromatrix equals 0:

det 0 = 0.

MATH10212 • Linear Algebra B • Lectures 15–16 • Properties of determinants • Last change: 20 May 2019 31

Lectures 15–16: Properties of determinants [Lay 3.2]

Theorem 3.2.3: Row Operations.Let A be a square matrix.

(a) If a multiple of one row of A isadded to another row to producea matrix B, then

detB = detA.

(b) It two rows of A are swapped toproduce B, then

detB = − detA.

(c) If one row of A is multiplied by kto produce B, then

detB = k detA.

Theorem 3.2.5: The determinant ofthe transposed matrix. If A is ann× n matrix then

detAT = detA.

Example. Let

A =

1 2 00 1 00 3 4

,then

AT =

1 0 02 1 30 0 4

and

detA = 1 · det

[1 03 4

]−2 · det

[0 00 4

]+0 · det

[0 10 3

]= 1 · 4− 2 · 0 + 0 · 0= 4.

Similarly

detAT = 1 · det

[1 30 4

]+ 0 + 0

= 1 · 4 = 4;

we got the same value.

Corollary: Cofactor expansionacross a column. For any column j,

detA = a1jC1j + a2jC2j + · · ·+ anjCnj

Example. We have already computedthe determinant of the matrix

A = det

2 0 70 1 01 0 4

by expansion across the first row. Nowwe do it by expansion across the thirdcolumn:

detA = 7 · (−1)1+3 · det

[0 11 0

]+0 · (−1)2+3 · det

[2 01 0

]+4 · (−1)3+3 · det

[2 00 1

]= −7− 0 + 8

= 1.

Proof: Columns of A are rows of AT

which has the same determinant as A.

Corollary: Column operations. LetA be a square matrix.

(a) If a multiple of one column of Ais added to another column to pro-duce a matrix B, then

detB = detA.


(b) It two columns of A are swappedto produce B, then

detB = − detA.

(c) If one column of A is multiplied byk to produce B, then

detB = k detA.

Proof: Column operations on A are rowoperation of AT which has the same de-terminant as A.

Computing determinants. In com-putations by hand, the quickest methodof computing determinants is to workwith both columns and rows:

• If a row or a column has a conve-nient scalar factor, take it out ofthe determinant.

• If convenient, swap two rows ortwo columns— but do not forgetto change the sign of the determi-nant.

• Adding to a row (column) scalarmultiples of other rows (columns),simplify the matrix to create a row(column) with just one nonzero en-try.

• Expand across this row (column).

• Repeat until you get the value ofthe determinant.

Example.

det

1 0 31 3 11 1 −1

C3−3C1= det

1 0 01 3 −21 1 −4

2 out of C3= 2 · det

1 0 01 3 −11 1 −2

−1 out of C3= −2 · det

1 0 01 3 11 1 2

expand

= −2 · 1 · (−1)1+1 · det

[3 11 2

]R1−3R2= −2 · det

[0 −51 2

]R1↔R2= −2 · (−1) · det

[1 20 −5

]= 2 · (−5)

= −10.

Example. Compute the determinant

∆ = det

1 1 1 1 01 1 1 0 11 1 0 1 11 0 1 1 10 1 1 1 1

Solution: Subtracting Row 1 fromRows 2, 3, and 4, we rearrange the de-


terminant as

∆ = det

1 1 1 1 00 0 0 −1 10 0 −1 0 10 −1 0 0 10 1 1 1 1

After expanding the determinant acrossthe first column, we get

∆ = 1 · (−1)1+1 · det

0 0 −1 10 −1 0 1−1 0 0 11 1 1 1

= det

0 0 −1 10 −1 0 1−1 0 0 11 1 1 1

R4+R3= det

0 0 −1 10 −1 0 1−1 0 0 10 1 1 2

After expansion across the 1st column

∆ = (−1) · (−1)3+1 · det

0 −1 1−1 0 11 1 2

= − det

0 −1 1−1 0 11 1 2

R3+R2= − det

0 −1 1−1 0 10 1 3

After expanding the determinant across

the first column we have

∆ = −(−1) · (−1)2+1 · det

[−1 11 3

]

= − det

[−1 11 3

]R3+R2= − det

[−1 10 4

]= −(−4)

= 4.

As an exercise, I leave you with this ques-tion: wouldn’t you agree that the deter-minant of the similarly constructed ma-trix of size n × n should be ±(n − 1)?But how can one determine the correctsign?

And one more exercise: you should nowbe able to instantly compute

det

1 2 3 4 01 2 3 0 51 2 0 4 51 0 3 4 50 2 3 4 5

.

Theorem 3.2.4: Invertible matri-ces. A square matrix A is invertible ifand only if

detA 6= 0.

Recall that this theorem has been al-ready known to us in the case of 2 × 2matrices A.

Example. The matrix

A =

2 0 70 1 01 0 4

has the determinant

detA = 2 · det

[1 00 4

]+ 7 · det

[0 11 0

]= 2 · 4 + 7 · (−1)

= 8− 7 = 1


hence A is invertible. Indeed,

A−1 =

4 0 −70 1 0−1 0 2

—check!

Theorem 3.2.6: MultiplicativityProperty. If A and B are n × n ma-trices then

detAB = (detA) · (detB)

Example. Take

A =

[1 03 2

]and B =

[1 50 1

],

then

detA = 2 and detB = 1.

But

AB =

[1 03 2

] [1 50 1

]=

[1 53 17

]

and

det(AB) = 1 · 17− 5 · 3 = 2,

which is exactly the value of

(detA) · (detB).

Corollary. If A is invertible,

detA−1 = (detA)−1.

Proof. Set B = A−1, then

AB = I

and Theorem 3.2.6 yields

detA · detB = det I = 1.

Hence

detB = (detA)−1.

Cramer’s Rule [Lay 3.3]

Cramer’s Rule is an explicit (closed) for-mula for solving systems of n linear equa-tions with n unknowns and nonsigular(invertible) matrix of coefficients. It hasimportant theoretical value, but is un-suitable for practical application.

For any n×n matrix A and any b ∈ Rn,denote by Ai(b) the matrix obtainedfrom A by replacing column i by the vec-tor b:

Ai(b) =[a1 · · · ai−1 b ai+1 · · · an

].

Theorem 3.3.7: Cramer’s Rule. LetA be an invertible n × n matrix. Forany b ∈ Rn, the unique solution x of thelinear system

Ax = b

is given by

xi =detAi(b)

detA, i = 1, 2, . . . , n.

For example, for a system

x1 + x2 = 3

x1 − x2 = 1

Cramer’s rule gives the answer

x1 =

det

[3 11 −1

]det

[1 11 −1

] =−4

−2= 2

x2 =

det

[1 31 1

]det

[1 11 −1

] =−2

−2= 1


Theorem 3.3.8: An Inverse For-mula. Let A be an invertible n× n ma-trix. Then

A−1 =1

detA

C11 C21 · · · Cn1

C12 C22 · · · Cn2...

......

C1n C2n · · · Cnn

where cofactors Cji are given by

Cji = (−1)j+i detAji

and Aji is the matrix obtained from Aby deleting row j and column i.

Notice that for a 2× 2 matrix

A =

[a bc d

]

the cofactors are

C11 = (−1)1+1 · d = d

C21 = (−1)2+1 · b = −bC12 = (−1)1+2 · c = −cC22 = (−1)2+2 · a = a,

and we get the familiar formula

A−1 =1

detA

[d −b−c a

]

=1

ad− bc

[d −b−c a

].

MATH10212 • Linear Algebra B • Lecture 17 • Eigenvalues and eigenvectors • Last change: 20 May 2019 36

Lecture 17: Eigenvalues and eigenvectors

Quiz

What is the value of this determinant?

∆ = det

1 2 30 2 31 2 6

Quiz

Is Γ positive, negative, or zero?

Γ = det

0 0 710 93 0

87 0 0

Eigenvectors. An eigenvector of ann × n matrix A is a nonzero vector xsuch that

Ax = λx

for some scalar λ.

Eigenvalues. A scalar λ is called aneigenvalue of A if there is a non-trivialsolution x of

Ax = λx;

x is called an eigenvector correspond-ing to λ.

Eigenvalue. A scalar λ is an eigenvalueof A iff the equation

(A− λI)x = 0

has a non-trivial solution.

Eigenspace. The set of all solutions of

(A− λI)x = 0

is a subspace of Rn called theeigenspace of A corresponding to theeigenvalue λ.

Theorem: Linear independence ofeigenvectors. If v1, . . . ,vp are eigen-vectors that correspond to distinct eigen-values of λ1, . . . , λp of an n×n matrix A,then the set

{v1, . . . ,vp }

is linearly independent.

MATH10212 • Linear Algebra B • Lecture 17 • The characteristic equation • Last change: 20 May 2019 37

Lecture 18: The characteristic equation

Characteristic polynomial

The polynomial

det(A− λI)

in variable λ is characteristic polyno-mial of A.

For example, in the 2× 2 case

det

[a− λ bc d− λ

]= λ2−(a+d)λ+(ad−bc).

det(A− λI) = 0

is the characteristic equation for A.

Characterisation of eigenvalues

Theorem. A scalar λ is an eigenvalueof an n × n matrix A if and only if λsatisfies the characteristic equation

det(A− λI) = 0

Zero as an eigenvalue. λ = 0 is aneigenvalue of A if and only if detA = 0.

The Invertible Matrix Theorem(continued). An n× n matrix A is in-vertible iff :

(s) The number 0 is not an eigenvalueof A.

(t) The determinant of A is not zero.

Theorem: Eigenvalues of a triangu-lar matrix. The eigenvalues of a trian-gular matrix are the entries on its maindiagonal.

Proof. This immediately follows fromthe fact the the determinant of a trian-gular matrices is the product of its diag-onal elements. Therefore, for example,for the 3× 3 triangular matrix

A =

d1 a b0 d2 c0 0 d3

its characteristic polynomial det(A−λI)equals

det

d1 − λ a b0 d2 − λ c0 0 d3 − λ

and therefore

det(A− λI) = (d1 − λ)(d2 − λ)(d3 − λ)

has roots (eigenvalues of A)

λ = d1, d2, d3.

MATH10212 • Linear Algebra B • Lectures 18–21 • Diagonalisation 38

Lectures 19–22: Diagonalisation • Last change: 20

May 2019

Similar matrices. A is similar to B ifthere exist an invertible matrix P suchthat

A = PBP−1.

Example:

Occasionally the similarity relation is de-noted by symbol ∼.

Similarity is an equivalence relation: itis

• reflexive: A ∼ A

• symmetric: A ∼ B implies B ∼ A

• transitive: A ∼ B and B ∼ C im-plies A ∼ C.

Conjugation. Operation

AP = P−1AP

is called conjugation of A by P .

Properties of conjugation

• IP = I

• (AB)P = APBP ; as a corollary:

• (A+B)P = AP +BP

• (c · A)P = c · AP

• (Ak)P = (AP )k

• (A−1)P = (AP )−1

• (AP )Q = APQ

Theorem: Similar matrices havethe same characteristic polynomial.If matrices A and B are similar then theyhave the same characteristic polynomialsand eigenvalues.

Diagonalisable matrices. A squarematrix A is diagonalisable if it is simi-lar to a diagonal matrix, that is,

A = PDP−1

for some diagonal matrix D and someinvertible matrix P .

Of course, the diagonal entries of D areeigenvalues of A.

The Diagonalisation Theorem. Ann × n matrix A is diagonalisable iff Ahas n linearly independent eigenvectors.In fact,

A = PDP−1

where D is diagonal iff the columns ofP are n linearly independent eigenvec-tors of A.

Theorem: Matrices with all eigen-values distinct. An n× n matrix withn distinct eigenvalues is diagonalisable.

Example: Non-diagonalisable ma-trices

A =

[2 10 2

]has repeated eigenvalues λ1 = λ2 = 2.

A is not diagonalisable. Why?

MATH10212 • Linear Algebra B • Lecture 25 • Symmetric matrices and inner product • Last change: 20 May 2019 39

Lectures 23-24: Symmetric matrices and inner prod-

uct

Symmetric matrices. A matrix A issymmetric if AT = A.

An example:

A =

1 2 32 3 43 4 5

.Notice that symmetric matrices are nec-essarily square.

Inner (or dot) product. For vectorsu,v ∈ Rn their inner product (alsocalled scalar product or dot product)u �v is defined as

u �v = uTv

=[u1 u2 · · ·un

]v1v2...vn

= u1v1 + u2v2 + · · ·+ unvn

Properties of dot product

Theorem 6.1.1. Let u,v,w ∈ Rn, andc be a scalar. Then

(a) u �v = v �u

(b) (u + v) �w = u �w + v �w

(c) (cu) �v = c(u �v)

(d) u �u > 0, and u �u = 0 if and onlyif u = 0

Properties (b) and (c) can be combinedas

(c1u1 + · · ·+ cpup) �w = c1(u1�w) + · · ·+ cp(up

�w)

The length of a vector v is defined as

‖v‖ =√

v �v =√v21 + v22 + · · ·+ v2n

In particular,

‖v‖2 = v �v.

We call v a unit vector if ‖v‖ = 1.

Orthogonal vectors. Vectors u and vare orthogonal to each other if

u �v = 0

Theorem. Vectors u and v are orthog-onal if and only if

‖u‖2 + ‖v‖2 = ‖u + v‖2.

That is, vectors u and v are orthogonalif and only if the Pythagoras Theoremholds for the triangle formed by u andv as two sides (so that its third side isu + v).

Proof is a straightforward computation:

‖u + v‖2 = (u + v) � (u + v)

= u �u + u �v + v �u + v �v

= u �u + u �v + u �v + v �v

= u �u + 2u �v + v �v

= ‖u‖2 + 2u �v + ‖v‖2.


Hence

‖u + v‖2 = ‖u‖2 + ‖v‖2 + 2u �v,

and

‖u + v‖2 = ‖u‖2 + ‖v‖2 iff u �v = 0.

�

Orthogonal sets. A set of vectors

{u1, . . . ,up }

in Rn is orthogonal if

ui�uj = 0 whenever i 6= j.

Theorem 6.2.4. If

S = {u1, . . . ,up }

is an orthogonal set of non-zero vectorsin Rn then S is linearly independent.

Proof. Assume the contrary that Sis linearly dependent. This means thatthere exist scalars c1, . . . , cp, not all ofthem zeroes, such that

c1u1 + · · ·+ cpup = 0.

Forming the dot product of the left handside and the right hand side of this equa-tion with ui, we get

(c1u1 + · · ·+ cpup) �ui = 0 �ui = 0

After opening the brackets we have

c1u1�ui + · · ·+ ci−1ui−1 + ciui

�ui + ci+1ui+1�ui + · · ·+ cpup

�ui = 0.

In view of orthogonality of the set S, alldot products on the left hand side, withthe exception of ui

�u, equal 0. Hencewhat remains from the equality is

ciui�ui = 0

Since ui is non-zero,

ui�ui 6= 0

and therefore ci = 0. This argumentworks for every index i = 1, . . . , p. Weget a contradition with our assumptionthat one of ci is non-zero. �

An orthogonal basis for Rn is a basiswhich is also an orthogonal set.

For example, the two vectors[11

],

[1−1

]form an orthogonal basis of R2 (check!).

Theorem: Eigenvectors of symmet-ric matrices. Let

v1, . . . ,vp

be eigenvectors of a symmetric matrix Afor pairwise distinct eigenvalues

λ1, . . . , λp.

Then

{v1, . . . ,vp }

is an orthogonal set.

Proof. We need to prove the following:

If u and v are eigenvectorsfor A for eigenvalues λ 6= µ,then

u �v = 0.

For a proof, consider the following se-


quence of equalities:

λu �v = (λu) �v

= (Au) �v

= (Au)Tv

= (uTAT )v

= (uTA)v

= uT (Av)

= uT (µv)

= µuTv

= µu �v.

Henceλu �v = µu �v

and(λ− µ)u �v = 0.

Since λ− µ 6= 0, we have u �v = 0. �

Corollary. If A is an n × n symmetricmatrix with n distinct eigenvalues

λ1, . . . , λn

then the corresponding eigenvectors

v1, . . . ,vn

form an orthogonal basis of Rn.

An orthonormal basis in Rn is an or-thogonal basis

u1, . . . ,un

made of unit vectors,

‖ui‖ = 1 for all i = 1, 2, . . . , n.

Notice that this definition can reformu-lated as

ui�uj =

{1, if i = j0, if i 6= j

.

Theorem: Coordinates with re-spect to an orthonormal basis. If

{u1, . . . ,un }

is an orthonormal basis and v a vectorin Rn, then

v = x1u1 + · · ·+ xnun

where

xi = v �ui, for all i = 1, 2, . . . , n.

Proof. Let

v = x1u1 + · · ·+ xnun,

then, for i = 1, 2, . . . , n,

v �ui = x1u1�ui + · · ·+ xnun

�ui

= xiui�ui

= xi.

Theorem: Eigenvectors of symmet-ric matrices. Let A be a symmetricn× n matrix with n distinct eigenvalues

λ1, . . . , λn.

Then Rn has an orthonormal basis madeof eigenvectors of A.

MATH10212 • Linear Algebra B • Lecture 26 • Orthogonal matrices • Last change: 20 May 2019 42

Lecture 25: Orthogonal matrices

A square matrix A is said to be orthog-onal if

AAT = I.

Some basic properties of orthogonal ma-trices:

If A is orthogonal then

• AT is also orthogonal.

• A is invertible and A−1 = AT .

• detA = ±1.

• The identity matrix is orthogonal.

An example of an orthogonal matrix:

A =

[√22

√22√

22−√22

]

Theorem: Characterisation of Or-thogonal Matrices. For a square n×nmatrix A, the following conditions areequivalent:

(a) A is orthogonal .

(b) Columns of A form an orthormalbasis of Rn.

MATH10212 • Linear Algebra B • Lecture • Vector spaces • Last change: 20 May 2019 43

Lecture 26: Vector spaces

A vector space is a non-empty spaceof elements (vectors), with operations ofaddition and multiplication by a scalar.

By definition, the following holds for allu,v,w in a vector space V and for allscalars c and d.

1. The sum of u and v is in V .

2. u + v = v + u.

3. (u + v) + w = u + (v + w).

4. There is zero vector 0 such thatu + 0 = u.

5. For each u there exists −u suchthat u + (−u) = 0.

6. The scalar multiple cu is in V .

7. c(u + v) = cu + cv.

8. (c+ d)u = cu + du.

9. c(du) = (cd)u.

10. 1u = u.

Examples. Of course, the most impor-tant example of vector spaces is providedby standard spaces Rn of column vectorswith n components, with the usual op-erations of component-wise addition andmultiplication by scalar.

Another example is the set Pn of poly-nomials of degree at most n in variablex,

Pn = {a0 + a1x+ · · ·+ anxn}

with the usual operations of addition ofpolynomials multiplication by constant.

And here is another example: the setC[0, 1] of real-valued continuous func-tions on the segment [0, 1], with theusual operations of addition of functionsand multiplication by constant.

Definitions

Most concepts introduced in the previ-ous lectures for the vector spaces Rn canbe transferred to arbitrary vector spaces.Let V be a vector space.

Given vectors v1,v2, . . . ,vp in V andscalars c1, c2, . . . , cp, the vector

y = c1v1 + · · · cpvp

is called a linear combina-tion of v1,v2, . . . ,vp with weightsc1, c2, . . . , cp.

If v1, . . . ,vp are in V , then the set ofall linear combination of v1, . . . ,vp is de-noted by Span{v1, . . . ,vp} and is calledthe subset of V spanned (or gener-ated) by v1, . . . ,vp.

That is, Span{v1, . . . ,vp} is the collec-tion of all vectors which can be writtenin the form

c1v1 + c2v2 + · · ·+ cpvp

with c1, . . . , cp scalars.

Span{v1, . . . ,vp} is a vector subspaceof V , that is, it is closed with respect toaddition of vectors and multiplying vec-tors by scalars.

V is a subspace of itself.

{0 } is a subspace, called the zero sub-space.

An indexed set of vectors

{v1, . . . ,vp }

in V is linearly independent if the vec-tor equation

x1v1 + · · ·+ xpvp = 0

has only trivial solution.

MATH10212 • Linear Algebra B • Lecture • Vector spaces • Last change: 20 May 2019 44

The set{v1, . . . ,vp }

in V is linearly dependent if there ex-ist weights c1, . . . , cp, not all zero, suchthat

c1v1 + · · ·+ cpvp = 0

Theorem: Characterisation of lin-early dependent sets. An indexed set

S = {v1, . . . ,vp }

of two or more vectors is linearly depen-dent if and only if at least one of thevectors in S is a linear combination ofthe others.

A basis of V is a linearly independentset which spans V . A vector space Vis called finite dimensional if it hasa finite basis. Any two bases in a fi-nite dimensional vector space have thesame number of vectors; this number iscalled the dimension of V and is de-noted dimV .

If B = (b1, . . . ,bn) is a basis of V , everyvector x ∈ V can be written, and in aunique way, as a linear combination

x = x1b1 + · · ·xnbn;

the scalars x1, . . . , xn are called coordi-nates of x with respect to the basis B.

MATH10212 • Linear Algebra B • Lectures 23 and 24 • Linear transformations • Last change: 20 May 2019 45

Lectures 27: Linear transformations of abstract vector

spaces

A transformation T from a vectorspace V to a vector space W is a rulethat assigns to each vector x in V a vec-tor T (x) in W .

The set V is called the domain of T , theset W is the codomain of T .

A transformation

T : V −→ W

is linear if:

• T (u + v) = T (u) + T (v) for allvectors u,v ∈ V ;

• T (cu) = cT (u) for all vectors uand all scalars c.

Properties of linear transforma-tions. If T is a linear transformationthen

T (0) = 0

and

T (cu + dv) = cT (u) + dT (v).

The identity transformation of V is

V −→ V

x 7→ x.

The zero transformation of V is

V −→ V

x 7→ 0.

If

T : V −→ W

is a linear transformation, its kernel

KerT = {x ∈ V : T (x) = 0}

is a subspace of V , while its image

ImT = {y ∈ W : T (x) = y for some x ∈ V }

is a subspace of W .

MATH10212 • Linear Algebra B • Lectures 25 and 26 • Inner product spaces • Last change: 20 May 2019 46

Lectures 28 and 29: Inner product spaces

Inner (or dot) product is a functionwhich associate, with each pair of vec-tors u and v in a vector space V a realnumber denoted

u �v,

subject to the following axioms:

1. u �v = v �u

2. (u + v) �w = u �w + v �w

3. (cu) �v = c(u �v)

4. u �u > 0 and u �u = 0 if and onlyif u = 0

Inner product space. A vector spacewith an inner product is called an innerproduct space.

Example: School Geometry. The or-dinary Euclidean plane of school geome-try is an inner product space:

• Vectors: directed segments start-ing at the origin O

• Addition: parallelogram rule

• Product-by-scalar: stretching thevector by factor of c.

• Dot product:

u �v = ‖u‖‖v‖ cos θ,

where θ is the angle between vec-tors u and v.

Example: C[0, 1]. The vector spaceC[0, 1] of real-valued continuous func-tions on the segment [0, 1] becomes aninner product space if we define innerproduct by the formula

f � g =

∫ 1

0

f(x)g(x)dx.

In this example, the tricky bit is to showthat the dot product on C[0, 1] satisfiesthe axiom:

u �u > 0 and u �u = 0 if andonly if u = 0,

that is, if f is a continuous function on[0, 1] and ∫ 1

0

f(x)2dx = 0

then f(x) = 0 for all x ∈ [0, 1]. This re-quires the use of properties of continuousfunctions; since we study linear algebra,not analysis, I am leaving it to the read-ers as an exercise.

Length. The length of the vector u isdefined as

‖u‖ =√

u �u.

Notice that

‖u‖2 = u �u,

and‖u‖ = 0 ⇔ u = 0.

Orthogonality. We say that vectors uand v are orthogonal if

u �v = 0.

The Pythagoras Theorem. Vectorsu and v are orthogonal if and only if

‖u‖2 + ‖v‖2 = ‖u + v‖2.

Proof is a straightforward computation,exactly the same as in Lectures 25–27:

‖u + v‖2 = (u + v) � (u + v)

= u �u + u �v + v �u + v �v

= u �u + u �v + u �v + v �v

= u �u + 2u �v + v �v

= ‖u‖2 + 2u �v + ‖v‖2.


Hence

‖u + v‖2 = ‖u‖2 + ‖v‖2 + 2u �v,

and

‖u + v‖2 = ‖u‖2 + ‖v‖2

if and only if

u �v = 0.

�

The Cauchy-Schwarz Inequality. Inthe Euclidean plane,

u �v = ‖u‖‖v‖ cos θ

hence|u �v| 6 ‖u‖‖v‖.

The following Theorem shows that thisis is a general property of inner productspaces.

Theorem: The Cauchy-Schwarz In-equality. In any inner product space,

|u �v| 6 ‖u‖‖v‖.

The Cauchy-Schwarz Inequality:an example. In the vector space Rn

with the dot product of

u =

u1...un

and v =

v1...vn

defined as

u �v = uTv = u1v1 + · · ·+ unvn,

the Cauchy-Schwarz Inequality becomes

|u1v1 + · · ·+ unvn| 6√u21 + · · ·+ u2n ·

√v21 + · · ·+ v2n.

or, which is its equivalent form,

(u1v1 + · · ·+ unvn)2 6 (u21 + · · ·+ u2n) · (v21 + · · ·+ v2n).

The Cauchy-Schwarz Inequality:one more example. In the inner prod-

uct space C[0, 1] the Cauchy-Schwarz In-equality takes the form

∣∣∣∣∫ 1

0

f(x)g(x)dx

∣∣∣∣ 6√∫ 1

0

f(x)2dx ·

√∫ 1

0

g(x)2dx,

or, equivalently, (∫ 1

0

f(x)g(x)dx

)2

6

(∫ 1

0

f(x)2dx

)·(∫ 1

0

g(x)2dx

).

Proof of the Cauchy-Schwarz In-equality given here is different from theone in the textbook by Lay. Our proof isbased on the following simple fact fromschool algebra:

Let

q(t) = at2 + bt+ c

be a quadratic function invariable t with the property


thatq(t) > 0

for all t. Then

b2 − 4ac 6 0.

Now consider the function q(t) in vari-able t defined as

q(t) = (u + tv) � (u + tv)

= u �u + 2tu �v + t2v �v

= ‖u‖2 + (2u �v)t+ (‖v‖2)t2.

Notice q(t) is a quadratic function in tand

q(t) > 0.

Hence

(2u �v)2 − 4‖u‖2‖v‖2 6 0

which can be simplified as

|u �v| 6 ‖u‖‖v‖.

�

Distance. We define distance betweenvectors u and v as

d(u,v) = ‖u− v‖.

It satisfies axioms of metric:

1. d(u,v) = d(v,u).

2. d(u,v) > 0.

3. d(u,v) = 0 iff u = v.

4. The Triangle Inequality:

d(u,v) + d(v,w) > d(u,w).

Axioms 1–3 immediately follow from theaxioms for dot product, but the TriangleInequality requires some attention.

Proof of the Triangle Inequality. Itsuffices to prove

‖u + v‖ 6 ‖u‖+ ‖v‖,

or‖u + v‖2 6 (‖u‖+ ‖v‖)2,

or

(u+v) � (u+v) 6 ‖u‖2+2‖u‖‖v‖+‖v‖2,

or

u �u+2u �v+v �v 6 u �u+2‖u‖‖v‖+v �v,

or2u �v 6 2‖u‖‖v‖

But this is the Cauchy-Schwarz Inequal-ity. Now observe that all rearrange-ments are reversible, hence the TriangleInequality holds. �

MATH10212 • Linear Algebra B • Lecture 27 •Orthogonalisation and the Gram-Schmidt Process • Last change: 20 May 201949

Lecture 27: Orthogonalisation and the Gram-Schmidt

Process

Orthogonal basis. A basis v1, . . . ,vn

in the inner product space V is calledorthogonal if vi

�vj = 0 for i 6= j.

Coordinates in respect to an or-thogonal basis:

u = c1v1 + · · ·+ cnvn

iff

ci =u �vi

vi�vi

Orthonormal basis. A basisv1, . . . ,vn in the inner product space V

is called orthonormal if it is orthogonaland ‖vi‖ = 1 for all i.

Equivalently,

vi�vj =

{1 if i = j0 if i 6= j

.

Coordinates in respect to an or-thonormal basis:

u = c1v1 + · · ·+ cnvn

iffci = u �vi

The Gram-Schmidt Orthogonalisation Process makes an orthogonal basis from abasis x1, . . . ,xp:

v1 = x1

v2 = x2 −x2�v1

v1�v1

v1

v3 = x3 −x3�v1

v1�v1

v1 −x3�v2

v2�v2

v2

...

vp = xp −xp�v1

v1�v1

v1 −xp�v2

v2�v2

v2 − · · · −xp�vp−1

vp−1 �vp−1vp−1

Main Theorem about inner prod-uct spaces. Every finite dimensionalinner product space has an orthonormalbasis and is isomorphic to on of Rn with

the standard dot product

u �v = uTv =[u1 · · · un

] v1...vn

.

MATH10212 • Linear Algebra B • Lectures 28–30 • Revision • Last change: 20 May 2019 50

Lecture 30: Revision, Systems of Linear Equations

Systems of linear equations

Here A is an m× n matrix.

We associate with A systems of simul-taneous linear equations

Ax = b,

where b ∈ Rm and

x =

x1...xn

is the column vector of unknowns.

Especially important is the homoge-neous system of simultaneous linearequations

Ax = 0.

The set

null A = {x ∈ Rn : Ax = 0}

is called the null space of A; it is a vec-tor subspace of Rn and coincides withthe solution space of the homogeneoussystem Ax = 0.

If a1, . . . , an are columns of A, so that

A =[a1 · · · an

],

then the column space of A is definedas

Col A = {c1a1 + · · ·+ cnan : ci ∈ R}= Span {a1, . . . , an}

and equals to the span of the system ofvectors a1, . . . , an, that is, the set of allpossible linear combinations of vec-tors a1, . . . , an.

The column space of A has another im-portant characterisation:

Col A = {b ∈ Rm : Ax = b has a solution}.

Assume now that A is a square n×nmatrix.

If we associate with A the linear trans-formation

TA : Rn −→ Rn

v 7→ Av

then the column space of A gets yet an-other interpretation: it is the image ofthe linear transformation TA:

Col A = Im TA

= {v ∈ Rn : ∃v ∈ Rn s.t. TA(w) = v},

and the null space of A is the kernel ofTA:

null A = kerTA

= {v ∈ R : TA(v) = 0}.


Linear dependence

If a1, . . . , ap are vectors in a vector spaceV , an expression

c1a1 + · · ·+ cpap, ci ∈ Ris called a linear combination of thevectors a1, . . . , ap, and coefficients ci arecalled weights in this linear combina-tion. If

c1a1 + · · ·+ cpap = 0

we say that weights c1, . . . , cp de-fine a linear dependency of vectorsa1, . . . , ap. We always have a trivial(or zero) dependency, in which allci = 0. We say that vectors a1, . . . , ap

are linearly dependent if the admit anon-zero (or non-trivial) linear depen-dency, that is, a dependency in which atleast one weight ci 6= 0.

Let us arrange the weights c1, . . . , cp in acolumn

c =

c1...cp

and form a matrix A using vectorsa1, . . . , ap as columns, then

c1a1 + · · ·+ cpap =[a1 · · · ap

] c1...cp

= Ac

and linear dependencies between vec-tors a1, . . . , ap are nothing else but solu-tions of the homogeneous system of lin-ear equation

Ax = 0.

Therefore vectors a1, . . . , ap are linearlydependent if and only if the system

Ax = 0

has a non-zero solution.

Elementary row transformation

Elementary row transformations

Ri ↔ Rj, i 6= j

Ri ← Ri + λRj, i 6= j

Ri ← λRi, λ 6= 0

performed on a matrix A amount to mul-tiplication of A on the left by corre-sponding elementary matrices,

A← EA,

and therefore do not change dependen-cies between columns:

Ac = 0⇔ (EA)c = 0.

Therefore if certain columns of A werelinearly dependent (respectively, inde-pendent) then the same columns of EAremain linearly dependent (respectively,independent)

Rank of a matrix

We took it without proof that if U is avector subspace of Rn, then the maximalsystems of linearly independent subsetsin U contain the same number of vectors.This number is called the dimension of

U and is denoted dimU .

A maximal system of linearly indepen-dent vectors in U is called a basis of U .

Each basis of U contains dimU vectors.


An important theorem:

dim null A + dim Col A = n

dim Col A is called the rank of A and isdenoted

rankA = dim Col A.

Therefore

dim null A + rank A = n

.

Theorem. If rankA = p then amongcolumns of A there are p linearly inde-pendent columns

Theorem. Since elementary row trans-formations of A do not change depen-dency of columns, e=elementary row op-eration do not change rankA.

math10212 linear algebra lecture notes...math10212 linear algebra b lecture 1 linear systems last...

Documents