linear algebra - ise.ufl.edu · c guan/linear algebra/pg 23 1. vectors 2. matrices 3. systems of...

Linear Algebra

Overviewc© Guan/Linear Algebra/pg 2

1. Vectors

2. Matrices

3. Systems of Linear Equations

4. The Gauss-Jordan Method

5. Linear Independence

6. The Inverse of a Matrix

7. Determinants

Table of Contentsc© Guan/Linear Algebra/pg 3

1. Vectors

2. Matrices





7. Determinants

Algebrac© Guan/Linear Algebra/pg 4

Algebra

I The part of mathematics in which letters and other generalsymbols are used to represent numbers and quantities informulas and equations, e.g., x + 2 = 5.

I Algebra gives methods for writing formulas and solvingequations, as compared to arithmetic approach.

Linear Algebrac© Guan/Linear Algebra/pg 5

Linear Algebra is similar to the algebra, except that in the place ofordinary single numbers, it deals with vectors and linear operations(e.g., addition and scalar multiplication).

Linear Algebra studies

I linear equations such as a1x1 + · · ·+ anxn = b,

I linear maps such as (x1, . . . , xn) 7→ a1x1 + . . .+ anxn,

I and their representations in vector spaces and throughmatrices.

Scalar and Vectorc© Guan/Linear Algebra/pg 6

I Scalar: A scalar is a number. For instance, a magnitude butno “direction,” other than perhaps positive or negative.

I Vector: A vector is a list of numbers.I One way: A vector is a point in a space. This list of numbers

is a way of identifying that point in space.I Another way: A vector is a magnitude and a direction, it is a

directed arrow pointing from the origin to the end point givenby the list of numbers.

I Examples:

Vectorc© Guan/Linear Algebra/pg 7

I Usually written in a bold letter or a small arrow overtop of thesymbol, e.g., a and ~a.

I The total number of the list of numbers in the vector is calledthe dimension of the vector.

I e.g., a1 =

231

is a 3-dimensional vector.

I e.g., a2 = 0 =

0000

is a 4-dimensional vector. It is also a

zero vector.

Vectorc© Guan/Linear Algebra/pg 8

Column vector vs Row vector:

I By default, a vector is referred as a column vector.

I For a (column) vector u, its transpose is a row vector uᵀ.

Vector Operationsc© Guan/Linear Algebra/pg 9

Vectors Addition and SubtractionFor u = [u1, u2, . . . , um]ᵀ and v = [v1, v2, . . . , vm]ᵀ,

I u + v =

I u− v =

Parallelogram and Triangular Method

Scalar Multiplication:

I ku =


Scalar Product of Two Vectors (also called dot product)

I The scalar product of two vectors u = [u1, u2, . . . , um]ᵀ andv = [v1, v2, . . . , vm]ᵀ (denoted as u · v) is the numberu1v1 + u2v2 + . . .+ umvm.


Scalar Product of Two Vectors

I Two vectors are perpendicular (orthogonal) if and only if theirscalar product equals to 0.

I Also u · v = ||u||||v|| cos θ where ||u|| is the length of thevector u and θ is the angle between vectors u and v.

Vector Space and Axiomsc© Guan/Linear Algebra/pg 12

Vector Space: A vector space is a collection of vectors, which maybe added together and multiplied (“scaled”) by numbers.

1 Associativity of addition: u + (v + w) = (u + v) + w

2 Commutativity of addition: u + v = v + u

3 Identity element of addition: There exists an element 0 ∈ V, called thezero vector, such that v + 0 = v for all v ∈ V

4 Inverse elements of addition: For every v ∈ V, there exists an element−v ∈ V, called the additive inverse of v, such that v + (−v) = 0

5 Compatibility of scalar multiplication with field multiplication:a(bv) = (ab)v

6 Identity element of scalar multiplication: 1v = v

7 Distributivity of scalar multiplication with respect to vector addition:a(u + v) = au + av

8 Distributivity of scalar multiplication with respect to field addition:(a + b)v = av + bv

For example, R2 is a vector space. Note that every vector spacecontains vector 0.


1. Vectors

2. Matrices





7. Determinants

Matrixc© Guan/Linear Algebra/pg 14

I A matrix is any rectangular array of numbers.

I If a matrix A has m rows and n columns, we call A a m × nmatrix and m × n is referred as the order of matrix A.

A =

a11 a12 . . . a1na21 a22 . . . a2n

......

...am1 am2 . . . amn

I The number in the ith row and jth column of A is called the

ijth element of A and is written as aij .

Matrix Operationsc© Guan/Linear Algebra/pg 15

I The Scalar Multiple of a Matrix (multiplying each element ofA by the scalar)

I Addition/subtraction of Two Matrices in the same order(add/subtract the corresponding elements)

Matrix Operationsc© Guan/Linear Algebra/pg 16

I The transpose of a Matrix: switch the row and column indicesof the matrix (note (Aᵀ)ᵀ = A)

I Matrix Multiplication: C = ABI Condition: Number of columns in A = number of rows in B.I ij element of C =scalar product of row i of A × column j of B.I If A is m × r and B is r × n, then C is m × n.

Matrix: A linear operatorc© Guan/Linear Algebra/pg 17

I A matrix does things (linear transformation) to Vectors:transform a vector to another vector.

I Linear transformation can be considered a type of function:mapping a vector to another vector.

I Examples:

A =

[0 1−1 0

]rotate 90 degree clockwise.

A =

[cos θ sin θ− sin θ cos θ

]rotate θ degree clockwise.

Properties of Matrix Multiplication and EXCELc© Guan/Linear Algebra/pg 18

I Row i of AB = (row i of A)B

I Column j of AB = A(column j of B)

I Matrix multiplication is associative. That is, A(BC ) = (AB)C

I Matrix multiplication is distributive. That is,A(B + C ) = AB + AC and (B + C )D = BD + CD

Matrix Multiplication with EXCEL: Input Matrices A (D2:F3) andB (D5:E7) and use array function “MMULT(D2:F3, D5:E7)”(Control Shift Enter)


1. Vectors

2. Matrices





7. Determinants

The Linear Equation Systemc© Guan/Linear Algebra/pg 20

The linear equation system

a11x1 + a12x2 + . . . + a1nxn = b1a21x1 + a22x2 + . . . + a2nxn = b2

......

... =...

am1x1 + am2x2 + . . . + amnxn = bm.

It can be written as

Ax = b (matrix representation) or A|b (augmented matrix), where

A =

a11 a12 . . . a1na21 a22 . . . a2n

......

...am1 am2 . . . amn

, x =

x1x2...xn

, b =

b1b2...bn

An Examplec© Guan/Linear Algebra/pg 21

The Linear Equation Systemc© Guan/Linear Algebra/pg 22

All three cases for a linear equation system:

I The system has no solution.

I The system has a unique solution.

I The system has in infinite number of solutions.


1. Vectors

2. Matrices





7. Determinants

Elementary Row Operations (EROs)c© Guan/Linear Algebra/pg 24

For a given matrix A, we can use EROs to yield a new matrix A′

through one of three procedures:

1: Obtain A′ by multiplying any row of A by a nonzero scalar.

2: Obtain A′ by multiplying a row by a nonzero scalar and addthe result to another row.

3: Obtain A′ by interchanging any two rows of A.

After ERO, A and A′ are row equivalent and the correspondinglinear equation systems are equivalent.

Reduced Row Echelon Formc© Guan/Linear Algebra/pg 25

A matrix is in reduced row echelon form if it satisfies the followingconditions:

I All rows consisting of only zeroes are at the bottom.

I The leading entry (also called the pivot) of a nonzero row isalways strictly to the right of the leading entry of the rowabove it.

I The leading entry in each nonzero row is a 1 (i.e., a leading 1).

I Each column containing a leading 1 has zeros in all its otherentries. 1 0 a1 0 b1

0 1 a2 0 b20 0 0 1 b3

Remark: The leading entry of each row of a matrix is the leftmost

non-zero element of that row.

The Gauss-Jordan Methodc© Guan/Linear Algebra/pg 26

Purpose: Use EROs for A|b to finally transform A into a newmatrix A′|b′ in reduced row echelon form.

1. To solve Ax = b, write down the augmented matrix A|b2. Use ERO type 3 to swap the rows so that all rows with all

zero entries are on the bottom

3. Use ERO type 3 to swap the rows so that the row with theleftmost nonzero entry is on top

4. Use ERO type 1 to multiply the top row by a scalar so thattop row’s leading entry becomes 1.

5. Use ERO type 2 to add/subtract multiples of the top row tothe other rows so that all other entries in the columncontaining the top row’s leading entry are all zero.

6. Repeat steps 3-5 for the next leftmost nonzero entry until allthe leading entries are 1.

7. Double-check if the leading entry of each nonzero row is tothe right of the leading entry of the row above it. If not, useERO type 3 to swap the rows.


After the Gauss-Jordan method is applied, the variables aregrouped into two categories:

I Basic variable: a variable that appears with a coefficient of 1in a single equation and a coefficient of 0 in all otherequations.

I Nonbasic variable: any variable that is not a basic variable.


I Case 1: A′x = b′ contains at least one row of the form

[0 0 . . . 0|c](c 6= 0).

In this case, Ax = b has no solution.

I Case 2: If Case 1 does not hold and there are no nonbasicvariables, then Ax = b will have a unique solution.

I Case 3: If Case 1 does not hold and there is at least onenonbasic variable, then Ax = b will have an infinite number ofsolutions.


1. Vectors

2. Matrices





7. Determinants

Linear Independencec© Guan/Linear Algebra/pg 31

I A linear combination of the vectors in a vector space V is anyvector of the form

c1v1 + c2v2 + . . .+ ckvk

where c1, . . . , ck are arbitrary scalars.

I Trivial linear combination: c1 = c2 = . . . ck = 0.

I A set V of vectors is linearly independent if the trivial linearcombination is the only combination to make it equal to 0.Otherwise, it is linearly dependent.

I Example: Any set of vectors containing the 0 vector is alinearly dependent set.

The Rank of a Matrixc© Guan/Linear Algebra/pg 32

I Let A be a m ×m matrix and denote the rows of A byr1, r2, . . . , rm. Define R = {r1, r2, . . . , rm}.

I The rank of A is the number of vectors in the largest linearlyindependently subset of R.

I Examples:

A =

[0 00 0

]B =

[2 23 3

]C =

[0 11 0

]

Gauss-Jordan Method to Find Rank of Matrixc© Guan/Linear Algebra/pg 33

I For a matrix A, apply Gauss-Jordan method to obtain thefinal matrix A.

I It can be shown Rank(A)= Rank(A) = number of nonzerorows in A.

I Method to show if a set of vectors is linearly independent

I Given a set of vectors V = {v1, v2, . . . , vm}I Create a matrix A with the ith row to be vi . Thus, A has m

rows.I Use Gauss-Jordan to find rank A.I If rank A = m (referred as full rank), then V is linearly

independent. Otherwise, i.e., rank A < m, V is linearlydependent.


1. Vectors

2. Matrices





7. Determinants

Inverse of a Matrixc© Guan/Linear Algebra/pg 35

I A square matrix

I Diagonal elements of a square matrix

I Identity Matrix


I require a square matrix A is (m ×m).

I require full rank, i.e., Rank(A)=m.

I If AB = BA = Im, then B is the inverse of A, denoted asB = A−1.

I Link to reciprocal for real numbers.

I Inverting Matrices with EXCEL (“MINVERSE”)


I Gauss-Jordan method to get A−1:I Step 1: Create the m × 2m matrix A|Im.I Step 2: Use EROs to transform A|Im to Im|B. Then B = A−1.

If Rank(A) < m, then A has no inverse.


1. Vectors

2. Matrices





7. Determinants

Determinantc© Guan/Linear Algebra/pg 39

I require a square matrix A is (m ×m).

I the determinant of A is denoted as detA or |A|.I For a 1× 1 matrix, detA = a11.

I For a 2× 2 matrix, detA = a11a22 − a21a12.


I For a general m ×m matrix, use

detA = (−1)i+1ai1(detAi1)+(−1)i+2ai2(detAi2)+. . .+(−1)i+maim(detAim)

Here Aij is the ijth minor of a, which is the (m− 1)× (m− 1)matrix obtained from A after deleting the ith row and jthcolumn of A.


Geometrically, determinant can be viewed as the signed volumescaling factor of the linear transformation described by the matrix.

linear algebra - ise.ufl.edu · c guan/linear algebra/pg 23 1. vectors 2. matrices 3. systems of...

Documents