Matrix (mathematics)

Specific entries of a matrix are often referenced by using pairs of subscripts.

In mathematics, a matrix (plural matrices, or less commonly matrixes) is a rectangular array of numbers, such as

(OR)

A matrix is a rectangular array of elements. If it has m rows and n columns then the matrix is called an m n× matrix. It is called the size of this matrix. Let A be an m n× matrix, the element on the ith row and jth column is called the (i, j)th entry of the matrix. Notation: Let A be an m n× matrix. We will denote it by A=( )i ja , where ija is

the (i, j)th entry of A.

Definition

A matrix is a rectangular arrangement of numbers.[1] For example,

An alternative notation uses large parentheses instead of box brackets:

The horizontal and vertical lines in a matrix are called rows and columns, respectively. The numbers in the matrix are called its entries or its elements. To specify a matrix's size, a matrix with m rows and n columns is called an m-by-n matrix or m × n matrix, while m and n are called its dimensions. The above is a 4-by-3 matrix.

A matrix where one of the dimensions equals one is also called a vector, and may be interpreted as an element of real coordinate space. An m × 1 matrix (one column and m rows) is called a column vector and a 1 × n matrix (one row and n columns) is called a row vector. For example, the second row vector of the above matrix A is

Notation

A common shorthand is A = [ai,j] i=1,...,m; j=1,...,n or more briefly A = [ai,j]m×n

to define an m × n matrix A. Usually the entries ai,j are defined separately for all integers 1 ≤ i ≤ m and 1 ≤ j ≤ n. They can however sometimes be given by one formula; for example the 3-by-4 matrix

can alternatively be specified by A = [i − j] i=1,2,3; j=1,...,4.

Some programming languages start the numbering of rows and columns at zero, in which case the entries of an m-by-n matrix are indexed by 0 ≤ i ≤ m − 1 and 0 ≤ j ≤ n − 1.[2] This article follows the more common convention in mathematical writing where enumeration starts from 1

EQUAL MATRICES: Two matrices are said to be equal if they are of the same size and their (i, j)th entries are all equal.

Basic operations(Algebraic operations on matrices) Main articles: Matrix addition, Scalar multiplication, Transpose, and Row operations

There are a number of operations that can be applied to modify matrices called matrix addition, scalar multiplication and transposition.[3] These form the basic techniques to eal with matrices.

Definition: (Matrix addition) Let ( ) ( ),i j i jA a B b= = be two m n× matrices of

numbers. Then, A B+ is an m n× matrix such that its (i, j)th entry is i j i ja b+ i.e.

( ) ( ) ( )i j i j i j i ja b a b+ = + .

Example: 1 6 1 2 3 3 3 9 2

0 1 5 1 0 1 1 1 6

− + = − −

Definition: (Scalar multiplication) Let ( )i jA a= be an m n× matrix of numbers

and let abe another number. Then, kA, the scalar multiplication of the number a to the matrix A , is an m n× matrix such that its (i, j)th entry is

i jk a⋅ i.e. ( ) ( )i j i jk a k a= ⋅ .

Example: 1 1 0 3 3 0

32 5 7 6 15 21

− − =

, ( ) ( )4 1 3 5 7 4 12 20 28= .

2 4

2 4 8

6 12

=

.

Remark: Let ( )i jB b= . Then, B− is defined to be ( )i jb− . Note that ( 1)B B− = − .

A B− is defined to be ( ) ( ) ( )i j i j i j i ja b a b− = − .

Addition and scalar multiplication satisfy the following laws. Theorem: Let , ,A B Cbe matrices of the same size and let ,k k′be numbers. Let 0denote the zero matrix of the same size. Then, (i) Associative Law for Addition: ( ) ( )A B C A B C+ + = + + (ii) Commutative Law for Addition: A B B A+ = + (iii) Neutral Element for Addition: 0 0A A+ = + (iv) Existence of Inverse: Let ( 1)A A− = − . Then, ( ) ( ) 0A A A A+ − = − + = (v) Distributive Law: ( )k A B kA kB+ = + , ( )k k A kA k A′ ′+ = + (vi) ( ) ( )kk A k k A′ ′= (vii) 1A A= . Definition: (Matrix multiplication) Let ( )ikA a= be an m r× matrix and let

( )pqB b= be an r n× matrix. Then, we can define ABas an m n× matrix such

that its (i, j)th entry is equal to 1 1 2 21

l r

i j i j ir rj il ljl

a b a b a b l a b=

=

+ + + = =∑L .

Remark: We can form the product ABwhenever the number of columns of Aequal to the number of rows of B . To compute the (i, j)th entry of AB , we use the ith row of Aand the jth column of B .

Example: 1 0 3 0 1 0 0 0 1 0 0

1 2 2 1 1 0 1 2 3 0 4

0 5 1 0 0 0 1 5 5 0 6

− − = − −

Theorem: Let , ,A B Cbe matrices. Then, whenever the products and sums are defined, (i) Associative Law: ( ) ( )AB C A BC= (ii) Distributive Law: ( )A B C AB AC+ = + , ( )B C A BA CA+ = + . (iii) ( ) ( ) ( )k AB kA B A kB= = , where k is a number.

Familiar properties of numbers extend to these operations of matrices: for example, addition is commutative, i.e. the matrix sum does not depend on the order of the summands: A + B = B + A.[4] The transpose is compatible with addition and scalar multiplication, as expressed by (cA)T = c(AT) and (A + B)T = AT + BT. Finally, (AT)T = A.

Row operations are ways to change matrices. There are three types of row operations: row switching, that is interchanging two rows of a matrix, row multiplication, multiplying all entries of a row by a non-zero constant and finally row addition which means adding a multiple of a row to another row. These row operations are used in a number of ways including solving linear equations and finding inverses.

Row vector

In linear algebra, a row vector or row matrix is a 1 × n matrix, that is, a matrix consisting of a single row:[1]

The transpose of a row vector is a column vector:

Square matrices

A square matrix is a matrix which has the same number of rows and columns. An n-by-n matrix is known as a square matrix of order n.

1 5

7 2

is a 2 2× square matrix.

Any two square matrices of the same order can be added and multiplied. A square matrix A is called invertible or non-singular if there exists a matrix B such that

AB = I n.[13]

This is equivalent to BA = I n.[14] Moreover, if B exists, it is unique and is called the

inverse matrix of A, denoted A−1.

Diagonal matrix In linear algebra, a diagonal matrix is a square matrix in which the entries outside the main

diagonal (↘) are all zero. The diagonal entries themselves may or may not be zero. Thus, the

matrix D = (di,j) with n columns and n rows is diagonal if:

For example, the following matrix is diagonal:

The term diagonal matrix may sometimes refer to a rectangular diagonal matrix , which is an m-

by-n matrix with only the entries of the form di,i possibly non-zero; for example,

, or

The entries Ai,i form the main diagonal of a matrix. The trace, tr(A) of a square matrix A is the sum of its diagonal entries. While, as mentioned above, matrix multiplication is not commutative, the trace of the product of two matrices is independent of the order of the factors: tr(AB) = tr(BA).[15]

ZERO MATRIX:

Definition: The matrix with all entries equal to zero is called a zero matrix. An n n× matrix ( )i ja such that 1i ja = if i j= and 0i ja = otherwise is called the

identity matrix of order n. It is usually denoted by nI .

Examples: 0 0 0 0 0

0 0 0 0 0

0 0 0 0 0

is the 3 5× zero matrix. 2 3

1 0 01 0

, 0 1 00 1

0 0 1

I I

= =

.

Remark: We will not have special notation for zero matrix. Usually, we just denote it by 0 . Its size is determined by the context.

Identity matrix In linear algebra, the identity matrix or unit matrix of size n is the n-by-n square matrix with

ones on the main diagonal and zeros elsewhere. It is denoted by In, or simply by I if the size is

immaterial or can be trivially determined by the context. (In some fields, such as quantum

mechanics, the identity matrix is denoted by a boldface one, 1; otherwise it is identical to I.)

Some mathematics books use U and E to represent the Identity Matrix (meaning "Unit Matrix"

and "Elementary Matrix", or from the German "Einheitsmatrix",[1] respectively), although I is

considered more universal.

The important property of matrix multiplication of identity matrix is that for m-by-n A

Involutory matrix In mathematics, an involutory matrix is a matrix that is its own inverse. That is, matrix A is an

involution iff A2 = I. One of the three classes of elementary matrix is involutory, namely therow-

interchange elementary matrix. A special case of another class of elementary matrix, that which

represents multiplication of a row or column by −1, is also involutory; it is in fact a trivial example

of a signature matrix, all of which are involutory.

Involutory matrices are all square roots of the identity matrix. This is simply a consequence of the

fact that any nonsingular matrix multiplied by its inverse is the identity. If A is an n × nmatrix,

then A is involutory if and only if ½(A + I) is idempotent.

An involutory matrix which is also symmetric is an orthogonal matrix, and thus represents

an isometry (a linear transformation which preserves Euclidean distance). A reflection matrix is an

example of an involutory matrix.

Nilpotent matrix

In linear algebra, a nilpotent matrix is a square matrix N such that

for some positive integer k. The smallest such k is sometimes called the degree of N.

Examples: The matrix

is nilpotent, since M2 = 0. More generally, any triangular matrix with 0's along the main diagonal is nilpotent.

For example, the matrix

is nilpotent, with

Though the examples above have a large number of zero entries, a typical nilpotent matrix does not. For example, the matrices

both square to zero, though neither matrix has zero entries.

Triangular matrix

In the mathematical discipline of linear algebra, a triangular matrix is a special kind of square matrix where the entries either below or above the main diagonal are zero. Because matrix equations with triangular matrices are easier to solve they are very important in numerical analysis. The LU decomposition gives an algorithm to decompose any invertible matrix A into a normed lower triangle matrix L and an upper triangle matrix U.

A matrix of the form

is called lower triangular matrix or left triangular matrix , and analogously a matrix of the form

is called upper triangular matrix or right triangular matrix .

If all entries outside the main diagonal are zero, A is called a diagonal matrix. If only all entries above (below) the main diagonal are zero, A is called a lower triangular matrix (upper triangular matrix, respectively). For example, if n = 3, they look like

(diagonal), (lower) and (upper triangular matrix).

The standard operations on triangular matrices conveniently preserve the triangular form: the sum and product of two upper triangular matrices is again upper triangular. The inverse of an upper triangular matrix is also upper triangular, and of course we can multiply an upper triangular matrix by a constant and it will still be upper triangular. This means that the upper triangular matrices form a subalgebra of the ring of square matrices for any given size. The analogous result holds for lower triangular matrices. Note, however, that the product of a lower triangular with an upper triangular matrix does not preserve triangularity.

Tridiagonal matrix

In linear algebra, a tridiagonal matrix is a matrix that is "almost" a diagonal matrix. To be exact: a tridiagonal matrix has nonzero elements only in the main diagonal, the first diagonal below this, and the first diagonal above the main diagonal.

For example, the following matrix is tridiagonal:

A determinant formed from a tridiagonal matrix is known as a continuant.[1

Transpose

In linear algebra, the transpose of a matrix A is another matrix AT (also written A′, A tr or tA) created by any one of the following equivalent actions:

• write the rows of A as the columns of AT • write the columns of A as the rows of AT • reflect A by its main diagonal (which starts from the top left) to obtain AT

Formally, the (i,j) element of AT is the (j,i) element of A.

[AT] ij = [A] ji

If A is a m × n matrix then AT is a n × m matrix. The transpose of a scalar is the same scalar.

Example: Let1 7

1 0

2 3

A

= −

. Then,1 1 2

7 0 3TA

− =

.

NOTE: Let ,A Bbe matrices and k be a number. Then, whenever the sum and product are defined, (i) ( )T T TA B A B+ = + (ii) ( )T TA A= (iii) ( )T TkA kA=

(iv) ( )T T TAB B A= .

Symmetric matrix

In linear algebra, a symmetric matrix is a square matrix, A, that is equal to its transpose

The entries of a symmetric matrix are symmetric with respect to the main diagonal (top left to bottom right). So if the entries are written as A = (aij), then

for all indices i and j. The following 3×3 matrix is symmetric:

A matrix is called skew-symmetric or antisymmetric if its transpose is the same as its negative. The following 3×3 matrix is skew-symmetric:

Skew-symmetric matrix

In linear algebra, a skew-symmetric (or antisymmetric or antimetric [1]) matrix is a square matrix A whose transpose is also its negative; that is, it satisfies the equation:

or in component form, if : for all and

For example, the following matrix is skew-symmetric:

Compare this with a symmetric matrix whose transpose is the same as the matrix

or an orthogonal matrix, the transpose of which is equal to its inverse:

The following matrix is neither symmetric nor skew-symmetric:

Every diagonal matrix is symmetric, since all off-diagonal entries are zero. Similarly, each diagonal element of a skew-symmetric matrix must be zero, since each is its own negative.

Orthogonal matrix

In linear algebra, an orthogonal matrix is a square matrix with real entries whose columns (or rows) are orthogonal unit vectors (i.e., orthonormal). Because the columns are unit vectors in addition to being orthogonal, some people use the term orthonormal to describe such matrices.

Equivalently, a matrix Q is orthogonal if its transpose is equal to its inverse:

alternatively,

(OR) Definition: An n n× matrix A is called an orthogonal matrix whenever

TA A I= . EXAMPLE:

1 0 1 0 1 0 cos sin, , ,

0 1 0 1 0 1 sin cos

θ θθ θ

− − − − −

Conjugate transpose "Adjoint matrix" redirects here. An adjugate matrix is sometimes called a "classical adjoint matrix".

In mathematics, the conjugate transpose , Hermitian transpose , or adjoint matrix of an m-by-

n matrix A with complex entries is the n-by-m matrix A* obtained from A by taking

thetranspose and then taking the complex conjugate of each entry (i.e. negating their imaginary

parts but not their real parts). The conjugate transpose is formally defined by

where the subscripts denote the i,j-th entry, for 1 ≤ i ≤ n and 1 ≤ j ≤ m, and the overbar denotes a

scalar complex conjugate. (The complex conjugate of a + bi, where a and b are reals, isa − bi.)

This definition can also be written as

where denotes the transpose and denotes the matrix with complex conjugated entries.

Other names for the conjugate transpose of a matrix are Hermitian conjugate , or transjugate .

The conjugate transpose of a matrix A can be denoted by any of these symbols:

� or , commonly used in linear algebra

� (sometimes pronounced "A dagger"), universally used in quantum mechanics

� , although this symbol is more commonly used for the Moore-Penrose

pseudoinverse

In some contexts, denotes the matrix with complex conjugated entries, and thus the

conjugate transpose is denoted by or .

EXAMPLE:

then

Hermitian matrix A Hermitian matrix (or self-adjoint matrix ) is a square matrix with complex entries which is

equal to its own conjugate transpose – that is, the element in the ith row and jth column is equal

to the complex conjugate of the element in the jth row and ith column, for all indices i and j:

If the conjugate transpose of a matrix is denoted by , then the Hermitian property can be

written concisely as

Hermitian matrices can be understood as the complex extension of a real symmetric matrix.

For example,

is a Hermitian matrix

Skew-Hermitian matrix

In linear algebra, a square matrix with complex entries is said to be skew-Hermitian or antihermitian if its conjugate transpose is equal to its negative.[1] That is, the matrix A is skew-Hermitian if it satisfies the relation

where denotes the conjugate transpose of a matrix. In component form, this means that

for all i and j, where ai,j is the i,j-th entry of A, and the overline denotes complex conjugation.

Skew-Hermitian matrices can be understood as the complex versions of real skew-symmetric matrices, or as the matrix analogue of the purely imaginary numbers.[2]

Unitary matrix

In mathematics, a unitary matrix is an n by n complex matrix U satisfying the condition

where is the identity matrix in n dimensions and is the conjugate transpose (also called the Hermitian adjoint) of U. Note this condition says that a matrix U is unitary if and only if it has an inverse which is equal to its conjugate transpose

A unitary matrix in which all entries are real is an orthogonal matrix. Just as an orthogonal matrix G preserves the (real) inner product of two real vectors,

so also a unitary matrix U satisfies

for all complex vectors x and y, where stands now for the standard inner product on .

If is an n by n matrix then the following are all equivalent conditions:

1. is unitary 2. is unitary 3. the columns of form an orthonormal basis of with respect to this inner

product 4. the rows of form an orthonormal basis of with respect to this inner product 5. is an isometry with respect to the norm from this inner product 6. U is a normal matrix with eigenvalues lying on the unit circle.

Row operations : Let AX b= be a system of mequations in nunknowns. To solve this system, we apply three basic operations on the system of equations to get an equivalent but simpler system. The three operations are

(i) Multiply an equation by a nonzero number (ii) Interchange two equations (iii) Add to a certain equation a multiple of another equation.

(OR)

Three Elementary Row Operations

1. Switch any tow rows.

2. Multiply a row by a nonzero constant.

3. Replace a row by the sum of that row and a multiple of another row.

Two matrices are called row equivalent if one can be transformed into the other using a sequence of row operations. Since row operations do not effect the solution space, any two row equivalent matrices have the same solution space.

Elementary matrix In mathematics , an elementary matrix is a simple matrix which differs from the identity matrix in a minimal way. The elementary matrices generate the general linear group of invertible matrices , and left (respectively, right) multiplication by an elementary matrix represent elementary row operations (respectively, elementary column operations).

In algebraic K-theory, "elementary matrices" refers only to the row-addition matrices.

Use in solving systems of equations

Elementary row operations do not change the solution set of the system of linear equations represented by a matrix, and are used in Gaussian elimination (respectively, Gauss-Jordan elimination) to reduce a matrix to row echelon form (respectively, reduced row echelon form).

The acronym "ero" is commonly used for "elementary row operations".

Operations

There are three types of elementary matrices, which correspond to three types of row operations (respectively, column operations):

Row switching

A row within the matrix can be switched with another row.

Row multiplication

Each element in a row can be multiplied by a non-zero constant.

Row addition

A row can be replaced by the sum of that row and a multiple of another row.

The elementary matrix for any row operation is obtained by executing the operation on an identity

matrix.

(OR)

Definition: On any given matrix, the elementary row operations are listed in the following.

(i) Multiply a certain row by a nonzero number. (ii) Interchange two rows. (iii) Adding to a certain row a multiple of another row.

Remark: Elementary row operations are reversible.

Example

Use the elementary row operations to put the following in rref.

Solution

We follow the procedures:

Rank (linear algebra) The column rank of a matrix A is the maximal number of linearly independent columns of A.

Likewise, the row rank is the maximal number of linearly independent rows of A.

Since the column rank and the row rank are always equal, they are simply called the rank of A.

Alternative definitions

column rank – dimension of column space

The maximal number of linearly independent columns of the m×n matrix A with

entries in the field F is equal to the dimension of the column space of A (the column space being

the subspace of Fm generated by the columns of A, which is in fact just the image of A as a linear

map).

row rank – dimension of row space

Since the column rank and the row rank are the same, we can also define the rank of A as the

dimension of the row space of A, or the number of rows in a basis of the row space.

determinantal rank – size of largest non-vanishing minor

Another equivalent definition of the rank of a matrix is the greatest order of any non-zero minor in

the matrix (the order of a minor being the size of the square sub-matrix of which it is the

determinant). Like the decomposition rank characterization, this does not give an efficient way of

computing the rank, but it is useful theoretically: a single non-zero minor witnesses a lower bound

(namely its order) for the rank of the matrix, which can be useful to prove that certain operations

do not lower the rank of a matrix.

Properties

We assume that A is an m-by-n matrix over either the real numbers or the complex numbers, and

we define the linear map f by f(x) = Ax as above.

1. only a zero matrix has rank zero.

2. .

3. f is injective if and only if A has rank n (in this case, we say that A has full column rank).

4. f is surjective if and only if A has rank m (in this case, we say that A has full row rank).

5. In the case of a square matrix A (i.e., m = n), then A is invertible if and only if A has

rank n (that is, A has full rank).

6. If B is any n-by-k matrix, then

As an example of the "<" case, consider the product

Both factors have rank 1, but the product has rank 0.

7. If B is an n-by-k matrix with rank n, then

8. If C is an l-by-m matrix with rank m, then

9. The rank of A is equal to r if and only if there exists an invertible m-by-m matrix X and an

invertible n-by-n matrix Y such that

where Ir denotes the r-by-r identity matrix.

10. Sylvester’s rank inequality: If A is a m-by-n matrix and B n-by-k, then

.[5]

This is a special case of the next inequality.

11. The inequality due to Frobenius: if AB, ABC and BC are defined, then

.[6]

12. Subadditivity: when A and B are of the

same dimension. As a consequence, a rank-k matrix can be written as the sum of k rank-1

matrices, but not fewer.

13. The rank of a matrix plus the nullity of the matrix equals the number of columns of the matrix

(this is the "rank theorem" or the "rank-nullity theorem").

14. The rank of a matrix and the rank of its corresponding Gram matrix are equal

This can be shown by proving equality of their null spaces. Null space of the Gram matrix is given

by vectors x for which ATAx = 0. If this condition is fulfilled, also holds 0 = xTATAx = | Ax | 2.

This proof was adapted from.[7]

15. If A * denotes the conjugate transpose of A (i.e., the adjoint of A), then

.

Row echelon form A matrix is in reduced echelon form, if

1) in each row, the first nonzero entry is 1 and all the other entries in that column are zero.

2) a row in which all the entries are zero, is below all those rows which have nonzero entries.

3) if the rows r1 and r2 , r1 < r2 , have their first nonzero elements in the columns c1 and c2 , respectively, then c1 < c2 .

(OR)

Definition: Let ( )i jA a= be a matrix.A is said to be in row echelon form or called an row echelon matrix whenever

(i) All zero rows are on the bottom. (ii) For the ith row let ji be the first column with nonzero entry, then

j1<j2<j3<… It is said to be in row reduced echelon form or called a row reduced echelon matrix if in addition to (i), (ii) the following condition is satisfied.

(iii) For each nonzero row, the first nonzero entry from the left is 1 and it is the only nonzero entry in its column.

(OR)

Definition: Given a matrix A, its rank is defined to be the number of nonzero rows in its row echelon form.

(OR)

Definition

An m x n matrix is in reduced row echelon form if it satisfies the following properties:

1. All zero rows, if any, are at the bottom of the matrix

2. The first nonzero entry of each row is a one. This one is called the leading one or the corner.

3. Each corner is to the right and below the preceding corner.

4. The columns containing a leading one have zeros in all other entries.

If only 1, 2, and 3 are satisfied, then the matrix is in row echelon form.

Examples of Echelon and Row reduced echelon matrices:

2 3 2 0 4 5 6

0 0 1 1 3 2 0

0 0 0 0 0 6 2

0 0 0 0 0 0 0

− −

, 1 2 3

0 0 1

0 0 0

, 0 1 3 0 0 4

0 0 0 1 0 3

0 0 0 0 1 2

−

. The first two are

echelon matrices and the third one is a reduced echelon matrix. Theorem: Given any matrixA , one can reduce this matrix into echelon form or row reduced echelon form by applying elementary row operations.

Proposition 4. Every matrix A is equivalent to a unique matrix AR , which is in reduced echelon form. The rank of A = the rank of AR = the number of nonzero rows in AR .

Proof. Omitted.

Proposition 5. If A is an n x n matrix, then

the rank of A = n <=> AR = I .

Proof. 1) AR = I => the rank of A = n (= the number of nonzero rows) 2) The rank of A = n => the rank of AR = n => AR = I .

Proposition 6. If A is an n x n matrix, then the following statements are equivalent

(i) A -1 exists

(ii) the rank of A is n

(iii) AR = I

(iv) the rows(columns) of A are linearly independent.

Proof. Follows from Proposition 5 and results of the section 1.4.. (computation of an inverse: (A | I) ~ ··· ~ (AR | B), if AR = I , then B = A -1).

Proposition 1 => (ii) <=> (iv)

Example 1: Existence of an inverse

2) Solvability of linear systems of equations:

Proposition 7. A linear system of equations has solutions if and only the ranks of the coefficient matrix and the augmented coefficient matrix coincide.

Example: Reduce the following matrix to row reduced echelon form. 2 2 1 6 4 2 2 1 6 4 2 2 1 6 4

4 4 1 10 13 0 0 3 2 5 0 0 3 2 5

6 6 0 20 19 6 6 0 20 19 0 0 3 2 7

− − − → − → − →

2 2 1 6 4 2 2 1 6 4 2 2 1 0 1

0 0 3 2 5 0 0 3 2 5 0 0 3 2 5

0 0 0 4 2 0 0 0 2 1 0 0 0 2 1

− − − → − → − → − →

2 2 1 0 1 2 2 0 0 3 1 1 0 0 3 2

0 0 3 0 6 0 0 3 0 6 0 0 3 0 6

0 0 0 2 1 0 0 0 2 1 0 0 0 2 1

− → → → →

1 1 0 0 3 2 1 1 0 0 3 2

0 0 1 0 2 0 0 1 0 2

0 0 0 2 1 0 0 0 1 1 2

→ →

.

Examples

This matrix is in reduced row echelon form:

The following matrix is also in row echelon form, but not in reduced row form:

However, the matrix below is not in row echelon form, as the leading coefficient of row 3 is not

strictly to the right of the leading coefficient of row 2, and the main diagonal is not made up of only

ones:

Example: Find the rank of 1 2 2 3 2

2 4 3 4 5

5 10 8 11 12

− − −

.

Solution: Since the reduced echelon form is 1 2 0 1 4

0 0 1 2 1

0 0 0 0 0

− −

. The rank is

then equal to 2.

Non-uniqueness

Every non-zero matrix can be reduced to an infinite number of echelon forms (they can all be

multiples of each other, for example) via elementary matrix transformations. However, all

matrices and their row echelon forms correspond to exactly one matrix in reduced row echelon

form.

Systems of linear equations

A system of linear equations is said to be in row echelon form if its augmented matrix is

in row echelon form. Similarly, a system of equations is said to be in reduced row echelon

form or canonical form if its augmented matrix is in reduced row echelon form.

Augmented matrix Definition: A is called the coefficient matrix and the ( 1)m n× + matrix ( )A b is called the augmented matrix of the system of linear equations.

In linear algebra, the augmented matrix of a matrix is obtained by changing a matrix in some

way.

Given the matrices A and B, where:

Then, the augmented matrix (A|B) is written as:

This is useful when solving systems of linear equations or the augmented matrix may also be

used to find the inverse of a matrix by combining it with the identity matrix.

Gaussian elimination

In linear algebra, Gaussian elimination is an algorithm for solving systems of linear equations, finding the rank of a matrix, and calculating the inverse of an invertible square matrix. Gaussian elimination is named after German mathematician and scientist Carl Friedrich Gauss.

Elementary row operations are used to reduce a matrix to row echelon form. Gauss–Jordan elimination, an extension of this algorithm, reduces the matrix further to reduced row echelon form. Gaussian elimination alone is sufficient for many applications.

Gaussian elimination: Let AX b= be the matrix representation of a system of mequations in nunknowns. Apply elementary row operations to the augmented matrix ( )A b to reduce it to row reduced echelon form. If the

first nonzero entry of a nonzero row is on the last column then the system has no solution. Otherwise, we can apply backward substitution to get solutions of the system.

Example:1. Solve the following system by applying elementary row operations to the augmented matrix.

2 2 3 2

2 4 3 4 5

5 10 8 11 12

x y z w

x y z w

x y z w

+ − + =+ − + =+ − + =

Solution:

The augmented matrix is 1 2 2 3 2

2 4 3 4 5

5 10 8 11 12

− − −

. In the following, we apply

elementary row operations to this matrix reducing it to reduced echelon form.

1 2 2 3 2 1 2 2 3 2 1 2 2 3 2

2 4 3 4 5 0 0 1 2 1 0 0 1 2 1

5 10 8 11 12 0 0 2 4 2 0 0 0 0 0

− − − − → − → − → − −

1 2 0 1 4

0 0 1 2 1

0 0 0 0 0

− −

. This is the augmented matrix of the following system

2 4

2 1

x y w

z w

+ − =− =

. Hence, 4 2

1 2

x y w

z w

= − += +

Answer: 4 2 , , 1 2 , ,x s t y s z t w t= − + = = + = where ,s tare arbitrary.

Example:2. In practice, one does not usually deal with the systems in terms of equations but

instead makes use of the augmented matrix (which is also suitable for computer manipulations).

This, then, is the Gaussian Elimination algorithm applied to the augmented matrix of the system

above, beginning with:

which, at the end of the first part of the algorithm, looks like this:

That is, it is in row echelon form.

At the end of the algorithm, if the Gauss–Jordan elimination is applied:

That is, it is in reduced row echelon form, or row canonical form.

Gauss–Jordan elimination In linear algebra , Gauss–Jordan elimination is a version of Gaussian elimination that puts

zeros both above and below each pivot element as it goes from the top row of the given matrix

to the bottom. In other words, Gauss–Jordan elimination brings a matrix to reduced row

echelon form , whereas Gaussian elimination takes it only as far as row echelon form . Every

matrix has a reduced row echelon form, and this algorithm is guaranteed to produce it.

Gauss–Jordan elimination is considerably less efficient than Gaussian elimination

with backsubstitution when solving a system of linear equations. However, it is well suited for

calculating the matrix inverse.

It is named in after Carl Friedrich Gauss and Wilhelm Jordan, because it is a modification of

Gaussian elimination as described by Jordan in 1887. However, the method also appears in an

article by Clasen published in the same year. Jordan and Clasen probably discovered Gauss–

Jordan elimination independently (Althoen & McLaughlin 1987).

Finding the inverse of a matrix

Suppose A is a matrix and you need to calculate its inverse. The identity matrix is augmented to the right of A, forming a matrix (the block matrix B = [A,I]). Through application of elementary row operations and the Gaussian elimination algorithm, the left block of B can be reduced to the identity matrix I, which leaves A − 1 in the right block of B.

If the algorithm is unable to reduce A to triangular form, then A is not invertible.

In practice, inverting a matrix is rarely required. Most of the time, one is really after the solution of a particular system of linear equations.[3]

Application to finding inverses

If Gauss–Jordan elimination is applied on a square matrix, it can be used to calculate the

matrix's inverse. This can be done by augmenting the square matrix with the identity matrix of the

same dimensions, and through the following matrix operations:

If the original square matrix, A, is given by the following expression:

Then, after augmenting by the identity, the following is obtained:

By performing elementary row operations on the [AI] matrix until it reaches reduced row echelon

form, the following is the final result:

The matrix augmentation can now be undone, which gives the following:

or

A matrix is non-singular (meaning that it has an inverse matrix) if and only if the identity matrix

can be obtained using only elementary row operations.

System of linear equations General form

A general system of m linear equations with n unknowns can be written as

Here are the unknowns, are the coefficients of the

system, and are the constant terms.

Often the coefficients and unknowns are real or complex numbers, but integers and rational

numbers are also seen, as are polynomials and elements of an abstract algebraic structure.

Matrix equation

The vector equation is equivalent to a matrix equation of the form

where A is an m×n matrix, x is a column vector with n entries, and b is a column vector

with m entries.

The number of vectors in a basis for the span is now expressed as the rank of the matrix.

Solution set

The solution set for the equationsx − y = −1 and 3x + y = 9 is the single point (2, 3).

A solution of a linear system is an assignment of values to the variables x1, x2, ..., xn such that

each of the equations is satisfied. The set of all possible solutions is called the solution set .

A linear system may behave in any one of three possible ways:

1. The system has infinitely many solutions.

2. The system has a single unique solution.

3. The system has no solution.

Consistency

The equations 3x + 2y = 6 and3x + 2y = 12 are inconsistent.

The equations of a linear system are consistent if they possess a common solution,

and inconsistent otherwise. When the equations are inconsistent, it is possible to derive

a contradiction from the equations, such as the statement that 0 = 1.

For example, the equations

are inconsistent. In attempting to find a solution, we tacitly assume that there is a solution; that is,

we assume that the value of x in the first equation must be the same as the value of x in the

second equation (the same is assumed to simultaneously be true for the value of y in both

equations). Applying the substitution property (for 3x+2y) yields the equation 6 = 12, which is a

false statement. This therefore contradicts our assumption that the system had a solution and we

conclude that our assumption was false; that is, the system in fact has no solution . The graphs

of these equations on the xy-plane are a pair ofparallel lines.

It is possible for three linear equations to be inconsistent, even though any two of the equations

are consistent together. For example, the equations

are inconsistent. Adding the first two equations together gives 3x + 2y = 2, which can be

subtracted from the third equation to yield 0 = 1. Note that any two of these equations have a

common solution. The same phenomenon can occur for any number of equations.

In general, inconsistencies occur if the left-hand sides of the equations in a system are linearly

dependent, and the constant terms do not satisfy the dependence relation. A system of equations

whose left-hand sides are linearly independent is always consistent.

System of Equations

Matrix algebra is used for solving systems of equations. Can you illustrate this concept?

Matrix algebra is used to solve a system of simultaneous linear equations. In fact, for many mathematical procedures such as the solution to a set of nonlinear equations, interpolation, integration, and differential equations, the solutions reduce to a set of simultaneous linear equations. Let us illustrate with an example for interpolation. A general set of m linear equations and n unknowns,

11212111 cxaxaxa nn =+++ LL

22222121 cxaxaxa nn =+++ LL

…………………………………… …………………………………….

mnmnmm cxaxaxa =+++ ........2211

can be rewritten in the matrix form as

⋅⋅=

⋅⋅

mnmnmm

n

n

c

c

c

x

x

x

aaa

aaa

aaa

2

1

2

1

21

22221

11211

..

..

..

MM

MM

Denoting the matrices by [ ]A , [ ]X , and [ ]C , the system of equation is

[ ][ ] [ ]CXA = , where [ ]A is called the coefficient matrix, [ ]C is called the right hand side

vector and [ ]X is called the solution vector.

Sometimes [ ][ ] [ ]CXA = systems of equations are written in the augmented form. That is

[ ]

=

nmnmm

n

n

c

c

c

a......aa

a......aa

a......aa

CA2

1

21

22221

11211

M

M

M

M

M

M

MM

A system of equations can be consistent or inconsistent. What does that mean?

A system of equations [ ][ ] [ ]CXA = is consistent if there is a solution, and it is inconsistent if there is no solution. However, a consistent system of equations does not mean a unique solution, that is, a consistent system of equations may have a unique solution or infinite solutions (Figure 1).

Figure 5.1. Consistent and inconsistent system of equations flow chart.

Consistent System Inconsistent System

Unique Solution Infinite Solutions

[A][X]= [B]

Example 1

Give examples of consistent and inconsistent system of equations.

Solution

a) The system of equations

=

4

6

31

42

y

x

is a consistent system of equations as it has a unique solution, that is,

=

1

1

y

x.

b) The system of equations

=

3

6

21

42

y

x

is also a consistent system of equations but it has infinite solutions as given as follows. Expanding the above set of equations,

32

642

=+=+

yx

yx

you can see that they are the same equation. Hence, any combination of ( )yx, that satisfies

642 =+ yx

is a solution. For example ( ) ( )1,1, =yx is a solution. Other solutions include

( ) )25.1,5.0(, =yx , ( ) )5.1 ,0(, =yx , and so on. c) The system of equations

=

4

6

21

42

y

x

is inconsistent as no solution exists.

How can one distinguish between a consistent and inconsistent system of equations?

A system of equations [ ][ ] [ ]CXA = is consistent if the rank of A is equal to the rank of

the augmented matrix [ ]CAM

A system of equations [ ][ ] [ ]CXA = is inconsistent if the rank of A is less than the rank

of the augmented matrix [ ]CAM . But, what do you mean by rank of a matrix?

Rank The rank of a matrix is defined as the order of the largest square submatrix whose determinant is not zero. (OR) The rank of any matrix A, written as rank(A) is the size of the largest nonvanishing determinant contained in A.

142

630

42

63

268

32012

68

32 :Example

=

=

=

≠−=

rank

rank

Equivalently, then rank is the (maximum) number of linearly independent rows (or columns) of matrix A, where naaa ,...,, 21 is a set of linearly independent vectors iff

∑ ==

n

jjj ak

10 implies 021 ==⋅⋅⋅== nkkk .

Example:

68

32 0,00

68

320

6

3

8

221

2

121 ==⇒=

⇒=

+

kk

k

kkk

42

63 21

2

121 20

42

630

4

6

2

3kk

k

kkk −=⇒=

⇒=

+

Properties of the rank of a matrix: If A~ m×n, 0 ≤ rank(A) = integer ≤ min(m, n). If I~n×n, rank(I) = n, rank(0) = 0. rank(A) = n, if A is an orthogonal matrix of dimension n. rank(A) = rank(A′) = rank( AA′ ) = rank( AA ′ ). If A and B are of the same order, rank(A+B )≤ rank(A)+rank(B). If AB is defined, rank(AB)≤min[rank(A), rank(B)]. If A is diagonal, rank(A)= number of nonzero elements. If A is idempotent, rank(A) = tr(A). The rank of a matrix is not changed if one row (column) is multiplied by a nonzero constant or if such a multiple of one row (column) is added to another row(column) .

A square matrix of order n is nonsingular iff it is of full rank rank(A) = n or, equivalently, | A |≠0 The rank of a matrix is unchanged by premultiplying or postmultiplying by a nonsingular matrix. Thus if there are nonsingular matrices E and F for which

kAkkII

EAF =⇒×

= )rank( ~,00

0

Example 2

What is the rank of

[ ]

=321

502

213

A ?

Solution

The largest square submatrix possible is of order 3 and that is ][ A itself. Since ,023)det( ≠−=A the rank of .3][ =A

Example 3

What is the rank of

[ ]

=715

502

213

A ?

Solution

The largest square submatrix of ][ A is of order 3 and that is ][ A itself. Since 0)det( =A , the rank of ][ A is less than 3. The next largest square submatrix would be a 2×2 matrix. One of the square submatrices of ][ A is

[ ]

=

02

13B

and 02)det( ≠−=B . Hence the rank of ][ A is 2. There is no need to look at other 22× submatrices to establish that the rank of ][ A is 2.

Example 4

How do I now use the concept of rank to find if

=

2.279

2.177

8.106

112144

1864

1525

3

2

1

x

x

x

is a consistent or inconsistent system of equations?

Solution

The coefficient matrix is

[ ]

=112144

1864

1525

A

and the right hand side vector is

[ ]

=2.279

2.177

8.106

C

The augmented matrix is

[ ]

=2.279112144

2.1771864

8.1061525

M

M

M

B

Since there are no square submatrices of order 4 as ][B is a 3×4 matrix, the rank of ][B is at most 3. So let us look at the square submatrices of ][B of order 3; if any of these square submatrices have determinant not equal to zero, then the rank is 3. For example, a submatrix of the augmented matrix ][B is

=112144

1864

1525

][D

has 084)det( ≠−=D . Hence the rank of the augmented matrix ][B is 3. Since ][][ DA = , the rank of ][ A is 3. Since the rank of the augmented matrix ][B equals the rank of the coefficient matrix ][ A , the system of equations is consistent.

Example 5

Use the concept of rank of matrix to find if

=

0.284

2.177

8.106

21389

1864

1525

3

2

1

x

x

x

is consistent or inconsistent?

Solution

The coefficient matrix is given by

[ ]

=21389

1864

1525

A

and the right hand side

[ ]

=0.284

2.177

8.106

C

The augmented matrix is

[ ]

=0.284:21389

2.177:1864

8.106:1525

B

Since there are no square submatrices of order 4 as ][B is a 4×3 matrix, the rank of the augmented ][B is at most 3. So let us look at square submatrices of the augmented matrix ][B of order 3 and see if any of these have determinants not equal to zero. For example, a square submatrix of the augmented matrix ][B is

[ ]

=21389

1864

1525

D

has 0)det( =D . This means, we need to explore other square submatrices of order 3 of the augmented matrix ][B and find their determinants. That is,

[ ]

=0.284213

2.17718

8.10615

E

0)det( =E

[ ]

=0.2841389

2.177864

8.106525

F

0)det( =F

[ ]

=0.284289

2.177164

8.106125

G

0)det( =G All the square submatrices of order 3×3 of the augmented matrix ][B have a zero determinant. So the rank of the augmented matrix ][B is less than 3. Is the rank of augmented matrix ][B equal to 2?. One of the 22× submatrices of the augmented matrix ][B is

[ ]

=

864

525H

and 0120)det( ≠−=H

So the rank of the augmented matrix ][B is 2. Now we need to find the rank of the coefficient matrix ][B .

[ ]

=21389

1864

1525

A

and 0)det( =A

So the rank of the coefficient matrix ][ A is less than 3. A square submatrix of the coefficient matrix ][ A is

[ ]

=

18

15J

03)det( ≠−=J So the rank of the coefficient matrix ][ A is 2. Hence, rank of the coefficient matrix ][ A equals the rank of the augmented matrix [B]. So the system of equations ][][][ CXA = is consistent.

Example 6

Use the concept of rank to find if

=

0.280

2.177

8.106

21389

1864

1525

3

2

1

x

x

x

is consistent or inconsistent.

Solution

The augmented matrix is

[ ]

=0.280:21389

2.177:1864

8.106:1525

B

Since there are no square submatrices of order 4×4 as the augmented matrix ][B is a 4×3 matrix, the rank of the augmented matrix ][B is at most 3. So let us look at square submatrices of the augmented matrix (B) of order 3 and see if any of the 3×3 submatrices have a determinant not equal to zero. For example, a square submatrix of order 3×3 of

][B

[ ]

=21389

1864

1525

D

det(D) = 0 So it means, we need to explore other square submatrices of the augmented matrix ][B

[ ]

=0.280213

2.17718

8.10615

E

00.120det( ≠≠E . So the rank of the augmented matrix ][B is 3. The rank of the coefficient matrix ][ A is 2 from the previous example. Since the rank of the coefficient matrix ][ A is less than the rank of the augmented matrix ][B , the system of equations is inconsistent. Hence, no solution exists for

][][][ CXA = .

If a solution exists, how do we know whether it is unique?

In a system of equations ][][][ CXA = that is consistent, the rank of the coefficient

matrix ][ A is the same as the augmented matrix ][ CA . If in addition, the rank of the

coefficient matrix ][ A is same as the number of unknowns, then the solution is unique; if the rank of the coefficient matrix ][ A is less than the number of unknowns, then infinite solutions exist.

Unique solution ifrank (A) = number of unknowns

Infinite solutions ifrank (A) < number of unknowns

Consistent System ifrank (A) = rank (A.B)

Inconsistent System ifrank (A) < rank (A.B)

[A] [X] = [B]

Figure 5.2. Flow chart of conditions for consistent and inconsistent system of equations.

Example 7

We found that the following system of equations

=

2.279

2.177

8.106

112144

1864

1525

3

2

1

x

x

x

is a consistent system of equations. Does the system of equations have a unique solution or does it have infinite solutions?

Solution

The coefficient matrix is

[ ]

=112144

1864

1525

A and the right hand side is [ ]

=2.279

2.177

8.106

C

While finding out whether the above equations were consistent in an earlier example, we found that the rank of the coefficient matrix (A) equals rank of augmented matrix [ ]CAM equals 3. The solution is unique as the number of unknowns = 3 = rank of (A).

Example 8

We found that the following system of equations

=

0.284

2.177

8.106

21389

1864

1525

3

2

1

x

x

x

is a consistent system of equations. Is the solution unique or does it have infinite solutions.

Solution

While finding out whether the above equations were consistent, we found that the rank of the coefficient matrix ][ A equals the rank of augmented matrix ( )CAM equals 2 Since the rank of 2][ =A < number of unknowns = 3, infinite solutions exist.

If we have more equations than unknowns in [A] [X] = [C], does it mean the system is inconsistent?

No, it depends on the rank of the augmented matrix [ ]CAM and the rank of ][ A . a) For example

=

0.284

2.279

2.177

8.106

21389

112144

1864

1525

3

2

1

x

x

x

is consistent, since rank of augmented matrix = 3 rank of coefficient matrix = 3 Now since the rank of (A) = 3 = number of unknowns, the solution is not only consistent but also unique. b) For example

=

0.280

2.279

2.177

8.106

21389

112144

1864

1525

3

2

1

x

x

x

is inconsistent, since rank of augmented matrix = 4 rank of coefficient matrix = 3

c) For example

=

0.280

6.213

2.177

8.106

21389

21050

1864

1525

3

2

1

x

x

x

is consistent, since rank of augmented matrix = 2 rank of coefficient matrix = 2 But since the rank of ][ A = 2 < the number of unknowns = 3, infinite solutions exist.

Example 9

Determine if

=35

23][B is the inverse of

−−

=35

23[A]

Solution

]][[ AB

−−

=35

23

35

23

10

01

= ][ I=

Since ][][][ IAB = ,

][B is the inverse of [A] and ][ A is the inverse of ][B . But, we can also show that

]][[ BA

−−

=35

23

35

23

=10

01I][=

to show that ][ A is the inverse of ][B .

Can I use the concept of the inverse of a matrix to find the solution of a set of equations [A] [X] = [C]?

Yes, if the number of equations is the same as the number of unknowns, the coefficient matrix ][ A is a square matrix. Given

][][][ CXA =

Then, if 1][ −A exists, multiplying both sides by 1][ −A .

][][]][[][ 11 CAXAA −− =

][][][][ 1 CAXI −=

][][][ 1 CAX −=

This implies that if we are able to find 1][ −A , the solution vector of ][][][ CXA = is

simply a multiplication of 1][ −A and the right hand side vector, ][C .

How do I find the inverse of a matrix?

If ][ A is a nn× matrix, then 1][ −A is a nn× matrix and according to the definition of inverse of a matrix

][][][ 1 IAA =− Denoting

⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅

⋅⋅⋅⋅

=

nnnn

n

n

aaa

aaa

aaa

A

21

22221

11211

][

⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅

⋅⋅⋅⋅

=−

''2

'1

'2

'22

'21

'1

'12

'11

1][

nnnn

n

n

aaa

aaa

aaa

A

⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅

⋅⋅⋅

=

10

1

0

010

001

][ I

Using the definition of matrix multiplication, the first column of the 1][ −A matrix can then be found by solving

⋅⋅=

⋅⋅

⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅⋅

⋅⋅⋅⋅

0

0

1

'1

'21

'11

21

22221

11211

nnnnn

n

n

a

a

a

aaa

aaa

aaa

Similarly, one can find the other columns of the 1][ −A matrix by changing the right hand side accordingly. Definition: Let Abe an n n× matrix. The matrix Bsuch that nBA AB I= = is called the inverse of Aand will be denoted by 1A− . A is then said to be invertible. Remark: Given AX b= , if A is invertible then 1X A b−= .

another way to find the inverse of a matrix?

For finding the inverse of small matrices, the inverse of an invertible matrix can be found by

[ ] ( ) ( )AadjA

Adet

11 =−

where

( )

T

21

22221

11211

=

nnnn

n

n

CCC

CCC

CCC

Aadj

L

M

L

where ijC are the cofactors of ija . The matrix

nnn

n

n

CC

CCC

CCC

LL

MM

L

L

1

22221

11211

itself is called the matrix of cofactors from [A]. Cofactors are defined in Chapter 4. ( OR)

Inverse

If A is a square, nonsingular matrix of order n, a unique inverse matrix A 1− of order n exists, where IAAAA == −− 11 . The inverse matrix can be calculated as

( ) ( )

AM

AC

Aji

jiij 1 )('

1 −==

+−

where ( )Cij is the matrix of cofactors, and its transpose ( )Cij' is called the adjoint

matrix.

A= ( )

−

−=

−−

==

−

3

2

3

1

11

23

13

31

3211

31

32'

'1 CA

A ij

=

−

−

=−

10

01

3

2

3

1

11

31

321AA

Properties of inverse matrix

II =−1

( )A

AAAAAA1

,)(,)(111111 ==′=′= −−−−−−

( ) ABAB 111 −−− = assuming that both A and B are nonsingular and of the same order

AA ′=−1 iff A is orthogonal ( IAAAA =′=′ ), in which case A 1− and A′ are also orthogonal If A is nonsingular and symmetric, so is A 1−

NOTE: Theorem: Let ,A Bbe invertible n n× matrices. Then AB is invertible. In fact,

1 1 1( )AB B A− − −= . Theorem: Let Abe an invertible n n× matrix. Then TA is invertible. In fact,

1 1( ) ( )T TA A− −= .

Example 10

Find the inverse of

[ ]

=112144

1864

1525

A

Solution

From Example 4.6 in Chapter 4, we found ( ) 84det −=A

Next we need to find the adjoint of ][ A . The cofactors of A are found as follows.

The minor of entry 11a is

112144

1864

1525

11 =M112

18= 4−=

The cofactors of entry 11a is

( ) 1111

11 1 MC +−= 11M= 4−=

The minor of entry 12a is

112144

1864

1525

12 =M 1144

164= 80−=

The cofactor of entry 12a is

( ) 1221

12 1 MC +−= 12M−= )80(−−= 80= Similarly 38413 −=C

721 =C

11922 −=C

42023 =C

331 −=C

3932 =C

12033 −=C

Hence the matrix of cofactors of ][ A is

[ ]

−−−

−−=

120393

4201197

384804

C

The adjoint of matrix ][ A is T][C ,

( ) [ ]TCAadj =

−−−

−−=

120420384

3911980

374

Hence

[ ] ( ) ( )AadjA

Adet

11 =−

−−−

−−

−=

120420384

3911980

374

84

1

−−−

−=

429.1000.5571.4

4643.0417.19524.0

03571.008333.004762.0

If the inverse of a square matrix [A] exists, is it unique?

Yes, the inverse of a square matrix is unique, if it exists. The proof is as follows. Assume that the inverse of ][ A is ][B and if this inverse is not unique, then let another inverse of ][ A exist called ][C . If ][B is the inverse of ][ A , then

][][][ IAB = Multiply both sides by ][C ,

][][][][][ CICAB = ][][][][ CCAB =

Since [C] is inverse of ][ A , ][][][ ICA =

Multiply both sides by ][B ,

][][][ CIB = , ][][ CB = This shows that ][B and ][C are the same. So the inverse of ][ A is unique. Row reduction Main article: Gaussian elimination

In row reduction , the linear system is represented as an augmented matrix:

This matrix is then modified using elementary row operations until it reaches reduced row

echelon form. There are three types of elementary row operations:

Type 1 : Swap the positions of two rows.

Type 2 : Multiply a row by a nonzero scalar.

Type 3 : Add to one row a scalar multiple of another.

Because these operations are reversible, the augmented matrix produced always represents a

linear system that is equivalent to the original.

There are several specific algorithms to row-reduce an augmented matrix, the simplest of which

are Gaussian elimination and Gauss-Jordan elimination. The following computation shows

Gaussian elimination applied to the matrix above:

The last matrix is in reduced row echelon form, and represents the system x = −15, y = 8, z = 2. A

comparison with the example in the previous section on the algebraic elimination of variables

shows that these two methods are in fact the same; the difference lies in how the computations

are written down.

Cramer's rule

Cramer's rule is an explicit formula for the solution of a system of linear equations, with each

variable given by a quotient of two determinants. For example, the solution to the system

is given by

For each variable, the denominator is the determinant of the matrix of coefficients, while the

numerator is the determinant of a matrix in which one column has been replaced by the vector of

constant terms.

Though Cramer's rule is important theoretically, it has little practical value for large matrices,

since the computation of large determinants is somewhat cumbersome. (Indeed, large

determinants are most easily computed using row reduction.) Further, Cramer's rule has very

poor numerical properties, making it unsuitable for solving even small systems reliably, unless the

operations are performed in rational arithmetic with unbounded precision.

Homogeneous systems

A system of linear equations is homogeneous if all of the constant terms are zero:

A homogeneous system is equivalent to a matrix equation of the form

where A is an m × n matrix, x is a column vector with n entries, and 0 is the zero

vector with m entries.

Solution set

Every homogeneous system has at least one solution, known as the zero solution (or trivial

solution ),

(OR)

A homogeneous system of linear equation is a linear system of equations where the right hand sides of all the equations are zero. That is it is of the form

a11x1 + a12x2 + ... + a1nxn = 0 a21x1 + a22x2 + ... + a2nxn = 0 ... ... ... a m1x1 + am2x2 + ... + amnxn = 0

Notice that in matrix form this can be written as

Ax = 0

where A is the m x n matrix with entries aij, x the the n x 1 matrix with entries xi and 0 is the n x 1 zero matrix. The augmented matrix's last column is the zero column. Since the zero column is unaffected by elementary row operations, it is usually left out when the computations are performed. The solution (0,0, ... , 0) is called the trivial solution. Any other solution is called a nontrivial solution.

Theorem

Let

Ax = 0

be a homogeneous system with m x n matrix A. If m < n, then the system always has a nontrivial solution.

This theorem just states that if there are more variables than equations, then there is a nonzero solution.

Invertible matrix

In linear algebra, an n-by-n (square) matrix A is called invertible or nonsingular or nondegenerate if there exists an n-by-n matrix B such that

where I n denotes the n-by-n identity matrix and the multiplication used is ordinary matrix multiplication. If this is the case, then the matrix B is uniquely determined by A and is called the inverse of A, denoted by A−1. It follows from the theory of matrices that if

for square matrices A and B, then also

[1]

Non-square matrices (m-by-n matrices for which m ≠ n) do not have an inverse. However, in some cases such a matrix may have a left inverse or right inverse. If A is m-by-n and the rank of A is equal to n, then A has a left inverse: an n-by-m matrix B such that BA = I . If A has rank m, then it has a right inverse: an n-by-m matrix B such that AB = I .

While the most common case is that of matrices over the real or complex numbers, all these definitions can be given for matrices over any commutative ring.

A square matrix that is not invertible is called singular or degenerate. A square matrix is singular if and only if its determinant is 0. Singular matrices are rare in the sense that if you pick a random square matrix, it will almost surely not be singular.

Matrix inversion is the process of finding the matrix B that satisfies the prior equation for a given invertible matrix A.

Writing the transpose of the matrix of cofactors, known as an adjugate matrix, can also be an efficient way to calculate the inverse of small matrices, but this recursive method is inefficient for large matrices. To determine the inverse, we calculate a matrix of cofactors:

where |A| is the determinant of A, Cij is the matrix of cofactors, and AT represents the matrix transpose.

For most practical applications, it is not necessary to invert a matrix to solve a system of linear equations; however, for a unique solution, it is necessary that the matrix involved be invertible.

Inversion of 2×2 matrices

The cofactor equation listed above yields the following result for 2×2 matrices. Inversion of these matrices can be done easily as follows: [2]

This is possible because 1/(ad-bc) is the reciprocal of the determinant of the matrix in question, and the same strategy could be used for other matrix sizes.

Cramer's rule

Consider the linear system

which in matrix format is

Then, x and y can be found with Cramer's rule as:

and

The rules for 3×3 are similar. Given:

which in matrix format is

x, y and z can be found as follows:

Finding solution of a system of linear equations in the general case

Consider a system of linear equations represented in matrix multiplication form as

where the square matrix A is invertible and the vector is the column vector of the variables.

Then the theorem states that:

where Ai is the matrix formed by replacing the ith column of A by the column vector b.

This formula is, however, of limited practical value for larger matrices, as there are other more efficient ways of solving systems of linear equations, such as by Gauss elimination or, even better, LU decomposition.

Finding inverse matrix

Let A be an n×n matrix. Then

where Adj(A) denotes the adjugate matrix of A, det(A) is the determinant, and I is the identity matrix. If det(A) is invertible in R, then the inverse matrix of A is

If R is a field (such as the field of real numbers), then this gives a formula for the inverse of A, provided det(A) ≠ 0. The formula is, however, of limited practical value for large matrices, as there are other more efficient ways of generating the inverse matrix, such as by Gauss–Jordan elimination.

LU decomposition In linear algebra, the LU decomposition is a matrix decomposition which writes a matrix as the

product of a lower triangular matrix and an upper triangular matrix. The product sometimes

includes a permutation matrix as well. This decomposition is used in numerical analysis to solve

systems of linear equations or calculate the determinant.

Definitions

Let A be a square matrix. An LU decomposition is a decomposition of the form

where L and U are lower and upper triangular matrices (of the same size), respectively. This means that L has only zeros above the diagonal and U has only zeros below the diagonal. For a matrix, this becomes:

Solving linear equations

Given a matrix equation

we want to solve the equation for a given A and b. In this case the solution is done in two logical steps:

1. First, we solve the equation Ly = b for y 2. Second, we solve the equation Ux = y for x.

Note that in both cases we have triangular matrices (lower and upper) which can be solved

directly using forward and backward substitution without using the Gaussian elimination process

(however we need this process or equivalent to compute

the LU decomposition itself). Thus theLU decomposition is computationally efficient only when we

have to solve a matrix equation multiple times for different b; it is faster in this case to do an LU

decomposition of the matrix A once and then solve the triangular matrices for the different b, than

to use Gaussian elimination each time.

Inverse matrix

The matrices L and U can be used to calculate the matrix inverse by:

Computer implementations that invert matrices sometimes use this approach.[citation needed]

Small example

One way of finding the LU decomposition of this simple matrix would be to simply solve the

linear equations by inspection. You know that:

Such a system of equations is underdetermined. In this case any two non-zero elements

of L and U matrices are parameters of the solution and can be set arbitrarily to any non-zero

value. Therefore to find the unique LU decomposition, it is necessary to put some restriction

on L and Umatrices. For example, we can require the lower triangular matrix L to be a unit one

(i.e. set all the entries of its main diagonal to ones). Then the system of equations has the

following solution:

l21 = 1.5

u11 = 4

u12 = 3

u22 = − 1.5.

Substituting these values into the LU decomposition above:

1.5. Computation of the inverse matrix If

A - 1 =

u11 u12 ··· u1n

= ( 1 2 ..., n) u21 u22 ··· u2n

: : :

un1 un2 ··· unn

is the inverse matrix of A, then (AA - 1 = I => )

(1)

a11 a12 ··· a1n

a21 a22 ··· a2n

: : :

an1 an2 ··· ann

u1i

u2i

:

uni

=

0

:

0

1

0

:

0

<=>

(1)' A i = i

so that i is determined by solving the linear system of equations (1), for example, by applying the Gaussian elimination to the matrix

a11 ... a1n 0

: : 0

ai1 ... ain 1

: : 0

an1 ... ann 0

= (A | i )

and the inverse matrix is obtained by solving all n equations (1)', i = 1, ..., n. They all have the same coefficient matrix A so that they can all be solved at the same time by the Gaussian elimination.

a11 ... a1n 1 0 0 ... 0

a21 ... a2n 0 1 0 ... 0

: : :

an1 ... ann 0 0 ... 1

= (A | I ).

If it is possible to convert (A | I) into the form

1 0 ··· 0 b11 b12 ··· b1n

= (I | B) 0 1 0 ··· 0 b21 b22 ··· b2n

: : : :

0 ··· 1 bn1 bn2 ··· bnn

by means of the elementary row operations, then B = A - 1. This is so, because (I | B) represents the n linear systems of equations, where the ith system is

1· u1i + 0· u2i + ··· + 0uni = b1i

0· u1i + 1· u2i + 0 ··· + 0uni = b2i

:

0 + ··· + 0 + 1· uni = bni

<=>

i =

b1i

b2i

:

bni

1.6. LU decomposition Assume that a square matrix A can be written as a product

(1)

LU = A,

where L is a lower triangular matrix and U is an upper triangular matrix;

l11 0 0 0

l21 l 22 0 0

l31 l 32 l33 0

l41 l42 l43 l44

u11 u12 u13 u14

0 u22 u23 u24

0 0 u33 u34

0 0 0 u44

=

a11 a12 a13 a14

. a21 a22 a23 a24

a31 a32 a33 a34

a41 a42 a43 a44

Then the linear system of equations A = , i.e.,

(2)

A = (LU) = L (U ) = ,

can be solved by first solving the lower triangular system

(3)

L =

and then the upper triangular system

(4)

U = .

The equation (3) is solved by the method of forward substitution ,

(5)

y1 = b1 / l11 ,

yi

= (1 / lii ) [ bi - l ij yj ] , i = 2, 3, ..., n,

while the equation (4) is solved using the backward substitution,

(6)

xn = yn / unn ,

xi = (1 / uii ) [ yi - uij xj ] , i = n - 1, n - 2, ..., 1.

Note. LU decomposition is not unique: if A = LU, then A = LDD - 1U = (LD) (D - 1U) = L1 U1 is again an LU decomposition, if D is a diagonal matrix. An additional assumption l ii = 1 i = 1, ..., n, guarantees the uniqueness.

The construction of an LU decomposition can be done by the Crout's algorithm, for example.

Usually we construct an LU decomposition by means of the Gaussian elimination in such a way that L is formed from the coefficients used in the elimination while U is the matrix A itself after the elimination.

Find the LU decomposition of

A =

9 3 6

4 6 1

1 1 7

Solution: We use Gaussian elimination on A, so that the resulting upper triangular matrix will be U and the lower triangular matrix which is formed from the opposite numbers of the coefficients used will be L.

9 3 6

4 6 1

1 1 7

~

9 3 6

0 14/3 -5/3

0 2/3 19/3

~

9 3 6

0 14/3 -5/3

0 0 46/7

-Add the 1st row multiplied by -4/9 to the 2nd row

=>the 1st entry on the 2nd row of L is -(-4/9) = 4/9

-Add the 1st row multiplied by -1/9 to the 3rd row

=>the 1st entry on the 3rd row of L is -(-1/9)=1/9

-Add the 2nd row multiplied by -1/7 to the 3rd row

=>the 2nd entry on the 3rd row of L is -(-1/7) = 1/7

-We have the upper triangular matrix U

Moreover,

L =

1 0 0

4/9 1 0

1/9 1/7 1

so that the LU decomposition is

A =

9 3 6

=

4 6 1

1 1 7

1 0 0

4/9 1 0

1/9 1/7 1

9 3 6

= LU

0 14/3 -5/3

0 0 46/7

The result can be checked by multiplying L and U.

Crout matrix decomposition

In linear algebra, the Crout matrix decomposition is an LU decomposition which decomposes a matrix into a lower triangular matrix (L), an upper triangular matrix (U) and, although not always needed, a permutation matrix (P).

The Crout matrix decomposition algorithm differs slightly from the Doolittle method. Doolittle's method returns a unit lower triangular matrix and an upper triangular matrix, while the Crout method returns a lower triangular matrix and a unit upper triangular matrix.

3.4. The determinant of a matrix The determinant of a 1 x 1 matrix A = (a11 ) , det A, is a11

The determinant of an n x n matrix

A =

a11 a12 ... a1n

a21 a22 ... a2n

:

an1 an2 ... ann

is det A = a11 A11 + a12 A12 + ··· + a1n A1n =

a11 ... a1n

, a21 ... a2n

:

an1 ... ann

where A1j is (-1)1+j times the determinant of the (n - 1) x (n - 1) matrix, which is obtained from A by deleting the first row and the jth column. For example, the determinant of a 2 x 2 matrix is

a11 a12

a21 a22

= a11 (-1)1+1 a22 + a12 (-1)1+2a21 = a11 a22 - a12 a21

and the determinant of a 3 x 3 matrix is

a11 a12 a13

a21 a22 a23

a31 a32 a33

= a11

a22 a23

a32 a33

- a12

a21 a23

a31 a33

+ a13

a21 a22

. a31 a32

Example 1: Calculation of a determinant

It can be shown that det A = ai1 Ai1 + ai2 Ai2 + ··· + ain Ain ,

= a1j A1j + a2j A2j + ··· + anj Anj

where A1j is (-1)1+j times the determinant of the (n - 1) x (n - 1) matrix, which is obtained from A by deleting the ith row and the jth column.

The number Aij is called the cofactor of the element aij .

It follws from the definition that

1) if A has a 0 row or a 0 column, then det A = 0.

2) det AT = det A.

3) if a row or a column of A is multiplied by k, the determinant of the matrix obtained = kdet A => det (kA) = kndet A.

4) if any two rows (or columns) of A are interchanged, the determinant of the matrix obtained = -det A.

5) if any two rows (or columns) of A are the same, then det A = 0.

6) If c x (kth row) is added to the ith row, i k, the determinant of the matrix obtained = det A. The same rule applies to columns.

7)

a11 + x1 a12 ··· a1n

a21 + x2 a22 ··· a2n

: :

an1 + xn an2 ··· ann

=

a11 a12 ··· a1n

a21 a22 ··· a2n

: :

an1 an2 ··· ann

+

x1 a12 ··· a1n

x2 a22 ··· a2n

:

xn an2 ··· ann

The same rule applies to rows.

8) The determinant of an upper triangular matrix is the product of the diagonal elements.

Example 2: The determinant of an upper triangular matrix

9) det AB = det A· det B.

Example 3: The determinant of a product

The adjugate matrix of A is

adj A =

A11 A21 ··· An1

= (Aij )T

A12 A22 ··· An2

: :

A1n A2n ··· Ann

Proposition 8. A adj A = adj A · A = det A · I

Proof. Denote A adj A = (bij )

=> bij = ai1 Aj1 + ai2 Aj2 + ··· + ain Ajn

=>

1) If i j then bij is the determinant of the matrix obtained from A by replacing the jth row by the ith row. => two equal rows => det = 0 = bij .

2) If i = j , then bij = the determinant of A computed across the ith row.

=> A adj A =

det A 0 ··· 0

= det A· I 0 det A 0 ··· 0

: :

0 ··· 0 det A

Similarly adj A · A = det A· I.

Peoposition 9.

1) A has an inverse <=> det A 0

2) If det A 0, then

A -1 = (1 / det A) adj A.

Proof. 2) If det A 0, then (1 / det A) adj A · A = I => A -1 = (1 / det A) adj A

1) If A has an inverse, then I = AA - 1 => 1 = det I = det (AA -1) = det A ·det A -1 => det A 0.

Note. det A 0 <=> A -1 exists <=> the rank of A is n. If det A = 0, then the rank of A < n and thus the nullity of A = n - (the rank of A) 1 => The equation A = 0 has a solution 0.

Applications of determinant Everything in this section is based on the following fact:

A homogeneous linear system of n equations in n unknowns has nontrivial solutions if and only if the determinant of the coefficient matrix is zero.

A = 0 1. If det A 0 => A-1 exists <=> A-1 A = A-1 = 0 <=> = 2. If det A = 0, then the rank of A is not full; it is n - r < n and there are n - r unknowns which can be chosen arbitrarily. => the system has nontrivial ( 0) solutions.

Example 1: Calculation of a determinant

Calculate the determinant of the matrix

A =

1 2 0 0

.

0 1 0 2

0 2 3 0

3 0 1 0

Solution:

det A =

1 2 0 0

0 1 0 2

0 2 3 0

3 0 1 0

= (-1)1+1·1

1 0 2

2 3 0

0 1 0

+ (-1)1+2·2

0 0 2

0 3 0

3 1 0

+ 0

0 1 2

0 2 0

3 0 0

- 0

0 1 0

0 2 3

3 0 1

-Compute the determinant across the first row and obtain four 3-rowed determinants -Compute the first one of the 3-rowed determinants across the 3rd row(this row has two zeros) and the second one across the first row -The two last of the 3-rowed determinants vanish -Remember that we could equally well compute across columns too -Each subdeterminant has a coefficient of the form (-1)i+jaij