f t luk - computer scienceflaherje/pdf/lin1.pdf · computational linear algebra course ma th csci...
TRANSCRIPT
Computational Linear Algebra
�Course� �MATH� ����� CSCI� �����
� Semester� Fall ����
� Instructors�
� Joseph E Flaherty� aherje�csrpiedu
� Franklin T Luk� luk�csrpiedu
�Wesley Turner� turnerw�csrpiedu
Department of Computer ScienceRensselaer Polytechnic Institute
Troy� NY �����
�
OUTLINE
�� Introduction
��� Notation
��� Special matrices
�� Gaussian Elimination
��� Vector and matrix norms
��� Finite precision arithmetic
��� Factorizations
��� Pivoting
��� Accuracy estimation
�� Special Linear Systems
��� Symmetric positive de�nite systems
��� Banded and pro�le systems
��� Block tridiagonal systems
��� Symmetric inde�nite systems
��� Pro�le methods
��� Sparse Systems
�� Orthogonalization and Least Squares
��� Orthogonality and the singular value decomposition
��� Rotations and re�ections
��� QR factorizations
��� Least squares problems
�
OUTLINE
�� The Algebraic Eigenvalue Problem
��� Introduction
��� Power methods
��� Hessenberg and Schur forms� The QR algorithm
��� Symmetric problems
��� Jacobi iteration
�� Iterative Methods
��� Fixedpoint methods
��� The basic conjugate gradient method
��� Preconditioning
��� Krylov and generalizedresidual methods
� Introduction to Parallel Computing
�� Shared and distributedmemory computation
�� Matrix multiplication
�� Gaussian elimination
�
References
�� O� Axelsson ������� Iterative Solution Methods�Cambridge� Cambridge�
�� R� Barrett� M� Berry� T� Chan� J� Demmel� J� Do�nato� J� Dongarra� V� Eijkhout� V� Pozo� C� Romine�and H� van der Vorst ������� Templates for the
Solution of Linear Systems� Building Blocks for
Iterative Methods� SIAM� Philadelphia�
� A� Bj�orck ������� Numerical Methods for Least
Squares Problems� SIAM� Philadelphia�
�� W� Briggs ��� ��� A Multigrid Tutorial� SIAM�Philadelphia�
�� J�W� Demmel ������� Applied Numerical Linear
Algebra� SIAM� philadelphia�
�� A� George and J� Liu ��� ��� Computer Solution of
Large Sparse Positive De�nite Systems� Prentice�Hall� Englewood Cli�s�
�� G�H� Golub and C�F� Van Loan ������ Matrix
Computations� Third Edition� Johns Hopkins� Bal�timore�
� A� Greenbaum ������� Iterative Methods for Solv�
ing Linear Systems� SIAM Philadelphia�
�
�� W� Hackbusch ������� Iterative Solution of Large
Sparse Linear Systems of Equations� Springer�Verlag� Berlin�
��� N� Higham ������� Accuracy and Stability of Nu�
merical Algoritms� SIAM Philadelphia�
��� B� Parlett ��� ��� The Symmetric Eigenvalue Prob�
lem� Prentice�Hall� Englewood Cli�s�
��� Y� Saad ������� Iterative Methods for Sparse Lin�
ear Systems� PWS� Boston�
�� B� Smith� P� Bjorstad� and W� Gropp ������� Do�main Decomposition� Parallel Multilevel Meth�
ods for Elliptic Partial Di�erential Equations�Cambridge� Cambridge�
��� G�W� Stewart ������ Introduction to Matrix Com�
putations� Academic Press� New York�
��� G�W� Stewart and J��G� Sun ������ Matrix Per�
turbation Theory� Academic Press� New York�
��� L�N� Trefethen and D� Bau� III ������� Numerical
Linear Algebra� SIAM� Philadelphia� �
��� P� Wesseling ������� An Introduction to Multigrid
Methods� Wiley� Chichester��Text
�
� � D� Young ������� Iterative Solution of Large Lin�
ear Systems� Academic Press� New York�
�
�� Introduction
���� Notation
� Motivation�
� Linear algebra is involved in approximately �� of scien
ti�c computation
� Circuit and network analysis
� Potential theory
� Signal processing
� Acoustics and vibrations
� Optimization
� Finite di�erence and �nite element methods
� Tomography
� Optimal control
� Problems�
� Solve linear systems
� Solve least squares problems
� Solve eigenvalueeigenvector problems
�
Notation
� Reading Trefethen and Bau ������ Lecture �
� An m� n matrix
A �
�����
a�� a�� � � � a�na�� a�� � � � a�n��� ��� � � � ���
am� am� � � � amn
����� ���
� Bold letters denote vectors and matrices
� Italic letters denote scalars
� Lower case letters are elements of a matrix
� aij � �� i � �� �� ����m� j � �� �� ���� n� unless stated
� A � �m�n
� �m�n a vector space of m� n real matrices
� A vector x � �m�� � �m
x �
�����
x�x����
xm
����� � xT � �x�� x�� � � � � xm�
�
MATLAB
� Specify Algorithms using MATLAB
� Vector dot �scalar� product
c � xTy �
nXi��
xiyi ���
function c � dot�x� y�
� dot compute a dot product of the nvectors x and y
n � length�x�
c � �
for i � � n
c � c � x�i��y�i�
end
� Subscripts are enclosed in parentheses
� The MATLAB function length computes the dimension
of a vector
� The indicates a range
� In the for loop� i ranges over integers from � to n
� dot�x� y� is a library function in MATLAB
A SAXPY
� A saxpy ��scalar a x plus y�� is the vector
z � ax � y ���
function z � saxpy�a� x� y�
� saxpy compute the vector z � a � x � y where
� a is a scalar and x and y are vectors
n � length�x�
for i � � n
z�i� � a�x�i� � y�i�
end
� x and y have the same dimensions n
� Production software should check this
�
Matrix Multiplication
� Let A � �m�r� B � �r�n� and C � �m�n� then
cij �
rXk��
aikbkj� i � � m� j � � n ���
function C � matmy�A� B�
� matmy compute the matrix product C � AB
�m r� � size�A��
�r n� � size�B��
for i � � m
for j � � n
C�i�j� � ��
for k � � r
C�i�j� � C�i�j� � A�i�k��B�k�j��
end
end
end
� size�A� returns the row and column dimensions of A
� The �s suppress printing of results
� The complexity is mnr multiplications and additions
� Some details are omitted
��
Matrix Multiplication
� Matrix multiplication using the dot product
� Let aj denote the jth column of A
A � �a�� a�� � � � � ar� ���
� Replace the loop in ��� by a dot product
cij � aTi bj� i � � m� j � � n ���
� aTi is the ith column of AT or the ith row of A
function C � matmy�A� B�
� matmy compute the matrix product C � AB using
� the dot product
�m r� � size�A��
�r n� � size�B��
for i � � m
for j � � n
C�i�j� � dot�A�i�� r���B�� r�j���
end
end
� The range � r indicates an operation for an entire row or
column
� A � denotes transposition
� A�i� � r� is the ith row of A
��
Matrix Multiplication with Saxpys
� Reorder the loop structure regard the columns of C as linear
combinations of those of A�� �
� �
� �� �
�
��
��
��
�
��
��
�
��
��
�
�� �
��
�
��
� compute C � AB as
cj �
rXk��
bkjak� j � � n ��
� Implement the sum using saxpys
function C � smatmy�A� B�
� smatmy compute the matrix product C � AB
� using saxpys
�m r� � size�A��
�r n� � size�B��
C � zeros�m� n��
for j � � n
for k � � r
C�� m�j� � saxpy�B�k�j�� A�� m�k�� C�� m�j���
end
end
� zeros�m�n� returns a m� n matrix of zeros�
��
Outer Products
� Reorder the loops to perform an outer product
� An outer product of an mvector x and an nvector y is
the m� n matrix
C � xyT ���
or
C �
�����
x�x����
xm
����� �y�y� � � � yn� �
�����
x�y� x�y� � � � x�ynx�y� x�y� � � � x�yn��� ��� � � � ���
xmy� xmy� � � � xmyn
�����
� Problem � do matrix multiplication using outer products
� Show this for the �� � problem�� �
� �
� �� �
�
�
��
Outer Product
� Write an algorithm for a general system
� Di�erent loop structures perform di�erently on di�erent com
puters
dot product i� j� k
saxpy j� k� i
outer porduct k� i� j
��