advanced modern engineering mathematics -...

Advanced Modern Engineering Mathematics

Glyn Jamesfourth edition


Fourth Edition

We work with leading authors to develop the

strongest educational materials in mathematics,

bringing cutting-edge thinking and best

learning practice to a global market.

Under a range of well-known imprints, including

Prentice Hall, we craft high-quality print and

electronic publications which help readers to understand

and apply their content, whether studying or at work.

To find out more about the complete range of our

publishing, please visit us on the World Wide Web at:

www.pearsoned.co.uk


Fourth Edition

Glyn James Coventry UniversityandDavid Burley University of SheffieldDick Clements University of BristolPhil Dyke University of PlymouthJohn Searl University of EdinburghNigel Steele Coventry UniversityJerry Wright AT&T

Pearson Education LimitedEdinburgh GateHarlowEssex CM20 2JEEngland

and Associated Companies throughout the world

Visit us on the World Wide Web at:www.pearsoned.co.uk

First published 1993Second edition 1999Third edition 2004Fourth edition 2011

Pearson Education Limited 1993, 2011

The rights of Glyn James, David Burley, Dick Clements, Phil Dyke, John Searl, Nigel Steele and Jerry Wright to be identified as authors of this work have been asserted by them in accordance with the Copyright, Designs and Patents Act 1988.

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording or otherwise, without either the prior written permission of the publisher or a licence permitting restricted copying in the United Kingdom issued by the Copyright Licensing Agency Ltd, Saffron House, 610 Kirby Street, London EC1N 8TS.

All trademarks used herein are the property of their respective owners. The use of any trademark in this text does not vest in the author or publisher any trademark ownership rights in such trademarks, nor does the use of such trademarks imply any affiliation with or endorsement of this book by such owners.

Pearson Education is not responsible for third party internet sites.

ISBN: 978-0-273-71923-6

British Library Cataloguing-in-Publication DataA catalogue record for this book is available from the British Library

Library of Congress Cataloging-in-Publication DataAdvanced modern engineering mathematics / Glyn James ... [et al.]. 4th ed.

p. cm.ISBN 978-0-273-71923-6 (pbk.)

1. Engineering mathematics. I. James, Glyn.TA330.A38 2010620.00151dc22

2010031592

10 9 8 7 6 5 4 3 2 114 13 12 11 10

Typeset in 10/12pt Times by 35Printed by Ashford Colour Press Ltd., Gosport

Contents

Preface xixAbout the Authors xxiPublishers Acknowledgements xxiii

1.1 Introduction 2

1.2 Review of matrix algebra 2

1.2.1 Definitions 31.2.2 Basic operations on matrices 31.2.3 Determinants 51.2.4 Adjoint and inverse matrices 51.2.5 Linear equations 71.2.6 Rank of a matrix 9

1.3 Vector spaces 10

1.3.1 Linear independence 111.3.2 Transformations between bases 121.3.3 Exercises (14) 14

1.4 The eigenvalue problem 14

1.4.1 The characteristic equation 151.4.2 Eigenvalues and eigenvectors 171.4.3 Exercises (56) 231.4.4 Repeated eigenvalues 231.4.5 Exercises (79) 271.4.6 Some useful properties of eigenvalues 271.4.7 Symmetric matrices 291.4.8 Exercises (1013) 30

Chapter 1 Matrix Analysis 1

vi CONTENTS

1.5 Numerical methods 30

1.5.1 The power method 301.5.2 Gerschgorin circles 361.5.3 Exercises (1419) 38

1.6 Reduction to canonical form 39

1.6.1 Reduction to diagonal form 391.6.2 The Jordan canonical form 421.6.3 Exercises (2027) 461.6.4 Quadratic forms 471.6.5 Exercises (2834) 53

1.7 Functions of a matrix 54

1.7.1 Exercises (3542) 65

1.8 Singular value decomposition 66

1.8.1 Singular values 681.8.2 Singular value decomposition (SVD) 721.8.3 Pseudo inverse 751.8.4 Exercises (4350) 81

1.9 State-space representation 82

1.9.1 Single-inputsingle-output (SISO) systems 821.9.2 Multi-inputmulti-output (MIMO) systems 871.9.3 Exercises (5155) 88

1.10 Solution of the state equation 89

1.10.1 Direct form of the solution 891.10.2 The transition matrix 911.10.3 Evaluating the transition matrix 921.10.4 Exercises (5661) 941.10.5 Spectral representation of response 951.10.6 Canonical representation 981.10.7 Exercises (6268) 103

1.11 Engineering application: Lyapunov stability analysis 104

1.11.1 Exercises (6973) 106

1.12 Engineering application: capacitor microphone 107

1.13 Review exercises (120) 111

CONTENTS vii

2.1 Introduction 116

2.2 Engineering application: motion in a viscous fluid 116

2.3 Numerical solution of first-order ordinary differential equations 117

2.3.1 A simple solution method: Eulers method 1182.3.2 Analysing Eulers method 1222.3.3 Using numerical methods to solve engineering problems 1252.3.4 Exercises (17) 1272.3.5 More accurate solution methods: multistep methods 1282.3.6 Local and global truncation errors 1342.3.7 More accurate solution methods: predictorcorrector

methods 1362.3.8 More accurate solution methods: RungeKutta methods 1412.3.9 Exercises (817) 1452.3.10 Stiff equations 1472.3.11 Computer software libraries and the state of the art 149

2.4 Numerical solution of second- and higher-order differential equations 151

2.4.1 Numerical solution of coupled first-order equations 1512.4.2 State-space representation of higher-order systems 1562.4.3 Exercises (1823) 1602.4.4 Boundary-value problems 1612.4.5 The method of shooting 1622.4.6 Function approximation methods 164

2.5 Engineering application: oscillations of a pendulum 170

2.6 Engineering application: heating of an electrical fuse 174



3.1.1 Basic concepts 1833.1.2 Exercises (110) 1913.1.3 Transformations 1923.1.4 Exercises (1117) 1953.1.5 The total differential 1963.1.6 Exercises (1820) 199

Chapter 2 Numerical Solution of Ordinary Differential Equations 115

Chapter 3 Vector Calculus 181

viii CONTENTS

3.2 Derivatives of a scalar point function 199

3.2.1 The gradient of a scalar point function 1993.2.2 Exercises (2130) 203

3.3 Derivatives of a vector point function 203

3.3.1 Divergence of a vector field 2043.3.2 Exercises (3137) 2063.3.3 Curl of a vector field 2063.3.4 Exercises (3845) 2103.3.5 Further properties of the vector operator 2103.3.6 Exercises (4655) 214

3.4 Topics in integration 214

3.4.1 Line integrals 2153.4.2 Exercises (5664) 2183.4.3 Double integrals 2193.4.4 Exercises (6576) 2243.4.5 Greens theorem in a plane 2253.4.6 Exercises (7782) 2293.4.7 Surface integrals 2303.4.8 Exercises (8391) 2373.4.9 Volume integrals 2373.4.10 Exercises (92102) 2403.4.11 Gausss divergence theorem 2413.4.12 Stokes theorem 2443.4.13 Exercises (103112) 247

3.5 Engineering application: streamlines in fluid dynamics 248

3.6 Engineering application: heat transfer 250



4.2 Complex functions and mappings 259

4.2.1 Linear mappings 2614.2.2 Exercises (18) 2684.2.3 Inversion 2684.2.4 Bilinear mappings 2734.2.5 Exercises (919) 2794.2.6 The mapping w = z2 2804.2.7 Exercises (2023) 282

Chapter 4 Functions of a Complex Variable 257

CONTENTS ix

4.3 Complex differentiation 282

4.3.1 CauchyRiemann equations 2834.3.2 Conjugate and harmonic functions 2884.3.3 Exercises (2432) 2904.3.4 Mappings revisited 2904.3.5 Exercises (3337) 294

4.4 Complex series 295

4.4.1 Power series 2954.4.2 Exercises (3839) 2994.4.3 Taylor series 2994.4.4 Exercises (4043) 3024.4.5 Laurent series 3034.4.6 Exercises (4446) 308

4.5 Singularities, zeros and residues 308

4.5.1 Singularities and zeros 3084.5.2 Exercises (4749) 3114.5.3 Residues 3114.5.4 Exercises (5052) 316

4.6 Contour integration 317

4.6.1 Contour integrals 3174.6.2 Cauchys theorem 3204.6.3 Exercises (5359) 3274.6.4 The residue theorem 3284.6.5 Evaluation of definite real integrals 3314.6.6 Exercises (6065) 334

4.7 Engineering application: analysing AC circuits 335

4.8 Engineering application: use of harmonic functions 336

4.8.1 A heat transfer problem 3364.8.2 Current in a field-effect transistor 3384.8.3 Exercises (6672) 341



5.2 The Laplace transform 348

5.2.1 Definition and notation 3485.2.2 Transforms of simple functions 350

Chapter 5 Laplace Transforms 345

x CONTENTS

5.2.3 Existence of the Laplace transform 3535.2.4 Properties of the Laplace transform 3555.2.5 Table of Laplace transforms 3635.2.6 Exercises (13) 3645.2.7 The inverse transform 3645.2.8 Evaluation of inverse transforms 3655.2.9 Inversion using the first shift theorem 3675.2.10 Exercise (4) 369

5.3 Solution of differential equations 370

5.3.1 Transforms of derivatives 3705.3.2 Transforms of integrals 3715.3.3 Ordinary differential equations 3725.3.4 Simultaneous differential equations 3785.3.5 Exercises (56) 380

5.4 Engineering applications: electrical circuits and mechanical vibrations 381

5.4.1 Electrical circuits 3825.4.2 Mechanical vibrations 3865.4.3 Exercises (712) 390

5.5 Step and impulse functions 392

5.5.1 The Heaviside step function 3925.5.2 Laplace transform of unit step function 3955.5.3 The second shift theorem 3975.5.4 Inversion using the second shift theorem 4005.5.5 Differential equations 4035.5.6 Periodic functions 4075.5.7 Exercises (1324) 4115.5.8 The impulse function 4135.5.9 The sifting property 4145.5.10 Laplace transforms of impulse functions 4155.5.11 Relationship between Heaviside step and impulse functions 4185.5.12 Exercises (2530) 4235.5.13 Bending of beams 4245.5.14 Exercises (3133) 428

5.6 Transfer functions 428

5.6.1 Definitions 4285.6.2 Stability 4315.6.3 Impulse response 4365.6.4 Initial- and final-value theorems 4375.6.5 Exercises (3447) 4425.6.6 Convolution 4435.6.7 System response to an arbitrary input 4465.6.8 Exercises (4852) 450

CONTENTS xi

5.7 Solution of state-space equations 450

5.7.1 SISO systems 4505.7.2 Exercises (5361) 4545.7.3 MIMO systems 4555.7.4 Exercises (6264) 462

5.8 Engineering application: frequency response 462

5.9 Engineering application: pole placement 470

5.9.1 Poles and eigenvalues 4705.9.2 The pole placement or eigenvalue location technique 4705.9.3 Exercises (6570) 472



6.2 The z transform 483

6.2.1 Definition and notation 4836.2.2 Sampling: a first introduction 4876.2.3 Exercises (12) 488

6.3 Properties of the z transform 488

6.3.1 The linearity property 4896.3.2 The first shift property (delaying) 4906.3.3 The second shift property (advancing) 4916.3.4 Some further properties 4926.3.5 Table of z transforms 4936.3.6 Exercises (310) 494

6.4 The inverse z transform 494

6.4.1 Inverse techniques 4956.4.2 Exercises (1113) 501

6.5 Discrete-time systems and difference equations 502

6.5.1 Difference equations 5026.5.2 The solution of difference equations 5046.5.3 Exercises (1420) 508

Chapter 6 The z Transform 481

xii CONTENTS

6.6 Discrete linear systems: characterization 509

6.6.1 z transfer functions 5096.6.2 The impulse response 5156.6.3 Stability 5186.6.4 Convolution 5246.6.5 Exercises (2129) 528

6.7 The relationship between Laplace and z transforms 529

6.8 Solution of discrete-time state-space equations 530

6.8.1 State-space model 5306.8.2 Solution of the discrete-time state equation 5336.8.3 Exercises (3033) 537

6.9 Discretization of continuous-time state-space models 538

6.9.1 Eulers method 5386.9.2 Step-invariant method 5406.9.3 Exercises (3437) 543

6.10 Engineering application: design of discrete-time systems 544

6.10.1 Analogue filters 5456.10.2 Designing a digital replacement filter 5466.10.3 Possible developments 547

6.11 Engineering application: the delta operator and the transform 547

6.11.1 Introduction 5476.11.2 The q or shift operator and the operator 5486.11.3 Constructing a discrete-time system model 5496.11.4 Implementing the design 5516.11.5 The transform 5536.11.6 Exercises (3841) 554



7.2 Fourier series expansion 561

7.2.1 Periodic functions 5617.2.2 Fouriers theorem 5627.2.3 Functions of period 2 566

Chapter 7 Fourier Series 559

CONTENTS xiii

7.2.4 Even and odd functions 5737.2.5 Linearity property 5777.2.6 Exercises (17) 5797.2.7 Functions of period T 5807.2.8 Exercises (813) 5837.2.9 Convergence of the Fourier series 584

7.3 Functions defined over a finite interval 587

7.3.1 Full-range series 5877.3.2 Half-range cosine and sine series 5897.3.3 Exercises (1423) 593

7.4 Differentiation and integration of Fourier series 594

7.4.1 Integration of a Fourier series 5957.4.2 Differentiation of a Fourier series 5977.4.3 Coefficients in terms of jumps at discontinuities 5997.4.4 Exercises (2429) 602

7.5 Engineering application: frequency response and oscillating systems 603

7.5.1 Response to periodic input 6037.5.2 Exercises (3033) 607

7.6 Complex form of Fourier series 608

7.6.1 Complex representation 6087.6.2 The multiplication theorem and Parsevals theorem 6127.6.3 Discrete frequency spectra 6157.6.4 Power spectrum 6217.6.5 Exercises (3439) 623

7.7 Orthogonal functions 624

7.7.1 Definitions 6247.7.2 Generalized Fourier series 6267.7.3 Convergence of generalized Fourier series 6277.7.4 Exercises (4046) 629

7.8 Engineering application: describing functions 632



Chapter 8 The Fourier Transform 637

xiv CONTENTS

8.2 The Fourier transform 638

8.2.1 The Fourier integral 6388.2.2 The Fourier transform pair 6448.2.3 The continuous Fourier spectra 6488.2.4 Exercises (110) 651

8.3 Properties of the Fourier transform 652

8.3.1 The linearity property 6528.3.2 Time-differentiation property 6528.3.3 Time-shift property 6538.3.4 Frequency-shift property 6548.3.5 The symmetry property 6558.3.6 Exercises (1116) 657

8.4 The frequency response 658

8.4.1 Relationship between Fourier and Laplace transforms 6588.4.2 The frequency response 6608.4.3 Exercises (1721) 663

8.5 Transforms of the step and impulse functions 663

8.5.1 Energy and power 6638.5.2 Convolution 6738.5.3 Exercises (2227) 675

8.6 The Fourier transform in discrete time 676

8.6.1 Introduction 6768.6.2 A Fourier transform for sequences 6768.6.3 The discrete Fourier transform 6808.6.4 Estimation of the continuous Fourier transform 6848.6.5 The fast Fourier transform 6938.6.6 Exercises (2831) 700

8.7 Engineering application: the design of analogue filters 700

8.8 Engineering application: modulation, demodulation and frequency-domain filtering 703

8.8.1 Introduction 7038.8.2 Modulation and transmission 7058.8.3 Identification and isolation of the information-

carrying signal 7068.8.4 Demodulation stage 7078.8.5 Final signal recovery 7088.8.6 Further developments 709

CONTENTS xv

8.9 Engineering application: direct design of digital filters and windows 709

8.9.1 Digital filters 7098.9.2 Windows 7158.9.3 Exercises (3233) 719



9.2 General discussion 725

9.2.1 Wave equation 7259.2.2 Heat-conduction or diffusion equation 7289.2.3 Laplace equation 7319.2.4 Other and related equations 7339.2.5 Arbitrary functions and first-order equations 7359.2.6 Exercises (114) 740

9.3 Solution of the wave equation 742

9.3.1 DAlembert solution and characteristics 7429.3.2 Separated solutions 7519.3.3 Laplace transform solution 7569.3.4 Exercises (1527) 7599.3.5 Numerical solution 7619.3.6 Exercises (2831) 767

9.4 Solution of the heat-conduction/diffusion equation 768

9.4.1 Separation method 7689.4.2 Laplace transform method 7729.4.3 Exercises (3240) 7779.4.4 Numerical solution 7799.4.5 Exercises (4143) 785

9.5 Solution of the Laplace equation 785

9.5.1 Separated solutions 7859.5.2 Exercises (4454) 7939.5.3 Numerical solution 7949.5.4 Exercises (5559) 801

9.6 Finite elements 802

9.6.1 Exercises (6062) 814

Chapter 9 Partial Differential Equations 723

xvi CONTENTS

9.7 Integral solutions 815

9.7.1 Separated solutions 8159.7.2 Use of singular solutions 8179.7.3 Sources and sinks for the heat conduction equation 8209.7.4 Exercises (6367) 823

9.8 General considerations 824

9.8.1 Formal classification 8249.8.2 Boundary conditions 8269.8.3 Exercises (6874) 831

9.9 Engineering application: wave propagation under a moving load 831

9.10 Engineering application: blood-flow model 834



10.2 Linear programming 847

10.2.1 Introduction 84710.2.2 Simplex algorithm: an example 84910.2.3 Simplex algorithm: general theory 85310.2.4 Exercises (111) 86010.2.5 Two-phase method 86110.2.6 Exercises (1220) 869

10.3 Lagrange multipliers 870

10.3.1 Equality constraints 87010.3.2 Inequality constraints 87410.3.3 Exercises (2128) 875

10.4 Hill climbing 875

10.4.1 Single-variable search 87510.4.2 Exercises (2934) 88110.4.3 Simple multivariable searches 88210.4.4 Exercises (3539) 88710.4.5 Advanced multivariable searches 88810.4.6 Least squares 89210.4.7 Exercises (4043) 895

Chapter 10 Optimization 843

CONTENTS xvii

10.5 Engineering application: chemical processing plant 896

10.6 Engineering application: heating fin 898



11.2 Review of basic probability theory 906

11.2.1 The rules of probability 90711.2.2 Random variables 90711.2.3 The Bernoulli, binomial and Poisson distributions 90911.2.4 The normal distribution 91011.2.5 Sample measures 911

11.3 Estimating parameters 912

11.3.1 Interval estimates and hypothesis tests 91211.3.2 Distribution of the sample average 91311.3.3 Confidence interval for the mean 91411.3.4 Testing simple hypotheses 91711.3.5 Other confidence intervals and tests concerning means 91811.3.6 Interval and test for proportion 92211.3.7 Exercises (113) 924

11.4 Joint distributions and correlation 925

11.4.1 Joint and marginal distributions 92611.4.2 Independence 92811.4.3 Covariance and correlation 92911.4.4 Sample correlation 93311.4.5 Interval and test for correlation 93511.4.6 Rank correlation 93611.4.7 Exercises (1424) 937

11.5 Regression 938

11.5.1 The method of least squares 93911.5.2 Normal residuals 94111.5.3 Regression and correlation 94311.5.4 Nonlinear regression 94311.5.5 Exercises (2533) 945

11.6 Goodness-of-fit tests 946

11.6.1 Chi-square distribution and test 946

Chapter 11 Applied Probability and Statistics 905

xviii CONTENTS

11.6.2 Contingency tables 94911.6.3 Exercises (3442) 951

11.7 Moment generating functions 953

11.7.1 Definition and simple applications 95311.7.2 The Poisson approximation to the binomial 95511.7.3 Proof of the central limit theorem 95611.7.4 Exercises (4347) 957

11.8 Engineering application: analysis of engine performance data 958

11.8.1 Introduction 95811.8.2 Difference in mean running times and temperatures 95911.8.3 Dependence of running time on temperature 96011.8.4 Test for normality 96211.8.5 Conclusions 963

11.9 Engineering application: statistical quality control 964

11.9.1 Introduction 96411.9.2 Shewhart attribute control charts 96411.9.3 Shewhart variable control charts 96711.9.4 Cusum control charts 96811.9.5 Moving-average control charts 97111.9.6 Range charts 97311.9.7 Exercises (4859) 973

11.10 Poisson processes and the theory of queues 974

11.10.1 Typical queueing problems 97411.10.2 Poisson processes 97511.10.3 Single service channel queue 97811.10.4 Queues with multiple service channels 98211.10.5 Queueing system simulation 98311.10.6 Exercises (6067) 985

11.11 Bayes theorem and its applications 986

11.11.1 Derivation and simple examples 98611.11.2 Applications in probabilistic inference 98811.11.3 Exercises (6878) 991


Answers to Exercises 995

Index 1023

Preface

Throughout the course of history, engineering and mathematics have developed inparallel. All branches of engineering depend on mathematics for their description andthere has been a steady flow of ideas and problems from engineering that has stimulatedand sometimes initiated branches of mathematics. Thus it is vital that engineering stu-dents receive a thorough grounding in mathematics, with the treatment related to theirinterests and problems. As with the previous editions, this has been the motivation forthe production of this fourth edition a companion text to the fourth edition of ModernEngineering Mathematics, this being designed to provide a first-level core studiescourse in mathematics for undergraduate programmes in all engineering disciplines.Building on the foundations laid in the companion text, this book gives an extensivetreatment of some of the more advanced areas of mathematics that have applications invarious fields of engineering, particularly as tools for computer-based system model-ling, analysis and design. Feedback, from users of the previous editions, on subjectcontent has been highly positive indicating that it is sufficiently broad to provide thenecessary second-level, or optional, studies for most engineering programmes, wherein each case a selection of the material may be made. Whilst designed primarily for useby engineering students, it is believed that the book is also suitable for use by studentsof applied mathematics and the physical sciences.

Although the pace of the book is at a somewhat more advanced level than the com-panion text, the philosophy of learning by doing is retained with continuing emphasison the development of students ability to use mathematics with understanding to solveengineering problems. Recognizing the increasing importance of mathematical model-ling in engineering practice, many of the worked examples and exercises incorporatemathematical models that are designed both to provide relevance and to reinforce therole of mathematics in various branches of engineering. In addition, each chapter con-tains specific sections on engineering applications, and these form an ideal frameworkfor individual, or group, study assignments, thereby helping to reinforce the skills ofmathematical modelling, which are seen as essential if engineers are to tackle theincreasingly complex systems they are being called upon to analyse and design. Theimportance of numerical methods in problem solving is also recognized, and its treat-ment is integrated with the analytical work throughout the book.

Much of the feedback from users relates to the role and use of software packages,particularly symbolic algebra packages. Without making it an essential requirement theauthors have attempted to highlight throughout the text situations where the user couldmake effective use of software. This also applies to exercises and, indeed, a limitednumber have been introduced for which the use of such a package is essential. Whilstany appropriate piece of software can be used, the authors recommend the use ofMATLAB and/or MAPLE. In this new edition more copious reference to the use of these

xx PREFACE

two packages is made throughout the text, with commands or codes introduced andillustrated. When indicated, students are strongly recommended to use these packagesto check their solutions to exercises. This is not only to help develop proficiency in theiruse, but also to enable students to appreciate the necessity of having a sound knowledgeof the underpinning mathematics if such packages are to be used effectively. Throughoutthe book two icons are used:

An open screen indicates that the use of a software package would be useful(e.g. for checking solutions) but not essential.

A closed screen indicates that the use of a software package is essential orhighly desirable.

As indicated earlier, feedback on content from users of previous editions has beenfavourable, and consequently no new chapter has been introduced. However, inresponse to feedback the order of presentation of chapters has been changed, with aview to making it more logical and appealing to users. This re-ordering has necessitatedsome redistribution of material both within and across some of the chapters. Anothernew feature is the introduction of the use of colour. It is hoped that this will make the textmore accessible and student-friendly. Also, in response to feedback individual chaptershave been reviewed and updated accordingly. The most significant changes are:

Chapter 1 Matrix Analysis: Inclusion of new sections on Singular value decom-position and Lyapunov stability analysis.

Chapter 5 Laplace transform: Following re-ordering of chapters a more unifiedand extended treatment of transfer functions/transfer matrices for continuous-time state-space models has been included.

Chapter 6 Z-transforms: Inclusion of a new section on Discretization ofcontinuous-time state-space models.

Chapter 8 Fourier transform: Inclusion of a new section on Direct design ofdigital filters and windows.

Chapter 9 Partial differential equations: The treatment of first order equationshas been extended and a new section on Integral solution included.

Chapter 10 Optimization: Inclusion of a new section on Least squares.A comprehensive Solutions Manual is available free of charge to lecturers adopting this

textbook. It will also be available for download via the Web at: www.pearsoned.co.ck/james.

AcknowledgementsThe authoring team is extremely grateful to all the reviewers and users of the text whohave provided valuable comments on previous editions of this book. Most of this hasbeen highly constructive and very much appreciated. The team has continued to enjoythe full support of a very enthusiastic production team at Pearson Education and wishesto thank all those concerned. Finally I would like to thank my wife, Dolan, for her fullsupport throughout the preparation of this text and its previous editions.

Glyn JamesCoventry UniversityJuly 2010

About the Authors

Glyn James retired as Dean of the School of Mathematical and Information Sciencesat Coventry University in 2001 and is now Emeritus Professor in Mathematics at theUniversity. He graduated from the University College of Wales, Cardiff in the late 1950s,obtaining first class honours degrees in both Mathematics and Chemistry. He obtaineda PhD in Engineering Science in 1971 as an external student of the University of Warwick.He has been employed at Coventry since 1964 and held the position of the Head ofMathematics Department prior to his appointment as Dean in 1992. His research interestsare in control theory and its applications to industrial problems. He also has a keeninterest in mathematical education, particularly in relation to the teaching of engineer-ing mathematics and mathematical modelling. He was co-chairman of the EuropeanMathematics Working Group established by the European Society for EngineeringEducation (SEFI) in 1982, a past chairman of the Education Committee of the Instituteof Mathematics and its Applications (IMA), and a member of the Royal Society Mathe-matics Education Subcommittee. In 1995 he was chairman of the Working Group thatproduced the report Mathematics Matters in Engineering on behalf of the professionalbodies in engineering and mathematics within the UK. He is also a member of theeditorial/advisory board of three international journals. He has published numerouspapers and is co-editor of five books on various aspects of mathematical modelling. Heis a past Vice-President of the IMA and has also served a period as Honorary Secretaryof the Institute. He is a Chartered Mathematician and a Fellow of the IMA.

David Burley retired from the University of Sheffield in 1998. He graduated in mathe-matics from Kings College, University of London in 1955 and obtained his PhD inmathematical physics. After working in the University of Glasgow, he spent most of hisacademic career in the University of Sheffield, being Head of Department for six years.He has long experience of teaching engineering students and has been particularlyinterested in encouraging students to construct mathematical models in physical andbiological contexts to enhance their learning. His research work has ranged throughstatistical mechanics, optimization and fluid mechanics. He has particular interest in theflow of molten glass in a variety of situations and the application of results in the glassindustry. Currently he is involved in a large project concerning heat transfer problemsin the deep burial of nuclear waste.

Dick Clements is Emeritus Professor in the Department of Engineering Mathematicsat Bristol University. He read for the Mathematical Tripos, matriculating at ChristsCollege, Cambridge in 1966. He went on to take a PGCE at Leicester University Schoolof Education (196970) before returning to Cambridge to research a PhD in AeronauticalEngineering (197073). In 1973 he was appointed Lecturer in Engineering Mathematicsat Bristol University and has taught mathematics to engineering students ever since,

xxii ABOUT THE AUTHORS

becoming successively Senior Lecturer, Reader and Professorial Teaching Fellow. He hasundertaken research in a wide range of engineering topics but is particularly interestedin mathematical modelling and in new approaches to the teaching of mathematics toengineering students. He has published numerous papers and one previous book, Math-ematical Modelling: A Case Study Approach. He is a Chartered Engineer, a CharteredMathematician, a member of the Royal Aeronautical Society, a Fellow of the Instituteof Mathematics and Its Applications, an Associate Fellow of the Royal Institute ofNavigation, and a Fellow of the Higher Education Academy. He retired from full time workin 2007 but continues to teach and pursue his research interests on a part time basis.

Phil Dyke is Professor of Applied Mathematics at the University of Plymouth. He wasHead of School of Mathematics and Statistics for 18 years then Head of School ofComputing, Communications and Electronics for four years but he now devotes histime to teaching and research. After graduating with a first in mathematics he gaineda PhD in coastal engineering modelling. He has over 35 years experience teachingundergraduates, most of this teaching to engineering students. He has run an interna-tional research group since 1981 applying mathematics to coastal engineering and shal-low sea dynamics. Apart from contributing to these engineering mathematics books, hehas written seven textbooks on mathematics and marine science, and still enjoys tryingto solve environmental problems using simple mathematical models.

John Searl was Director of the Edinburgh Centre for Mathematical Education at theUniversity of Edinburgh before his recent retirement. As well as lecturing on mathemat-ical education, he taught service courses for engineers and scientists. His most recentresearch concerned the development of learning environments that make for the effectivelearning of mathematics for 1620 year olds. As an applied mathematician who workedcollaboratively with (among others) engineers, physicists, biologists and pharmacologists,he is keen to develop the problem-solving skills of students and to provide them withopportunities to display their mathematical knowledge within a variety of practical con-texts. These contexts develop the extended reasoning needed in all fields of engineering.

Nigel Steele was Head of Mathematics at Coventry University until his retirement in2004. He has had a career-long interest in engineering mathematics and its teaching,particularly to electrical and control engineers. Since retirement he has been EmeritusProfessor of Mathematics at Coventry, combining this with the duties of HonorarySecretary of the Institute of Mathematics and its Applications. Having responsibility forthe Institutes education matters he became heavily involved with a highly successfulproject aimed at encouraging more people to study for mathematics and other maths-richcourses (for example Engineering) at University. He also assisted in the developmentof the mathematics content for the advanced Engineering Diploma, working to ensurethat students were properly prepared for the study of Engineering in Higher Education.

Jerry Wright is a Lead Member of Technical Staff at the AT&T Shannon Laboratory, NewJersey, USA. He graduated in Engineering (BSc and PhD at the University of Southampton)and in Mathematics (MSc at the University of London) and worked at the National PhysicalLaboratory before moving to the University of Bristol in 1978. There he acquired wideexperience in the teaching of mathematics to students of engineering, and became SeniorLecturer in Engineering Mathematics. He held a Royal Society Industrial Fellowshipfor 1994, and is a Fellow of the Institute of Mathematics and its Applications. In 1996 hemoved to AT&T Labs (formerly part of Bell labs) to continue his research in spokenlanguage understanding, human/computer dialog systems, and data mining.

Publishers Acknowledgements

We are grateful to the following for permission to reproduce copyright material:

TextExtract from Signal Processing in Electronic Communications, ISBN 1898563233, 1 ed.,Woodhead Publishing Ltd (Chapman, N, Goodhall, D, Steele, N).

In some instances we have been unable to trace the owners of copyright material, andwe would appreciate any information that would enable us to do so.

1 Matrix Analysis

Chapter 1 Contents

1.1 Introduction 2

1.2 Review of matrix algebra 2

1.3 Vector spaces 10

1.4 The eigenvalue problem 14

1.5 Numerical methods 30

1.6 Reduction to canonical form 39

1.7 Functions of a matrix 54

1.8 Singular value decomposition 66

1.9 State-space representation 82

1.10 Solution of the state equation 89

1.11 Engineering application: Lyapunov stability analysis 104

1.12 Engineering application: capacitor microphone 107


2 MATRIX ANALYSIS

IntroductionIn this chapter we turn our attention again to matrices, first considered in Chapter 5of Modern Engineering Mathematics, and their applications in engineering. At theoutset of the chapter we review the basic results of matrix algebra and briefly introducevector spaces.

As the reader will be aware, matrices are arrays of real or complex numbers, and havea special, but not exclusive, relationship with systems of linear equations. An (incorrect)initial impression often formed by users of mathematics is that mathematicians havesomething of an obsession with these systems and their solution. However, such systemsoccur quite naturally in the process of numerical solution of ordinary differential equa-tions used to model everyday engineering processes. In Chapter 9 we shall see that theyalso occur in numerical methods for the solution of partial differential equations, forexample those modelling the flow of a fluid or the transfer of heat. Systems of linearfirst-order differential equations with constant coefficients are at the core of the state-space representation of linear system models. Identification, analysis and indeed designof such systems can conveniently be performed in the state-space representation, withthis form assuming a particular importance in the case of multivariable systems.

In all these areas it is convenient to use a matrix representation for the systems underconsideration, since this allows the system model to be manipulated following the rulesof matrix algebra. A particularly valuable type of manipulation is simplification in somesense. Such a simplification process is an example of a system transformation, carriedout by the process of matrix multiplication. At the heart of many transformations arethe eigenvalues and eigenvectors of a square matrix. In addition to providing the meansby which simplifying transformations can be deduced, system eigenvalues provide vitalinformation on system stability, fundamental frequencies, speed of decay and long-termsystem behaviour. For this reason, we devote a substantial amount of space to theprocess of their calculation, both by hand and by numerical means when necessary. Ourtreatment of numerical methods is intended to be purely indicative rather than complete,because a comprehensive matrix algebra computational tool kit, such as MATLAB, isnow part of the essential armoury of all serious users of mathematics.

In addition to developing the use of matrix algebra techniques, we also demonstratethe techniques and applications of matrix analysis, focusing on the state-space system modelwidely used in control and systems engineering. Here we encounter the idea of a functionof a matrix, in particular the matrix exponential, and we see again the role of theeigenvalues in its calculation. This edition also includes a section on singular valuedecomposition and the pseudo inverse, together with a brief section on Lyapunov stabilityof linear systems using quadratic forms.

Review of matrix algebraThis section contains a summary of the definitions and properties associated with matricesand determinants. A full account can be found in chapters of Modern EngineeringMathematics or elsewhere. It is assumed that readers, prior to embarking on this chapter,have a fairly thorough understanding of the material summarized in this section.

1.1

1.2

1.2 REVIEW OF MATRIX ALGEBRA 3

1.2.1 Definitions

(a) An array of real numbers

is called an m n matrix with m rows and n columns. The aij is referred to as thei, jth element and denotes the element in the ith row and jth column. If m = nthen A is called a square matrix of order n. If the matrix has one column or onerow then it is called a column vector or a row vector respectively.

(b) In a square matrix A of order n the diagonal containing the elements a11, a22, . . . ,ann is called the principal or leading diagonal. The sum of the elements in thisdiagonal is called the trace of A, that is

(c) A diagonal matrix is a square matrix that has its only non-zero elements along theleading diagonal. A special case of a diagonal matrix is the unit or identity matrix Ifor which a11 = a22 = . . . = ann = 1.

(d) A zero or null matrix 0 is a matrix with every element zero.

(e) The transposed matrix AT is the matrix A with rows and columns interchanged,its i, jth element being aji.

(f ) A square matrix A is called a symmetric matrix if AT = A. It is called skewsymmetric if AT = A.

1.2.2 Basic operations on matrices

In what follows the matrices A, B and C are assumed to have the i, jth elements aij, bijand cij respectively.

Equality

The matrices A and B are equal, that is A = B, if they are of the same order m nand

aij = bij, 1 i m, 1 j n

Multiplication by a scalar

If is a scalar then the matrix A has elements aij.

A

a11 a12 a13 6 a1n

a21 a22 a23 6 a2n

7 7 7 7 7

am1 am2 am3 6 amn

=

trace A aiii=1

n

=

4 MATRIX ANALYSIS

Addition

We can only add an m n matrix A to another m n matrix B and the elements of thesum A + B are

aij + bij, 1 i m, 1 j n

Properties of addition

(i) commutative law: A + B = B + A

(ii) associative law: (A + B ) + C = A + (B + C )

(iii) distributive law: (A + B ) = A + B, scalar

Matrix multiplication

If A is an m p matrix and B a p n matrix then we define the product C = AB as them n matrix with elements

, i = 1, 2, . . . , m; j = 1, 2, . . . , n

Properties of multiplication

(i) The commutative law is not satisfied in general; that is, in general AB BA.Order matters and we distinguish between AB and BA by the terminology:pre-multiplication of B by A to form AB and post-multiplication of B by A toform BA.

(ii) Associative law: A(BC ) = (AB )C

(iii) If is a scalar then

(A)B = A(B ) = AB

(iv) Distributive law over addition:

(A + B )C = AC + BC

A(B + C ) = AB + AC

Note the importance of maintaining order of multiplication.

(v) If A is an m n matrix and if Im and In are the unit matrices of order m and nrespectively then

ImA = AIn = A

Properties of the transpose

If AT is the transposed matrix of A then

(i) (A + B )T = AT + BT

(ii) (AT)T = A

(iii) (AB )T = BTAT

cij = aikbkjk=1

p


1.2.3 Determinants

The determinant of a square n n matrix A is denoted by det A or | A |.If we take a determinant and delete row i and column j then the determinant

remaining is called the minor Mij of the i, jth element. In general we can take any rowi (or column) and evaluate an n n determinant | A | as

A minor multiplied by the appropriate sign is called the cofactor Aij of the i, jth elementso Aij = (1)i+j Mij and thus

Some useful properties

(i) | AT | = | A |

(ii) | AB | = | A | | B |

(iii) A square matrix A is said to be non-singular if | A | 0 and singular if | A | = 0.

1.2.4 Adjoint and inverse matrices

Adjoint matrix

The adjoint of a square matrix A is the transpose of the matrix of cofactors, so for a3 3 matrix A

Properties

(i) A(adj A) = | A |I

(ii) | adj A | = | A | n1, n being the order of A

(iii) adj (AB ) = (adj B )(adj A)

Inverse matrix

Given a square matrix A if we can construct a square matrix B such that

BA = AB = I

then we call B the inverse of A and write it as A1.

| A | 1( )i+j aijMijj=1

n

=

| A | aij Aijj=1

n

=

adj AA11 A12 A13A21 A22 A23A31 A32 A33

T

=

6 MATRIX ANALYSIS

Properties

(i) If A is non-singular then |A | 0 and A1 = (adj A)/|A |.

(ii) If A is singular then |A | = 0 and A1 does not exist.

(iii) (AB )1 = B 1A1.

All the basic matrix operations may be implemented in MATLAB and MAPLEusing simple commands. In MATLAB a matrix is entered as an array, with rowelements separated by spaces (or commas) and each row of elements separated by asemicolon(;), or the return key to go to a new line. Thus, for example,

A=[1 2 3; 4 0 5; 7 4 2]

gives

A=

1 2 3

4 0 5

7 4 2

Having specified the two matrices A and B the operations of addition, subtractionand multiplication are implemented using respectively the commands

C=A+B, C=A-B, C=A*B

The trace of the matrix A is determined by the command trace(A), and itsdeterminant by det(A).

Multiplication of a matrix A by a scalar is carried out using the command *, whileraising A to a given power is carried out using the command ^ . Thus, for example,3A2 is determined using the command C=3*A^2.

The transpose of a real matrix A is determined using the apostrophe key; thatis C=A (to accommodate complex matrices the command C=A. should be used).The inverse of A is determined by C=inv(A).

For matrices involving algebraic quantities, or when exact arithmetic is desirableuse of the Symbolic Math Toolbox is required; in which matrices must be expressedin symbolic form using the sym command. The command A=sym(A) generates thesymbolic form of A. For example, for the matrix

the commands

A=[2.1 3.2 0.6; 1.2 0.5 3.3; 5.2 1.1 0];

A=sym(A)

generate

A=

[21/10, 16/5, 3/5]

[6/5, 1/2, 33/10]

[26/5, 11/10, 0]

Symbolic manipulation can also be undertaken in MATLAB using the MuPADversion of Symbolic Math Toolbox.

A ==== 2.1 3.2 0.6

1.2 0.5 3.3

5.2 1.1 0


1.2.5 Linear equations

In this section we reiterate some definitive statements about the solution of the systemof simultaneous linear equations

a11x1 + a12x2 + . . . + a1nxn = b1a21x1 + a22x2 + . . . + a2nxn = b2

7 7

an1x1 + an2x2 + . . . + annxn = bn

There are several ways of setting up arrays in MAPLE; the easiest is to use thelinear algebra package LinearAlgebra so, for example, the commands:

with(LinearAlgebra):

A:=Matrix([[1,2,3],[4,0,5],[7,6,2]]);

return

with the command

b:=Vector([2,3,1]);

returning

Having specified two matrices A and B addition and subtraction are implementedusing the commands:

C:=A+B; and C:=AB;

Multiplication of a matrix A by a scalar k is implemented using the command k*A;so, for example, (2A + 3B ) is implemented by

2*A+3*B;

The product AB of two matrices is implemented by either of the following twocommands:

A.B; or Multiply(A,B);

(Note: A*B will not work)The transpose, trace, determinant, adjoint and inverse of a matrix A are returned

using, respectively, the commands:

Transpose(A);

Trace(A);

Determinant(A);

Adjoint(A);

MatrixInverse(A);

A =

1 2 3

4 0 5

7 6 2

b =

2

3

1

8

MATRIX ANALYSIS

or, in matrix notation,

that is,

A

x

=

b

(1.1)

where

A

is the matrix of coefficients and

x

is the vector of unknowns. If

b

=

0 theequations are called

homogeneous

, while if

b

0 they are called

nonhomogeneous

(or

inhomogeneous

). Considering individual cases:

Case (i)

If

b

0 and

|

A

|

0 then we have a unique solution

x

=

A

1

b

.

Case (ii)

If

b

=

0 and

|

A

|

0 we have the trivial solution

x

=

0.

Case (iii)

If

b

0 and

|

A

|

=

0 then we have two possibilities:

either

the equations are inconsistentand we have no solution

or

we have infinitely many solutions.

Case (iv)

If

b

=

0 and

|

A

|

=

0 then we have infinitely many solutions.

Case (iv) is one of the most important, since from it we can deduce the importantresult that

the homogeneous equation

A

x

=

0

has a non-trivial solution if and onlyif

|

A

|

=

0.

Provided that a solution to (1.1) exists it may be determined in MATLAB using thecommand

x=A\b

. For example, the system of simultaneous equations

x

+

y

+

z

=

6,

x

+

2

y

+

3

z = 14, x + 4y + 9z = 36

may be written in the matrix form

Entering A and b and using the command x = A\b provides the answer x = 1, y = 2, z = 3.

a11 a12 6 a1n a21 a22 6 a2n 7 7 7

an1 an2 6 ann

x1

x2

7

xn

b1

b2

7

bn

=

1 1 1

1 2 3

1 4 9

A

x

y

z

x

=

6

14

36

b


1.2.6 Rank of a matrix

The most commonly used definition of the rank, rank A, of a matrix A is that it is the orderof the largest square submatrix of A with a non-zero determinant, a square submatrixbeing formed by deleting rows and columns to form a square matrix. Unfortunately itis not always easy to compute the rank using this definition and an alternative definition,which provides a constructive approach to calculating the rank, is often adopted. First,using elementary row operations, the matrix A is reduced to echelon form

in which all the entries below the line are zero, and the leading element, marked *, ineach row above the line is non-zero. The number of non-zero rows in the echelon formis equal to rank A.

When considering the solution of equations (1.1) we saw that provided the determinantof the matrix A was not zero we could obtain explicit solutions in terms of the inverse matrix.However, when we looked at cases with zero determinant the results were much less clear.The idea of the rank of a matrix helps to make these results more precise. Defining theaugmented matrix (A : b) for (1.1) as the matrix A with the column b added to it thenwe can state the results of cases (iii) and (iv) of Section 1.2.5 more clearly as follows:

In MAPLE the commands


soln:=LinearSolve(A,b);

will solve the set of linear equations Ax = b for the unknown x when A, b given.Thus for the above set of equations the commands


A:=Matrix([[1,1,1],[1,2,3],[1,4,9]]);

b:=Vector([6,14,36]);

x:=LinearSolve(A,b);

return

x =

1

2

3

If

A

and (

A

:

b

) have different rank then we have no solution to (1.1). If the twomatrices have the same rank then a solution exists, and furthermore the solutionwill contain a number of free parameters equal to (

n

rank

A

).

10

MATRIX ANALYSIS

Vector spaces

Vectors and matrices form part of a more extensive formal structure called a vector space.The theory of vector spaces underpins many modern approaches to numerical methodsand the approximate solution of many of the equations that arise in engineering analysis.In this section we shall, very briefly, introduce some of the basic ideas of vector spacesnecessary for later work in this chapter.

Definition

A

real vector space

V

is a set of objects called

vectors

together with rules for additionand multiplication by real numbers. For any three vectors

a

,

b

and

c

in

V

and any realnumbers

and

the sum

a

+

b

and the product

a

also belong to

V

and satisfy thefollowing axioms:

In MATLAB the rank of the matrix

A

is generated using the command

rank(A)

.For example, if

the commands

A=[-1 2 2; 0 0 1; -1 2 0];

rank(A)

generate

ans=2

In MAPLE the command is also rank(A).

A ==== 1 2 2

0 0 11 2 0

1.3

(a)

a

+

b

=

b

+

a

(b)

a

+

(

b

+

c

)

=

(

a

+

b

)

+

c

(c) there exists a zero vector

0

such that

a

+

0

=

a

(d) for each

a

in

V

there is an element

a

in

V such that

a + (a) = 0

(e) (a + b) = a + b

(f ) ( + )a = a + a

(g) ()a = (a)

(h) 1a = a

1.3 VECTOR SPACES 11

It is clear that the real numbers form a vector space. The properties given are alsosatisfied by vectors and by m n matrices so vectors and matrices also form vectorspaces. The space of all quadratics a + bx + cx2 forms a vector space, as can be estab-lished by checking the axioms, (a)(h). Many other common sets of objects also formvector spaces. If we can obtain useful information from the general structure then thiswill be of considerable use in specific cases.

1.3.1 Linear independence

The idea of linear dependence is a general one for any vector space. The vector x is saidto be linearly dependent on x1, x2, . . . , xm if it can be written as

x = 1x1 + 2x2 + . . . + mxmfor some scalars 1, . . . , m. The set of vectors y1, y2, . . . , ym is said to be linearlyindependent if and only if

1y1 + 2 y2 + . . . + m ym = 0

implies that 1 = 2 = . . . = m = 0.Let us now take a linearly independent set of vectors x1, x2, . . . , xm in V and con-

struct a set consisting of all vectors of the form

x = 1x1 + 2x2 + . . . + mxmWe shall call this set S(x1, x2, . . . , xm). It is clearly a vector space, since all the axiomsare satisfied.

Show that

and

form a linearly independent set and describe S(e1, e2) geometrically.

Solution We have that

is only satisfied if = = 0, and hence e1 and e2 are linearly independent.

S(e1, e2) is the set of all vectors of the form , which is just the (x1, x2)

plane and is a subset of the three-dimensional Euclidean space.

Example 1.1

e1

1

0

0

= e2

0

1

0

=

0 e1 e2+0

= =

0

12 MATRIX ANALYSIS

If we can find a set B of linearly independent vectors x1, x2, . . . , xn in V such that

S(x1, x2, . . . , xn) = V

then B is called a basis of the vector space V. Such a basis forms a crucial part of thetheory, since every vector x in V can be written uniquely as

x = 1x1 + 2x2 + . . . + nxnThe definition of B implies that x must take this form. To establish uniqueness, let usassume that we can also write x as

x = 1x1 + 2x2 + . . . + nxnThen, on subtracting,

0 = (1 1)x1 + . . . + (n n)xnand since x1, . . . , xn are linearly independent, the only solution is 1 = 1, 2 = 2, . . . ;hence the two expressions for x are the same.

It can also be shown that any other basis for V must also contain n vectors and thatany n + 1 vectors must be linearly dependent. Such a vector space is said to havedimension n (or infinite dimension if no finite n can be found). In a three-dimensionalEuclidean space

, ,

form an obvious basis, and

, ,

is also a perfectly good basis. While the basis can change, the number of vectors in thebasis, three in this case, is an intrinsic property of the vector space. If we consider thevector space of quadratics then the sets of functions {1, x, x2} and {1, x 1, x(x 1)}are both bases for the space, since every quadratic can be written as a + bx + cx2 or asA + B(x 1) + Cx(x 1). We note that this space is three-dimensional.

1.3.2 Transformations between bases

Since any basis of a particular space contains the same number of vectors, we can lookat transformations from one basis to another. We shall consider a three-dimensionalspace, but the results are equally valid in any number of dimensions. Let e1, e2, e3 ande 1, e 2, e 3 be two bases of a space. From the definition of a basis, the vectors e 1, e 2 and e 3can be written in terms of e1, e2 and e3 as

(1.2)

e1

1

0

0

= e2

0

1

0

= e3

0

0

1

=

d1

1

0

0

= d2

1

1

0

= d3

1

1

1

=

e1 a11e1 a21e2 a31e3+ +=e2 a12e2 a22e2 a32e3+ +=e3 a13e3 a23e2 a33e3+ +=

1.3 VECTOR SPACES 13

Taking a typical vector x in V, which can be written both as

x = x1e1 + x2e2 + x3e3 (1.3)

and as

x = x1e1 + x2e2 + x3e3

we can use the transformation (1.2) to give

x = x1(a11e1 + a21e2 + a31e3) + x2(a12e1 + a22e2 + a32e3) + x3(a13e1 + a23e2 + a33e3)

= (x1a11 + x2a12 + x3a13)e1 + (x1a21 + x2a22 + x3a23)e2 + (x1a31 + x2a32 + x3a33)e3

On comparing with (1.3) we see that

x1 = a11x1 + a12x2 + a13x3

x2 = a21x1 + a22x2 + a23x3

x3 = a31x1 + a32x2 + a33x3

or

x = Ax

Thus changing from one basis to another is equivalent to transforming the coordinatesby multiplication by a matrix, and we thus have another interpretation of matrices.Successive transformations to a third basis will just give x = Bx, and hence thecomposite transformation is x = (AB )x and is obtained through the standard matrixrules.

For convenience of working it is usual to take mutually orthogonal vectors as abasis, so that and = ij, where ij is the Kronecker delta

Using (1.2) and multiplying out these orthogonality relations, we have

Hence

or in matrix form

ATA = I

It should be noted that such a matrix A with A1 = AT is called an orthogonalmatrix.

eiTej = ij ei

T ej

ij1 if i = j0 if i j

=

eiT ej akiek

T apjepp

k

= akiapjekTepp

k

akiapjkpp

k

akiakjk

= = =

akiakjk

ij=

14 MATRIX ANALYSIS

The eigenvalue problemA problem that leads to a concept of crucial importance in many branches of math-ematics and its applications is that of seeking non-trivial solutions x 0 to the matrixequation

Ax = x

This is referred to as the eigenvalue problem; values of the scalar for which non-trivial solutions exist are called eigenvalues and the corresponding solutions x 0 arecalled the eigenvectors. Such problems arise naturally in many branches of engineering.For example, in vibrations the eigenvalues and eigenvectors describe the frequency andmode of vibration respectively, while in mechanics they represent principal stressesand the principal axes of stress in bodies subjected to external forces. In Section 1.11,and later in Section 5.7.1, we shall see that eigenvalues also play an important role inthe stability analysis of dynamical systems.

For continuity some of the introductory material on eigenvalues and eigenvectors,contained in Chapter 5 of Modern Engineering Mathematics, is first revisited.

1.4

Which of the following sets form a basis for a three-dimensional Euclidean space?

(a) , , (b) , ,

(c) , ,

Given the unit vectors

, ,

find the transformation that takes these to the vectors

, ,

Under this, how does the vector x = x1e1 + x2e2 + x3e3 transform and what is the geometrical interpretation? What lines transform into scalar multiples of themselves?

Show that the set of all cubic polynomials forms a vector space. Which of the following sets of functions are bases of that space?

(a) {1, x, x2, x3}

(b) {1 x, 1 + x, 1 x3, 1 + x3}

(c) {1 x, 1 + x, x2(1 x), x2(1 + x)}

(d) {x(1 x), x(1 + x), 1 x3, 1 + x3}

(e) {1 + 2x, 2x + 3x2, 3x2 + 4x3, 4x3 + 1}

Describe the vector space

S(x + 2x3, 2x 3x5, x + x3)

What is its dimension?

1.3.3 Exercises

1

1

0

0

1

2

0

1

2

3

1

0

1

1

2

3

3

2

5

1

0

0

1

1

0

2

1

0

2

e1

1

0

0

= e2

0

1

0

= e3

0

0

1

=

e11

2------=

1

1

0

e21

2------=

1

10

e30

0

1

=

3

4

1.4 THE EIGENVALUE PROBLEM 15

1.4.1 The characteristic equation

The set of simultaneous equations

Ax = x (1.4)

where A is an n n matrix and x = [x1 x2 . . . xn]T is an n 1 column vector canbe written in the form

(I A)x = 0 (1.5)

where I is the identity matrix. The matrix equation (1.5) represents simply a set ofhomogeneous equations, and we know that a non-trivial solution exists if

Here c() is the expansion of the determinant and is a polynomial of degree n in ,called the characteristic polynomial of A. Thus

c() = n + cn1n1 + cn2n2 + . . . + c1 + c0and the equation c() = 0 is called the characteristic equation of A. We note that thisequation can be obtained just as well by evaluating |A I | = 0; however, the form(1.6) is preferred for the definition of the characteristic equation, since the coefficientof n is then always +1.

In many areas of engineering, particularly in those involving vibration or the controlof processes, the determination of those values of for which (1.5) has a non-trivialsolution (that is, a solution for which x 0) is of vital importance. These values of are precisely the values that satisfy the characteristic equation, and are called theeigenvalues of A.

Find the characteristic equation for the matrix

Solution By (1.6), the characteristic equation for A is the cubic equation

Expanding the determinant along the first column gives

= ( 1)[( 2)( + 1) 1] [2 ( + 1)]

c() = |I A | = 0 (1.6)

Example 1.2

A = 1 1 2

1 2 10 1 1

c ( ) = 1 1 2

1 2 10 1 1+

= 0

c ( ) 1( ) 2 1

1 + 11 21 + 1

=

16 MATRIX ANALYSIS

Thus

c() = 3 22 + 2 = 0

is the required characteristic equation.

For matrices of large order, determining the characteristic polynomial by directexpansion of |I A | is unsatisfactory in view of the large number of terms involvedin the determinant expansion. Alternative procedures are available to reduce the amountof calculation, and that due to Faddeev may be stated as follows.

The method of Faddeev

If the characteristic polynomial of an n n matrix A is written as

n p1n1 . . . pn1 pn

then the coefficients p1, p2, . . . , pn can be computed using

(r = 1, 2, . . . , n)

where

and

Br = Ar prI, where I is the n n identity matrix

The calculations may be checked using the result that

Bn = An pnI must be the zero matrix

Using the method of Faddeev, obtain the characteristic equation of the matrix A ofExample 1.2.

Solution

Let the characteristic equation be

c() = 3 p12 p2 p3

pr1r--- trace Ar=

ArA (r 1)=AB r1 r 2 3 6 n, , ,=( )

=

Example 1.3

A

1 1 21 2 1

0 1 1=


Then, following the procedure described above,

p1 = trace A = (1 + 2 1) = 2

Then, the characteristic polynomial of A is

c() = 3 22 + 2

in agreement with the result of Example 1.2. In this case, however, a check may becarried out on the computation, since

B3 = A3 + 2I = 0

as required.

1.4.2 Eigenvalues and eigenvectors

The roots of the characteristic equation (1.6) are called the eigenvalues of the matrix A(the terms latent roots, proper roots and characteristic roots are also sometimes used).By the Fundamental Theorem of Algebra, a polynomial equation of degree n hasexactly n roots, so that the matrix A has exactly n eigenvalues i, i = 1, 2, . . . , n. Theseeigenvalues may be real or complex, and not necessarily distinct. Corresponding to eacheigenvalue i, there is a non-zero solution x = ei of (1.5); ei is called the eigenvector ofA corresponding to the eigenvalue i. (Again the terms latent vector, proper vector andcharacteristic vector are sometimes seen, but are generally obsolete.) We note that ifx = ei satisfies (1.5) then any scalar multiple iei of ei also satisfies (1.5), so that theeigenvector ei may only be determined to within a scalar multiple.

B1 = A 2I = 1 1 21 0 1

0 1 3

A2 AB1

2 1 51 0 11 1 4

= =

p212--- trace A2 12--- 2 0 4+ +( ) 1= = =

B2 A2 I

3 1 51 1 11 1 3

= =

A3 AB2

2 0 00 2 00 0 2

= =

p3 = 13--- trace A3 = 13--- 2 2 2( ) 2=

18 MATRIX ANALYSIS

Determine the eigenvalues and eigenvectors for the matrix A of Example 1.2.

Solution

The eigenvalues i of A satisfy the characteristic equation c() = 0, and this has beenobtained in Examples 1.2 and 1.3 as the cubic

3 22 + 2 = 0

which can be solved to obtain the eigenvalues 1, 2 and 3.Alternatively, it may be possible, using the determinant form |I A |, or indeed (as

we often do when seeking the eigenvalues) the form |A I |, by carrying out suitablerow and/or column operations to factorize the determinant.

In this case

and adding column 1 to column 3 gives

Subtracting row 3 from row 1 gives

Setting |A I | = 0 gives the eigenvalues as 1 = 2, 2 = 1 and 3 = 1. The order inwhich they are written is arbitrary, but for consistency we shall adopt the convention oftaking 1, 2, . . . , n in decreasing order.

Having obtained the eigenvalues i (i = 1, 2, 3), the corresponding eigenvectors eiare obtained by solving the appropriate homogeneous equations

(A iI )ei = 0 (1.7)

When i = 1, i = 1 = 2 and (1.7) is

Example 1.4

A =1 1 2

1 2 10 1 1

A I = 1 1 2

1 2 10 1 1

1 1 1 1 2 0

0 1 1 1 +( )

1 1 11 2 0

0 1 1

=

1 +( )1 0 0

1 2 00 1 1

1 +( ) 1 ( ) 2 ( )=

1 1 21 0 1

0 1 3

e11

e12

e13

= 0


that is,

e11 + e12 2e13 = 0

e11 + 0e12 + e13 = 0

0e11 + e12 3e13 = 0

leading to the solution

where 1 is an arbitrary non-zero scalar. Thus the eigenvector e1 corresponding to theeigenvalue 1 = 2 is

e1 = 1[1 3 1]T

As a check, we can compute

and thus conclude that our calculation was correct.When i = 2, i = 2 = 1 and we have to solve

that is,

0e21 + e22 2e23 = 0

e21 + e22 + e23 = 0

0e21 + e22 2e23 = 0

leading to the solution

where 2 is an arbitrary scalar. Thus the eigenvector e2 corresponding to the eigenvalue 2 = 1 is

e2 = 2 [3 2 1]T

Again a check could be made by computing Ae2.Finally, when i = 3, i = 3 = 1 and we obtain from (1.7)

e111------

e123

----------e131------ 1= = =

Ae1 11 1 2

1 2 10 1 1

1

3

1

12

6

2

211

3

1

1e1= = = =

0 1 21 1 1

0 1 2

e21

e22

e23

= 0

e213------ = e22

2---------- = e23

1------ = 2

2 1 21 3 1

0 1 0

e31

e32

e33

= 0

20 MATRIX ANALYSIS

that is,

2e31 + e32 2e33 = 0

e31 + 3e32 + e33 = 0

0e31 + e32 + 0e33 = 0

and hence

Here again 3 is an arbitrary scalar, and the eigenvector e3 corresponding to the eigen-value 3 is

e3 = 3 [1 0 1]T

The calculation can be checked as before. Thus we have found that the eigenvalues ofthe matrix A are 2, 1 and 1, with corresponding eigenvectors

1 [1 3 1]T, 2 [3 2 1]T and 3 [1 0 1]T

respectively.

Since in Example 1.4 the i, i = 1, 2, 3, are arbitrary, it follows that there are aninfinite number of eigenvectors, scalar multiples of each other, corresponding to eacheigenvalue. Sometimes it is convenient to scale the eigenvectors according to someconvention. A convention frequently adopted is to normalize the eigenvectors so thatthey are uniquely determined up to a scale factor of 1. The normalized form of aneigenvector e = [e1 e2 . . . en]T is denoted by and is given by

where

For example, for the matrix A of Example 1.4, the normalized forms of the eigenvectorsare

1 = [1/11 3/11 1/11]T, 2 = [3/14 2/14 1/14]T

and

3 = [1/2 0 1/2]T

However, throughout the text, unless otherwise stated, the eigenvectors will alwaysbe presented in their simplest form, so that for the matrix of Example 1.4 we take1 = 2 = 3 = 1 and write

e1 = [1 3 1]T, e2 = [3 2 1]T and e3 = [1 0 1]T

e311------ = e32

0------ = e33

1------ = 3

e| e |-------=

| e | e12 e2

2 6 en2+ + +( )=


For a n n matrix A the MATLAB command p=poly(A) generates an n + 1 ele-ment row vector whose elements are the coefficients of the characteristic polynomialof A, the coefficients being ordered in descending powers. The eigenvalues of Aare the roots of the polynomial and are generated using the command roots(p).The command

[M,S]=eig(A)

generates the normalized eigenvectors of A as the columns of the matrix M and itscorresponding eigenvalues as the diagonal elements of the diagonal matrix S(M and S are called respectively the modal and spectral matrices of A and we shallreturn to discuss them in more detail in Section 1.6.1). In the absence of the left-hand arguments, the command eig(A) by itself simply generates the eigenvaluesof A.

For the matrix A of Example 1.4 the commands

A=[1 1 -2; -1 2 1; 0 1 1];

[M,S]=eig(A)

generate the output

0.3015 -0.8018 0.7071

M=0.9045 -0.5345 0.0000

0.3015 -0.2673 0.7071

2.0000 0 0

S=0 1.0000 0

0 0 -1.0000

These concur with our calculated answers, with 1 = 0.3015, 2 = 0.2673 and3 = 0.7071.

Using the Symbolic Math Toolbox in MATLAB we saw earlier that the matrix Amay be converted from numeric into symbolic form using the command A=sym(A).Then its symbolic eigenvalues and eigenvectors are generated using the sequence ofcommands

A=[1 1 2; -1 2 1; 0 1 1];

A=sym(A);

[M,S]=eig(A)

as

M=[3, 1, 1]

[2, 3, 0]

[1, 1, 1]

S=[1, 0, 0]

[0, 2, 0]

[0, 0, -1]

In MAPLE the command Eigenvalues(A); returns a vector of eigenvalues. Thecommand Eigenvectors(A) returns both a vector of eigenvalues as before anda matrix containing the eigenvalues, so that the ith column is an eigenvectorcorresponding to the eigenvalue in the ith entry of the preceding vector. Thus thecommands:

22 MATRIX ANALYSIS

Find the eigenvalues and eigenvectors of

Solution Now

= 2 2 cos + cos2 + sin2 = 2 2 cos + 1

So the eigenvalues are the roots of

2 2 cos + 1 = 0

that is,

= cos jsin

Solving for the eigenvectors as in Example 1.4, we obtain

e1 = [1 j]T and e2 = [1 j]T

In Example 1.5 we see that eigenvalues can be complex numbers, and that the eigen-vectors may have complex components. This situation arises when the characteristicequation has complex (conjugate) roots.

with(LinearAlgebra),

A:=Matrix([[1,1,-2],[-1,2,1];[0,1,-1]]);

Eigenvalues(A);

return

and the command

Eigenvectors(A);

returns

1

2

1

2

1

1

1 1 33 0 2

1 1 1

Example 1.5

A = cos sin sin cos

I A = cos sin sin cos


1.4.4 Repeated eigenvalues

In the examples considered so far the eigenvalues i (i = 1, 2, . . . ) of the matrix A havebeen distinct, and in such cases the corresponding eigenvectors can be found and arelinearly independent. The matrix A is then said to have a full set of linearly independenteigenvectors. It is clear that the roots of the characteristic polynomial c() may not allbe distinct; and when c() has p n distinct roots, c() may be factorized as

indicating that the root = i, i = 1, 2, . . . , p, is a root of order mi, where the integer miis called the algebraic multiplicity of the eigenvalue i. Clearly m1 + m2 + . . . + mp = n.When a matrix A has repeated eigenvalues, the question arises as to whether it ispossible to obtain a full set of linearly independent eigenvectors for A. We first considertwo examples to illustrate the situation.

Determine the eigenvalues and corresponding eigenvectors of the matrix

Solution We find the eigenvalues from

as 1 = 4, 2 = 3 = 2.

c ( ) 1( )m1 2( )

m2 6 p( )mp=

Example 1.6

A = 3 3 2

1 5 21 3 0

3 3 21 5 21 3

= 0

Check your answers using MATLAB or MAPLE whenever possible.

Using the method of Faddeev, obtain the characteristic polynomials of the matrices

(a) (b)

Find the eigenvalues and corresponding eigenvectors of the matrices

(a) (b)

(c) (d)

(e) (f)

(g) (h)

1.4.3 Exercises

5

3 2 1

4 5 12 3 4

2 1 1 2 0 1 1 0

1 1 1 1 1 1 1 0

6

1 1

1 1

1 2

3 2

1 0 40 5 4

4 4 3

1 1 2

0 2 2

1 1 3

5 0 6

0 11 6

6 6 2

1 1 01 2 1

2 1 1

4 1 1

2 5 4

1 1 0

1 4 20 3 1

1 2 4

24 MATRIX ANALYSIS

The eigenvectors are obtained from

(A I )ei = 0 (1.8)

and when = 1 = 4, we obtain from (1.8)

e1 = [1 1 1]T

When = 2 = 3 = 2, (1.8) becomes

so that the corresponding eigenvector is obtained from the single equation

e21 3e22 + 2e23 = 0 (1.9)

Clearly we are free to choose any two of the components e21, e22 or e23 at will, with theremaining one determined by (1.9). Suppose we set e22 = and e23 = ; then (1.9) meansthat e21 = 3 2, and thus

e2 = [3 2 ]T = (1.10)

Now = 2 is an eigenvalue of multiplicity 2, and we seek, if possible, two linearlyindependent eigenvectors defined by (1.10). Setting = 1 and = 0 yields

e2 = [3 1 0]T

and setting = 0 and = 1 gives a second vector

e3 = [2 0 1]T

These two vectors are linearly independent and of the form defined by (1.10), and it isclear that many other choices are possible. However, any other choices of the form (1.10)will be linear combinations of e2 and e3 as chosen above. For example, e = [1 1 1]satisfies (1.10), but e = e2 + e3.

In this example, although there was a repeated eigenvalue of algebraic multiplicity 2,it was possible to construct two linearly independent eigenvectors corresponding to thiseigenvalue. Thus the matrix A has three and only three linearly independent eigenvectors.

The MATLAB commands

A=[3 3 2; -1 5 2; -1 3 0];

[M,S]=eig(A)

generate

0.5774 -0.5774 -0.7513

M=-0.5774 -0.5774 0.1735

-0.5774 -0.5774 0.6361

4.0000 0 0

S= 0 2.0000 0

0 0 2.0000

1 3 21 3 21 3 2

e21

e22

e23

= 0

3

1

0

2

0

1

+


Determine the eigenvalues and corresponding eigenvectors for the matrix

Solution Solving |A I | = 0 gives the eigenvalues as 1 = 2 = 2, 3 = 1. The eigenvectorcorresponding to the non-repeated or simple eigenvalue 3 = 1 is easily found as

e3 = [1 1 1]T

When = 1 = 2 = 2, the corresponding eigenvector is given by(A 2I )e1 = 0

that is, as the solution of

e11 + 2e12 + 2e13 = 0 (i)e13 = 0 (ii)

e11 + 2e12 = 0 (iii)

Clearly the first column of M (corresponding to the eigenvalue 1 = 4) is a scalarmultiple of e1. The second and third columns of M (corresponding to the repeatedeigenvalue 2 = 3 = 2) are not scalar multiples of e2 and e3. However, both satisfy(1.10) and are equally acceptable as a pair of linearly independent eigenvectorscorresponding to the repeated eigenvalue. It is left as an exercise to show that bothare linear combinations of e2 and e3.

Check that in symbolic form the commands

A=sym(A);

[M,S]=eig(A)

generate

M=[-1, 3, 2]

[1, 1, 0]

[1, 0, 1]

S=[4, 0, 0]

[0, 2, 0]

[0, 0, 2]

In MAPLE the command Eigenvectors(A); produces corresponding results.Thus the commands


A:=Matrix([[3,-3,2],[-1,5,-2],[-1,3,0]]);

Eigenvectors(A);

return

2

2

4

-2 3 -1

0 1 1

1 0 1

Example 1.7

A = 1 2 2

0 2 1

1 2 2

26 MATRIX ANALYSIS

From (ii) we have e13 = 0, and from (i) and (ii) it follows that e11 = 2e12. We deducethat there is only one linearly independent eigenvector corresponding to the repeatedeigenvalue = 2, namely

e1 = [2 1 0]T

and in this case the matrix A does not possess a full set of linearly independenteigenvectors.

We see from Examples 1.6 and 1.7 that if an n n matrix A has repeated eigen-values then a full set of n linearly independent eigenvectors may or may not exist.The number of linearly independent eigenvectors associated with a repeated eigen-value i of algebraic multiplicity mi is given by the nullity qi of the matrix A iI,where

qi is sometimes referred to as the degeneracy of the matrix A iI or the geometricmultiplicity of the eigenvalue i, since it determines the dimension of the spacespanned by the corresponding eigenvector(s) ei.

Confirm the findings of Examples 1.6 and 1.7 concerning the number of linearlyindependent eigenvectors found.

Solution In Example 1.6, we had an eigenvalue 2 = 2 of algebraic multiplicity 2. Correspondingly,

and performing the row operation of adding row 1 to rows 2 and 3 yields

Adding 3 times column 1 to column 2 followed by subtracting 2 times column 1 fromcolumn 3 gives finally

indicating a rank of 1. Then from (1.11) the nullity q2 = 3 1 = 2, confirming thatcorresponding to the eigenvalue = 2 there are two linearly independent eigenvectors,as found in Example 1.6.

qi = n rank (A iI ), with 1 qi mi (1.11)

Example 1.8

A 2I = 3 2 3 2

1 5 2 21 3 2

= 1 3 2

1 3 21 3 2

1 3 20 0 0

0 0 0

1 0 0

0 0 0

0 0 0


In Example 1.7 we again had a repeated eigenvalue 1 = 2 of algebraic multiplicity 2.Then

Performing row and column operations as before produces the matrix

this time indicating a rank of 2. From (1.11) the nullity q1 = 3 2 = 1, confirming thatthere is one and only one linearly independent eigenvector associated with this eigen-value, as found in Example 1.7.

A 2I = 1 2 2 2

0 2 2 11 2 2 2

= 1 2 2

0 0 1

1 2 0

1 0 00 0 1

0 0 0


Obtain the eigenvalues and corresponding eigenvectors of the matrices

(a) (b)

(c) (d)

Given that = 1 is a three-times repeated eigenvalue of the matrix

using the concept of rank, determine how many linearly independent eigenvectors correspond to this value of . Determine a corresponding set of linearly independent eigenvectors.

Given that = 1 is a twice-repeated eigenvalue of the matrix

how many linearly independent eigenvectors correspond to this value of ? Determine a corresponding set of linearly independent eigenvectors.

1.4.5 Exercises

7

2 2 1

1 3 1

1 2 2

0 2 21 1 21 1 2

4 6 6

1 3 2

1 5 2

7 2 43 0 26 2 3

8

A

3 7 52 4 3

1 2 2

=

9

A = 2 1 1

1 0 11 1 2

1.4.6 Some useful properties of eigenvalues

The following basic properties of the eigenvalues 1, 2, . . . , n of an n n matrix Aare sometimes useful. The results are readily proved either from the definition of eigen-values as the values of satisfying (1.4), or by comparison of corresponding charac-teristic polynomials (1.6). Consequently, the proofs are left to Exercise 10.

28 MATRIX ANALYSIS

Property 1.1

Property 1.2

Property 1.3

Property 1.4

Property 1.5

Property 1.6

The sum of the eigenvalues of A is

ii=1

n

= trace A = aiii=1

n

The product of the eigenvalues of A is

where detA denotes the determinant of the matrix A.

i det A=i=1

n

The eigenvalues of the inverse matrix A1, provided it exists, are

, , . . . ,11----- 1

2----- 1

n-----

The eigenvalues of the transposed matrix AT are

1, 2, . . . , nas for the matrix A.

If k is a scalar then the eigenvalues of kA are

k1, k 2, . . . , k n

If k is a scalar and I the n n identity (unit) matrix then the eigenvalues of A kIare respectively

1 k, 2 k, . . . , n k


Property 1.7

1.4.7 Symmetric matrices

A square matrix A is said to be symmetric if AT = A. Such matrices form an importantclass and arise in a variety of practical situations. Two important results concerning theeigenvalues and eigenvectors of such matrices are

If the orthogonal eigenvectors of a symmetric matrix are normalized as

1, 2, . . . , n

then the inner (scalar) product is

Ti j = ij (i, j = 1, 2, . . . , n)

where ij is the Kronecker delta defined in Section 1.3.2.The set of normalized eigenvectors of a symmetric matrix therefore forms an ortho-

normal set (that is, it forms a mutually orthogonal normalized set of vectors).

Obtain the eigenvalues and corresponding orthogonal eigenvectors of the symmetricmatrix

and show that the normalized eigenvectors form an orthonormal set.

Solution The eigenvalues of A are 1 = 6, 2 = 3 and 3 = 1, with corresponding eigenvectors

e1 = [1 2 0]T, e2 = [0 0 1]T, e3 = [2 1 0]T

which in normalized form are

1 = [1 2 0]T/5, 2 = [0 0 1]T, 3 = [2 1 0]T/5

Evaluating the inner products, we see that, for example,

,

If k is a positive integer then the eigenvalues of Ak are

, , . . . ,1k 2

k nk

(a) the eigenvalues of a real symmetric matrix are real;

(b) for an n n real symmetric matrix it is always possible to find n linearlyindependent eigenvectors e1, e2, . . . , en that are mutually orthogonal sothat eTiej = 0 for i j.

Example 1.9

A = 2 2 0

2 5 0

0 0 3

1T1 = 15---

45--- 0+ + = 1 1

T3 = 25---25--- 0+ + = 0

30 MATRIX ANALYSIS

and that

Tij = ij (i, j = 1, 2, 3)

confirming that the eigenvectors form an orthonormal set.


Verify Properties 1.11.7 of Section 1.4.6.

Given that the eigenvalues of the matrix

are 5, 3 and 1:

(a) confirm Properties 1.11.4 of Section 1.4.6;

(b) taking k = 2, confirm Properties 1.51.7 of Section 1.4.6.

Determine the eigenvalues and corresponding eigenvectors of the symmetric matrix

and verify that the eigenvectors are mutually orthogonal.

The 3 3 symmetric matrix A has eigenvalues 6, 3 and 2. The eigenvectors corresponding to the eigenvalues 6 and 3 are [1 1 2]T and [1 1 1]T respectively. Find an eigenvector corresponding to the eigenvalue 2.

1.4.8 Exercises

10

11

A = 4 1 1

2 5 4

1 1 0

12

A = 3 3 33 1 13 1 1

13

Numerical methodsIn practice we may well be dealing with matrices whose elements are decimal numbersor with matrices of high orders. In order to determine the eigenvalues and eigenvectorsof such matrices, it is necessary that we have numerical algorithms at our disposal.

1.5.1 The power method

Consider a matrix A having n distinct eigenvalues 1, 2, . . . , n and correspondingn linearly independent eigenvectors e1, e2, . . . , en. Taking this set of vectors as thebasis, we can write any vector x = [x1 x2 . . . xn]T as a linear combination in theform

Then, since Aei = iei for i = 1, 2, . . . , n,

1.5

x 1e1 2e2 6 nen = ieii=1

n

+ + +=

Ax A ieii=1

n

iieii=1

n

= =

1.5 NUMERICAL METHODS 31

and, for any positive integer k,

or

(1.12)

Assuming that the eigenvalues are ordered such that

|1 | | 2 | . . . | n |

and that 1 0, we have from (1.12)

(1.13)

since all the other terms inside the square brackets tend to zero. The eigenvalue 1 andits corresponding eigenvector e1 are referred to as the dominant eigenvalue and eigen-vector respectively. The other eigenvalues and eigenvectors are called subdominant.

Thus if we introduce the iterative process

x(k+1) = Ax(k) (k = 0, 1, 2, . . . )

starting with some arbitrary vector x(0) not orthogonal to e1, it follows from (1.13)that

x(k) = Akx(0)

will converge to the dominant eigenvector of A.A clear disadvantage with this scheme is that if |1 | is large then Akx(0) will become

very large, and computer overflow can occur. This can be avoided by scaling the vectorx(k) after each iteration. The standard approach is to make the largest element of x(k)

unity using the scaling factor max(x(k)), which represents the element of x(k) having thelargest modulus.

Thus in practice we adopt the iterative process

Corresponding to (1.12), we have

where

R = [max(y(1))max(y(2)) . . . max(y(k))]1

Akx iikei

i=1

n

=

Akx 1k 1e1 i

i1-----

keii=2

n

+=

limk

Akx 1k1e1=

y(k+1) = Ax(k)

(k = 0, 1, 2, . . . ) (1.14)

and it is common to take x(0) = [1 1 . . . 1]T.

x k+1( ) yk+1( )

max y k+1( )( )----------------------------=

x k( ) = R1k 1e1 i

i1-----

keii=2

n

+

32 MATRIX ANALYSIS

Again we see that x(k) converges to a multiple of the dominant eigenvector e1. Also,since Ax(k) 1x(k), we have y(k+1) 1x(k), and since the largest element of x(k) is unity,it follows that the scaling factors max( y(k+1)) converge to the dominant eigenvalue 1.The rate of convergence depends primarily on the ratios

, , . . . ,

The smaller these ratios, the faster the rate of convergence. The iterative process repres-ents the simplest form of the power method, and a pseudocode for the basic algorithmis given in Figure 1.1.

Use the power method to find the dominant eigenvalue and the corresponding eigen-vector of the matrix

Solution Taking x(0) = [1 1 1]T in (1.14), we have

;

;

21-----

31-----

n1-----

{read in xT = [x1 x2 . . . xn]}m 0repeat

mold m{evaluate y = Ax}{find m = max(yi)}{xT = [y1/m y2/m . . . yn/m]}

until abs(m mold) tolerance{write (results)}

Figure 1.1 Outline pseudocode program for power method to calculate the maximum eigenvalue.

Example 1.10

A = 1 1 2

1 2 10 1 1

y 1( ) = Ax 0( ) = 1 1 2

1 2 10 1 1

1

1

1

0

2

0

= 20

1

0

= 11( ) = 2

x 1( ) = 12--- y1( ) =

0

1

0

y 2( ) = Ax 1( ) =1 1 2

1 2 10 1 1

0

1

0

1

2

1

= 20.5

1

0.5

= 22( ) = 2

x 2( ) = 12--- y2( ) =

12---

112---

1.5 NUMERICAL METHODS 33

;

Continuing with the process, we have

y(4) = 2[0.375 1 0.375]T

y(5) = 2[0.312 1 0.312]T

y(6) = 2[0.344 1 0.344]T

y(7) = 2[0.328 1 0.328]T

y(8) = 2[0.336 1 0.336]T

Clearly y(k) is approaching the vector , so that the dominant eigenvalue is2 and the corresponding eigenvector is , which conforms to the answerobtained in Example 1.4.

Find the dominant eigenvalue of

Solution Starting with x(0) = [1 1 1 1]T, the iterations give the following:

This indicates that the dominant eigenvalue is aproximately 3.16, with correspondingeigenvector [0.46 0.46 1 0.24]T.

y 3( ) = Ax 2( ) =

1 1 2

1 2 1

0 1 1

12---

112---

12---

212---

= 2

0.25

1

0.25

= 32( ) = 2

x 3( ) =0.25

1

0.25

2 13--- 1 1

3---[ ]T

13--- 1 1

3---[ ]T

Example 1.11

A =

1 0 1 00 1 1 0

1 1 2 10 0 1 1

Iteration k 1 2 3 4 5 6 7

Eigenvalue 3 2.6667 3.3750 3.0741 3.2048 3.1636 3.1642

1 0 0.3750 0.4074 0.4578 0.4549 0.4621 0.46211 0.6667 0.6250 0.4815 0.4819 0.4624 0.4621 0.4621

1 1 1 1 1 1 1 1

1 0 0.3750 0.1852 0.2651 0.2293 0.2403 0.2401

x1k( )

x2k( )

x3k( )

x4k( )

34 MATRIX ANALYSIS

The power method is suitable for obtaining the dominant eigenvalue and cor-responding eigenvector of a matrix A having real distinct eigenvalues. The smallesteigenvalue, provided it is non-zero, can be obtained by using the same method on theinverse matrix A1 when it exists. This follows since if Ax = x then A1x = 1x. Tofind the subdominant eigenvalue using this method the dominant eigenvalue must firstbe removed from the matrix using deflation methods. We shall illustrate such a methodfor symmetric matrices only.

Let A be a symmetric matrix having real eigenvalues 1, 2, . . . , n. Then, by result(b) of Section 1.4.7, it has n corresponding mutually orthogonal normalized eigen-vectors 1, 2, . . . , n such that

Tij = ij (i, j = 1, 2, . . . , n)

Let 1 be the

advanced modern engineering mathematics -...

Documents