An Interview with Gene Golub

Higham, Nicholas J.

2008

MIMS EPrint: 2008.8

Manchester Institute for Mathematical SciencesSchool of Mathematics

The University of Manchester

Reports available from: http://eprints.maths.manchester.ac.uk/And by contacting: The MIMS Secretary

School of Mathematics

The University of Manchester

Manchester, M13 9PL, UK

ISSN 1749-9097

An Interview with Gene Golub1

by Nicholas J. Higham2

On July 3, 2005 I interviewed Gene Golub (1932–2007) during a visit he made toThe University of Manchester to attend a workshop. This document provides an editedtranscript of the interview.

The bibliography contains some books and papers mentioned directly or indirectlyin the interview. For more on Golub and his work, I highly recommend Milestones in

Matrix Computation: The Selected Works of Gene H. Golub, with Commentaries [3].I am grateful to Louise Stait for transcribing the interview, and to Gail Corbett and

Sven Hammarling for help with the editing.

Gene Golub, July 2005. Photograph by N. J. Higham.

Gene Golub by John de Pillis (http://math.ucr.edu/˜jdp), 2007.

1Document dated February 5, 2008.2School of Mathematics, The University of Manchester, Manchester, M13 9PL, UK

(higham@ma.man.ac.uk, http://www.ma.man.ac.uk/˜higham).

1

NJH: How did you get interested inmathematics?

GHG: Originally, when I went to college,I wanted to become a chemist and thoughtof other things too, even becoming a polit-ical scientist, but I found that the subjectthat I liked to do the most and was best atwas mathematics. I enjoyed the manipula-tions a lot and I still do.

NJH: Did you have any particular men-tors at any stage of your career, up to, say,your Ph.D.?

GHG: Well, not very strong, but somepeople were quite influential. As a senior atthe University of Illinois I took a statisticscourse from a man named K. A. Bush. Hecalled me aside and said, “Would you liketo major in statistics? I think you woulddo very well there.” So he certainly playeda strong role there. Perhaps I should ex-plain my whole education. The first twoyears I went to a junior college in Chicago.It was called Wright Junior College and al-though none of the teachers were engaged inresearch, as far as I knew, they were excel-lent teachers: they were prepared and theywere careful. And then from there I decidedto go to the University of Chicago, which issuch a tremendous change, and one of myteachers warned me that it probably wastoo dramatic a change. Nevertheless, I wentto Chicago. The University of Chicago wasa great distance from where I lived; it tookan hour and a half to get there from whereI was living at the time. So after a year’stime I decided I really didn’t like it and thatI’d like to go away to university. But at thetime I was growing up, going away to uni-versity was really done by people who hadsome money behind them. I didn’t haveany; my mother was a widow. So I hadworked and I thought this would be my bigsplurge, going away to the University of Illi-nois for my final year. I came down to theUniversity of Illinois in 1952, and I took sev-eral courses because there was a necessity tocomplete certain qualifications. I also took

a programming course in January 1953 fromJ. P. Nash, who talked about programmingfor the Illiac. The Illiac had just becomeavailable and Nash was very influential inmy life; he taught me how to program. Atthat time programming and elementary nu-merical analysis were included together. Atthe end of the term I was graduating fromthe University of Illinois. I had no ideawhat I was going to do. I had applied forvarious positions, but Nash called me asideand he said, “Would you like to be an assis-tant in the computing laboratory?” I knewI liked going to school, and although I ap-plied elsewhere for real jobs I thought it wasfun to go to school. I liked computing a loteven then; I wasn’t particularly good at itbut I liked it. That summer I was alreadyan assistant and I guess I was making $95a month, so that was good money for methen. I had a task to do. I was supposed toprogram Milne’s method for solving differ-ential equations. Well, I didn’t really thinkit out very well. I think it was too big ofa project for me, and of course it was latershown that Milne’s method is a weakly sta-ble method. But I worked on it and it keptme busy during the summer. I became afull-time student during the autumn of thatyear, 1953. But my main interest was instatistics. I was really going to do comput-ing and statistics. I took a statistics coursefrom a man called C. R. Rao. C. R. Raois a great statistician and as it turns outhe was visiting Illinois just that year. Thecourse was probably beyond me, but he didlots of matrix manipulations: for instance,he didn’t call it this, but he developed blockGaussian elimination and so forth. So thatreally expanded my vision. Also at Illinoisthere was a large group of people who didpsychometrics and there was really a wellestablished programme. And of course theyhad the Illiac, so it was one of the first psy-chometrics groups to have a computer avail-able for it. In particular, there was a manthere named Charles Wrigley, who was an

2

assistant professor. Wrigley was from NewZealand originally and he had gotten hisPh.D. in England under Sir Cyril Burt, theman who had so much to do with the 11+examination and may have faked the datain fact. And Wrigley was a lovely man. Idon’t know how to describe him, but he wasa very kindly gentleman and he said, “OhGene, you are doing such interesting workand these matrix computations are so im-portant.” The thing we were able to do wascompute the eigenvalues and eigenvectors ofa 23 by 23 matrix in 15 minutes. Well, thatfor then was a big breakthrough becausethere were some classical cases in psycho-metrics that had never been done so exten-sively. I learnt a lot about factor analysisand in a way it’s just one step away fromthe singular value decomposition. Rao de-vised a method called maximum likelihoodfactor analysis, or canonical factor analysis,which was a new form of factor analysis.The big question in factor analysis at thattime was, “How many factors should therebe?”, and it’s equivalent to saying, “Howmany nonzero singular values are there?”Rao had devised a statistical test for do-ing this. Much of this work was actuallycontained in the work by Lawley on fac-tor analysis. So I was just very fortunate:J. P. Nash brought me to the computinglaboratory, through that laboratory I metCharles Wrigley, I met C. R. Rao, and I metmany other people. My first lecture was atthe International Congress of Psychologists.There I met Louis Guttman, who is con-sidered a great psychometrician, and HansEysenck from Britain, who was a very fa-mous man. I never met Sir Cyril Burt. Imet a lot of people through this connectionand, as you can tell, it influenced the sortof things I liked to do.

NJH: Who was your Ph.D. advisor?

GHG: I was going to write a thesis instatistics, but the man who I was going towork under left abruptly. Then one of theleading professors at the computing labora-

tory, a man named A. H. Taub, agreed tobe my advisor. Taub was quite a significantperson at that time; he had worked withJohn von Neumann, and he had close con-nections with the whole Los Alamos com-munity. He came to Illinois from the Uni-versity of Washington. Taub was also theadvisor of Bill Gear, Bob Gregory and otherpeople too. So he had some very good stu-dents. He was not an easy man to workwith. Taub took me on and he had a pa-per or a draft of a paper that von Neumannhad written on using Chebyshev polynomi-als, and my thesis grew from there. WhileI was working on my thesis Taub invitedRichard Varga to come by, just as I was fin-ishing. Taub wasn’t so interested in solvinglinear equations, but Varga came by andgave a talk and afterwards I told him what Ihad done and he said, “Oh, I’ve done some-thing similar, let’s write a paper together.”Then my advisor said to me, “I understandMr. Varga has done something similar. Ifhe publishes before you, you don’t get a de-gree because the thesis is supposed to beoriginal.” So I was a bit worried and Ididn’t really know how people behaved inthe academic world, but I was pretty con-fident. Then the next year while I was inEngland on my NSF fellowship, Varga and Iworked on our paper together. It was won-derful working with him, but as you mayknow when you are working on your thesisyou really know the subject area that youare working on, so I knew all the details andvarious little tricks and so forth. So the pa-per was really a joint collaboration. I learnta lot in working on it with him but I thinkI also helped substantially in the writing ofthe paper.

NJH: So that time in Cambridge wasduring the Ph.D.?

GHG: No, just after my Ph.D. I had anNSF fellowship and I was delayed to someextent; I didn’t finish as I had hoped ontime, so it took me a year before I went.Illinois was a wonderful place. The whole

3

computing laboratory had a great sense ofenthusiasm and there were a lot of youngPh.D.s who were beginning their careers.One was D. E. Muller of Muller’s method;he’s the son of a Nobel laureate. He’s a won-derful person; he’s still alive fortunately,and he developed Muller’s Method and didother things. Bill Gear was a graduate stu-dent. He came a couple of years after I did;he was pretty spectacular as a student. Thewhole atmosphere was really very warm andwe were often invited to faculty’s homes,so I in my small way try to keep this Illi-nois tradition of inviting people to my homeand trying to be friendly. I was treated tothat and I loved it, and it meant a lot tome. I should just say one other word aboutmy own family background. Both my par-ents were poor immigrants to the UnitedStates. They came in 1923 and they had re-ally inferior jobs: my father delivered breadand my mother sewed in a shop. They hadno idea what the academic world was like,but they knew it was good to go to col-lege and of course that was impressed onus. So I continued always as a student, andof course after a while, because of educa-tion, you are separated in a way from yourparents too. My father died when I was 16,but my mother was supportive throughoutmy career. She was very happy to see mecarry on and continue.

NJH: Where had they emigrated from?

GHG: My father came from Ukraine andjust this last week I went to the Ukraine tosee the town where he lived. It was sortof like family folklore and my mother camefrom Latvia; they met in Chicago. I’ve beenable to find on the Ellis Island website ex-actly when they left and when they arrivedand where they went to in Chicago. So thatwas the life experience for me, and the otherpeople I went to school with, they had simi-lar experiences. 1923 was an important yearin immigration in the United States. Theracism that we saw in Europe eventuallyended up in the holocaust. It was true ev-

erywhere, and in the United States organi-zations like the Klu Klux Klan pressured forimmigration controls and one of the thingsthat came about was that there were quo-tas for different nations. The quotas werereally established based on the nineteenth-century United States, so if you were En-glish or Swedish or French you came to theUnited States with relative ease. Peoplefrom eastern Europe after 1923 were prettywell frozen out so it was a bad time through-out the world, and racism prevailed every-where I think.

NJH: Going back to your year at Cam-bridge who were some of the people thatyou met there?

GHG: Well, I had met Velvel Kahan be-fore and we knew each other from a coupleof visits at Illinois. Illinois and Toronto hadwanted to build a second computer togetherand Velvel was their man, so to speak, so Iknew him and we shared an office with twoother people: one was Colin Cryer, who wasa research student, and another person wasNick Capon from Australia (I think he isretired now). So the four of us were in thistiny office. Velvel and I were the two post-docs there, and I knew David Wheeler be-cause Wheeler had visited Illinois. He wassort of my teacher in numerical analysis. I’dnever taken a numerical analysis course, buthe designed a library for the University ofIllinois Illiac, and I would read his codes andI learnt a lot by reading those codes. Forinstance, Gaussian elimination with partialpivoting: that’s what he had implementedand that’s how I learned how to do Gaus-sian elimination. And you know it’s veryinteresting; as I’ve stated before, what goesaround comes around. Gaussian elimina-tion with partial pivoting was very popularbecause the storage requirement was rela-tively small and then it sort of fell by thewayside when people were doing direct LUfactorization. Then of course it was broughtback into play again after a few years dur-ing the initial interest in vector and parallel

4

computing. So I learnt a lot. Another per-son who was at Illinois was Stanley Gill;if you recall, one of the earliest books onprogramming was by Wilkes, Wheeler, andGill. Gill had devised a version of Runge-Kutta that reduced the amount of storage,and so he spent a year at Illinois and Ihad several conversations with him. Hewas a very, very nice man. He was fromEngland—he became professor of comput-ing science at Imperial College—and then,unfortunately, he died early, and now ofcourse we are saddened to hear of Wheeler’sdeath. Wheeler was a very creative man,but very modest and a very pleasant person;he had wonderful ideas. Numerical analy-sis was not his primary interest but in theearly days that was a major component ofcomputer science.

NJH: I’d like to ask you about the back-ground to just a few of your papers. OneI picked out was the 1965 paper “Nu-merical Methods for Solving Linear LeastSquares Problems” in Numerische Mathe-matik, where you introduced the House-holder QR factorization. What is the back-ground to that?

GHG: After I had got my Ph.D. andspent some time in England, I then went toBerkeley and I was working with a physi-cist there for about five months and I reallydidn’t enjoy it. I wasn’t doing any numer-ical analysis; I loved doing numerical anal-ysis, and matrix computations. So I tooka job in Southern California at ThompsonRamo Wooldridge, and that was a very en-joyable experience. My boss was a mancalled Sam Conte. I had spent a summerthere before. I decided least squares was anice subject to work on, and I saw House-holder’s work on using orthogonal trans-formations for solving least squares prob-lems. So I worked it out in further de-tail, but really Householder did much of thework for least squares using what we callHouseholder transformations. But it reallygoes further back if you look at the book

by Turnbull and Aitken. Aitken had theidea much earlier to do that, but my pa-per introduces ideas of updating and col-umn selection and so forth, so it was a muchmore detailed analysis of how to use orthog-onal transformation. But I also made a biggaffe because I thought you could do iter-ative refinement in a straightforward man-ner, but we learned that only in certain cir-cumstances can you get a good approxima-tion.

NJH: A little later, in 1969, you have apaper “Matrix Decompositions and Statis-tical Calculations”. Was this motivated byyour earlier statistical interests?

GHG: Yes, and the fact that I workedon all these orthogonal decompositions andso forth. So I tried to get the statisti-cians interested in doing numerical compu-tations. A few people were interested in it,but I don’t think it had a heavy influence instatistics. I doubt if today people really usedecomposition methods rather than normalequations. Statisticians are fairly fixed intheir ways. That to me is rather sad forthe following reason. In the very begin-ning of statistics the statisticians were verymuch aware of computing and their limita-tions. So they developed ideas on this, andR. A. Fischer had ideas and some of thestatistical techniques were influenced by nu-merical computations. For instance, thereare what are called Latin square designs,and basically those lead to diagonal normalequations. So there may be good statisticalreasons for this too, but basically the de-sign of the statistical experiment is in someway an influence on numerical computation.I think the early statisticians were veryknowledgeable about the fact that com-puting was necessary. Of course we findthat Hotelling dismissed Gaussian elimina-tion because he thought that would not bea good way to proceed, and then he rein-vented the Newton-Schulz method for in-verting the matrix. So it’s quite interest-ing: there was a big gap. In modern times,

5

within the last fifteen years I’d say, there’sreally been more interest in numerical com-puting. There’s a book by John Chambers,who has done work in numerical computing.

I’ll digress here a bit. I spent a yearin Zurich and I had some connection withPeter Huber. Huber is really a very finestatistician and he worked with robustnessin statistics and various other things. Hetrained some students in Zurich and he wasvery much enamoured by computing. Oneof his students was a fellow by the nameof Werner Stuetzle, and Stuetzle came toStanford as a postdoc or assistant professorfor a few years. And then he was involvedwith several people. I don’t know if you arefamiliar with these names. Jerry Friedman,who came to Stanford; he’s a physicist, thenhe got involved in statistical work becauseof Tukey. Then two young men who workedwith Stuetzle, Rob Tibshirani and TrevorHastie, and Werner was really influentialwith those people. Tibshirani and TrevorHastie were younger, but they were all verygood friends and later Werner left and hebecame the head of statistics at the Uni-versity of Washington. These three guyshave gone on and had an enormous effectin statistical computing at Stanford. Theyhave a very well cited book, and then ofcourse later at Stanford they hired DavidDonoho. Statistics is often regarded as amodel for numerical analysis. It has some ofthe same kind of strengths and weaknessesthat we do. When I was a graduate stu-dent they were going through their theorymode and they were trying to prove thingsabout optimality and so forth. At least inthe US people had lost their way to someextent, and they were trying to prove the-oretical results. In the meantime, here inBritain, I think people have always had avery good eye for doing what you mightcall real statistics. In the US people ad-mired the way British statisticians worked,but somehow the theoretical was empha-sized, and it’s only within the last fifteen

years or so, as I see it, there has been are-emphasis on re-thinking using computa-tional methods in statistics, working in thatdirection of interesting statistical computa-tions. I think it’s heavily influenced the waystatistics is done today.

NJH: Another very influential paper ofyours is “Some Modified Matrix EigenvalueProblems” in SIAM Review in 1973. Thatwas one of your earlier papers on this topicand has led to a lot of other work, hasn’tit?

GHG: Well I’m pleased that you saythat because the origins are as follows. Iwas on sabbatical one year and I had beencollecting results. For instance, I workedon eigenvalue problems with homogeneousconstraints; that was at the suggestion ofHenrici, as a matter of fact, and I devised avery simple algorithm for doing a computa-tion, and then I discovered that this prob-lem, minimizing quadratic forms subject tolinear constraints, comes up in very manydifferent places. So I had been working on alot of small results, and then the time cameI had to show I had done something on mysabbatical. So I wrote down this review pa-per. I love that paper, in part because it hasa lot of nice useful tools for solving matrixproblems.

NJH: You must have fond memories ofSerra House, which was the Computer Sci-ence Department’s home on the Stanfordcampus. Is there anything in particularthat stands out in your memory about it?

GHG: Well, it was a terrific period, andoriginally everybody in the Computer Sci-ence Department and Computation Centerwas housed in Polya Hall. Polya was a pro-fessor at Stanford, and he was in fact evenalive when Polya Hall was built and he wasvery much loved by everybody. Our de-partment and the center kept on growing,and the center wanted of course more andmore space. I don’t know if you recall thelarge computation centres; they had roomsfull of computers and people and every-

6

thing else. So we outgrew our building andthen there is always a scramble to find morespace. The numerical analysts found them-selves in Serra House. We had the groundfloor of the house and people who workedin artificial intelligence had the upper floor.Forsythe’s office was still in Polya Hall whenhe was head of the department. When hedied he was still head of the department,and Jack Herriot, who worked with Georgeon administering the department, was alsoat Polya Hall. But the younger numericalanalysts and myself were all in that build-ing, and as we had visitors like Wilkinson,Dahlquist, and Gautschi and others theyall sat in Serra House. It had a nice at-mosphere. We had a very wide hallway,and I collected reprints and they were allin this hallway. Even our secretary satin the hallway as space got tighter andtighter. We had a kitchen, but the kitchenwas turned into a terminal room. It wasa very nice period. You may have seenthe directory for Serra House at one time,with people like Dahlquist and Wilkinsonand then the array of students that we hadlike Nick Trefethen, Randy LeVeque, Mar-sha Berger, Petter Bjorstad; there was justa great group of students that we had at thetime.

NJH: You mentioned Wilkinson, whoyou obviously knew very well for manyyears. Do you have any particular mem-ories of him?

GHG: Well I can’t sum it up in a verysimple statement. He was a wonderful man.I think not only was he brilliant, but hewas full of life and he knew so much. Heknew about music and literature, he loveda drink, he loved life. It was just good tobe in his presence. He was a very charm-ing, helpful person. Everybody felt happybeing around him and loved to talk to him.He had a nice sense of humour, and he wasaltogether a very good person. He was spe-cial and I miss him greatly.

NJH: Over the years you’ve been a

strong advocate of the Lanczos method.Did you ever meet Lanczos?

GHG: On two occasions. I met himat NPL one time. It was very auspiciousbecause I went down with Velvel Kahan,who was at Cambridge too, and Lanczostalked about the singular value decompo-sition. You know, one of the nicest de-scriptions of it is in his book Linear Dif-

ferential Operators. Afterwards I tried tospeak to him. I said “P-p-r-o-ff-essor L-l-anczos. . . ”, and I was wanting to tell himabout my work on Chebyshev polynomi-als. Then two powerful arms came aroundmy shoulders and moved me aside. It wasVelvel Kahan; he felt he had first priority.Lanczos later spoke at a meeting in Irelandthat John Miller had organized. He gave atalk about computing instruments, comput-ing methods, and he went slowly throughthe history until about 1940 and then heskipped, and the next thing he showed usan HP hand calculator. So he really hadn’tcome to terms with modern computing. Itried to talk to him afterwards about theSVD, for which I had by that time devel-oped an algorithm, but he didn’t seem veryinterested, and he died a couple of years af-ter that. He was of course a great influenceon all of us. I wish we could talk to hima little bit more because I’ve often felt theLanczos method is really the Stieltjes algo-rithm with a different inner product, and hemust have known that.

NJH: Another paper I’d like to ask youabout the background to is your 1969 paper“Calculation of Gauss Quadrature Rules”with Welsch in Mathematics of Computa-tion. How did you get interested in thatparticular topic?

GHG: Well, I’m not sure how I becameparticularly interested in quadrature. PaulConcus was a dear friend of mine and askedme some things about quadrature, and thenI realised that all that’s really necessary isto compute eigenvalues of matrices. I wasseated in Peter Lax’s office. I was on leave

7

at the Courant Institute and I was sort ofbored by what was taking place, so I hap-pened to see a book by Wilf. There hehad some of the relationships about com-puting the quadrature weights, and thenI worked out the fact that all you need isthe square of the first element of the eigen-vector. Actually, other people had figuredthis out too and known this before. WhatI was able to do is to recognize that whenyou mix that with the QR method you canorganize the QR method just to computethe first component of an eigenvector. SoI was really pleased when that all fell intoplace. Later on I saw Rutishauser, and Itold him that I had this algorithm for com-puting the quadrature rules and how I wasdoing it. The interesting thing is that hedenied it could be done that way. Ordinar-ily I’m not a very confident person, but Iknew there was a program that was work-ing and that actually calculations could bedone that way. What pleased me of coursewas that it eliminated all the use of all theselarge tables, and the knowledge that youcan compute a quadrature rule in this way,and with the weights, has really been veryuseful in many different studies.

NJH: Many of your papers are joint. Doyou have any particular way of working withyour collaborators?

GHG: No. I think in current times Ireally talk to a lot of young people and Isort of suggest a way of doing something orother and then they go off and do it, andthen there is a collaboration that way. Ilike working with people. I don’t have ahigh ability of writing on my own, so I liketo work with people who are willing to col-laborate in that fashion.

NJH: Have you ever felt that a result,paper or book of yours has not receivedthe attention it deserved, and, if so, whichand why?

GHG: The work I’ve done on matrices,moments and quadrature—I don’t thinkthat is widely known, and yet I think it’svery rich and I’m hoping there will be abook by Gerard Meurant and myself on thattopic. On the whole, sometimes I’m as-tounded by the fact that people know ofa paper that I have written. Actually, thebooks with Ortega are not so well known,and they contain a lot of good things.

NJH: Do you have any comments on therelationship between core linear algebra andnumerical linear algebra, and how it haschanged with time?

GHG: I feel most people who are inapplication areas learn linear algebra be-cause they really want to do matrix anal-ysis and I think the subject has been hi-jacked by the mathematicians who under-stand about applications; but when theyteach their courses they don’t necessarilyemphasize the aspects we’re familiar with.My own feeling is that the deeper you un-derstand the matrix computation aspect,the better you understand what linear al-gebra is about. For instance, rank is a wellknown concept in mathematics. In truth,rank is a discrete topic, but if you look atit from a numerical point of view it almosthas a continuous flavour to it. So I feel it’sunfortunate, but the mathematical worldhas hijacked the linear algebra topics whenit should really be matrix algebra, becausepeople I think are interested in matrix anal-ysis. If you recall, the journal SIMAX3 hasthe phrase “Matrix Analysis and Applica-tions” in the title. I did that very deliber-ately. I didn’t want to see another linear al-gebra journal. My motto for this is the bookby Bellman on matrix analysis and applica-tions, so in a sense the journal is a tributeto him.

NJH: You actually founded two SIAMjournals. Before SIMAX was SISC4 in 1980,

3The SIAM Journal on Matrix Analysis and Applications, founded in 1988.4The SIAM Journal on Scientific and Statistical Computing, renamed in 1993 the SIAM Journal on

Scientific Computing.

8

so what was the motivation for SISC?

GHG: Well, Jim Ortega and I were work-ing together on things. Ortega was really agreat friend and colleague. We recognizedthat such journals as SINUM5 were goingso heavily into the theoretical aspects ofcomputing, of numerical analysis. So wereally wanted to redress the subject, andit’s worked partially. But of course there isalways a drifting that takes place.

NJH: I think you were involved in set-ting up NA-NET. Is that right, and what’sthe background of that?

GHG: Jim Wilkinson was at Stanfordand he just said, “What’s the address ofso and so?”, “What’s the address of so andso?” So then I thought we should have analias for everybody. Of course it wasn’t soeasy to find people’s addresses then, so Imade up a little list. I had seven or eightnames and then we had somebody work-ing in the computing centre who helped meout, and so we devised the beginning of NA-NET and the alias format. Originally, asmessages came in we just circulated them,and then we decided to go into the digestformat. I only did a couple of digests beforewe had the current format and that’s reallybloomed under Cleve’s hand.

NJH: I must ask you about your bookMatrix Computations with Charlie VanLoan, originally published in 1983, now inits third edition (1996). When did you ac-tually start writing the book?

GHG: I gave some lectures at JohnsHopkins University. What happened is,they formed a Math Sciences Departmentat Johns Hopkins and the head was RogerHorn. I had known Roger Horn becausehe was in the first course of advanced nu-merical analysis I taught. I taught threequarters of the course; the first two quar-ters were out of Varga’s book. Roger Hornis a wonderful fellow, and I hadn’t realisedwhat impact that had on him because, for

instance, he told me he learned about thePerron–Frobenius theory from me and alsothe singular value decomposition, and thosehave both been of some importance to him.So Johns Hopkins had a series of lectures,and he invited me to give lectures. I gaveten lectures in two weeks or so. The classconsisted of a whole bunch of people; I don’tknow specifically who now, but Charlie VanLoan was there and also Richard Bartels.Originally the idea was for the three of usto write a book, but then Richard’s inter-ests were developing otherwise; it was goingto include a large component of math pro-gramming. So then it was just Charlie andI. He did most of the writing, as you prob-ably know, but it was really a great stim-ulant for both us. For instance, although Ihad done some work on total least squares,we developed a paper on total least squareswith a lot of analysis and showing how thecomputations will go, and so forth. So thatwas really a wonderful effort, and he wouldsend me some section and I would say, “Oh,well, we can do better like this.” There isthe Sylvester equation that we did work on;again I looked at it and realised we could doone eigenvalue computation: we did one up-per Hessenberg form and one Schur decom-position. So that was really a great periodfor both of us because we were incorporat-ing new material in the book and we werealso writing the book.

NJH: Were you surprised at how success-ful the book has been?

GHG: No, not at the beginning at anyrate, because the publisher kept on say-ing, “When are you going to get that bookout?”, and I kept on saying, “Don’t worry,it’s going to be a good book. This will re-ally have a market”—and it did. It’s nowsold over 50,000 copies, the three editions.If you Google the book you’ll find that thereare over 10,000 references, so Charlie said,“I think every 18 hours a paper is being

5The SIAM Journal on Numerical Analysis.

9

written which has reference to our book.”I’m pleased that it’s been a help to people.Electrical engineers I do know like it, andpeople in various specialties have found it ofgreat use. We are planning in about a year,when Charlie is no longer chairman and willbe on sabbatical, that I’ll take a sabbaticaland we will revise the book. So there is aquestion, how to limit a book: should youhave two volumes, and if you do have twovolumes what should they include, and soforth. I think it’s been a great windfall forJohns Hopkins. You know, most of theireditions probably sell in the hundreds, or afew thousand, so they have probably donewell, and I am pleased for that. I think thecommercial presses have acted in very poorways, and I think it’s good that academicinstitutions and professional organizationsget the benefits of the books.

NJH: You’ve always travelled exten-sively. Are there any countries you haven’tvisited academically that you would like tovisit?

GHG: I’d like to go to South Africa. I’vehad invitations, I’ve accepted them, andthen somehow I couldn’t quite make thelong trip. There are some excellent peoplethere.

NJH: One of your current research top-ics is the linear algebra of search engines.Are you surprised by this new applicationof this subject?

GHG: Not completely; I’m just sopleased. The way it happened is a stu-dent asked me about finding the stationaryprobability distribution of a large matrix. Idon’t think he knew exactly what he wasasking about, so I spent a few hours. Ofcourse, everything I suggested could havebeen done fifty years ago and it’s really anacceleration of the power method. Now Ihave another paper with Chen Greif wherewe use the Arnoldi method for acceleratingconvergence, and that seems to work quitewell too. I taught a course about linear al-gebra and computer science and one of the

students was from Google. He said, “Well,that’s only the stuff you talk about; how weorder things is just a small part of search en-gines.” Secondly, he said, “We use the sin-gular value decomposition all the time”, buthe didn’t go on any further to tell me howit’s being used. So we’re lucky at Stanford.Even I have to admit I have had a success-ful career. I think two great componentsare, first, Stanford has served as a magnet.People have come through, and I think Ihave greatly benefited by being in such astrong computer science department. Sec-ondly, the Bay Area. It’s not quite as truenow as it was, but we have these variouslaboratories in the area: Lawrence Liver-more Laboratory, Lawrence Berkeley Labo-ratory, NERSC, IBM had some activity inthe past, and Intel. So the area within afifty mile radius is honeycombed with orga-nizations, and some are interested in numer-ical mathematics—well, some should be in-terested in numerical mathematics—so it’sreally fortunate being here.

NJH: What about future research plans?

GHG: Well, I’ve never really had a fixedprogramme in research, and I just like todo what comes up. I’m concerned thatmaybe I’ve over-farmed what I know. Onthe other hand, there always seems to beanother application of Householder trans-formations and the singular value decompo-sition, but I reckon I’m coming to the endof that line of research. It’s a complicatedsituation. The younger people, of course,they go in different directions. There arestill problems of the sort that I mentioned,and maybe there aren’t so many people whoknow all the intricate details that I know,although some people know them better.

NJH: Is there anything I should haveasked that I didn’t—that you think youwould have wanted me to ask?

GHG: There are some people I wouldlike to talk about. One is George Forsythe.He established what we have at Stanford,although in retrospect I have been there

10

three times as long as he was there. Hecame in 1957, so it’s close to fifty years,and in fact I am hoping to have some sortof celebration of that6. He had some won-derful students, like Moler, Varah, Ortegaand Parlett. He really had first rate peo-ple, so really established a tone in numer-ical analysis that enabled us to build upsomething which was really good. Forsythehas been dead for over 30 years, but hewas an important person, not because hewas very original, because there aren’t somany things you can point to as contribu-tions from Forsythe, but he just pointedpeople in the right direction. In fact, onereason I worked on the SVD is because atthe end of a lecture by Ben Rosen, Forsythesaid, “Would somebody please figure outhow to compute the pseudo-inverse of amatrix.” He was very supportive of stu-dents. This came out again the other day:that he wanted to bring Moler to an earlyHouseholder meeting, and people actuallyopposed that, and Forsythe said, “Well, ifyou don’t want Moler then I’m not goingto attend”, so just think now what an im-portant component of a Householder meet-ing is having students there. Another man Igreatly admire—I’ll give you my icons in thefield, partially—is David Young. I’ve reada lot of his thesis and his early work, andhis thesis is genuinely a wonderful piece ofwork and I’m afraid people won’t know it sowell; there is that parameter that’s so hardto compute, it seems. But the analysis thereis just breathtaking in terms of the origi-nality and showing what you can do withan algorithm by choosing one little numberproperly. Again he led the way. Later onpeople wrote about block SOR and variousother aspects of the problem, but his thesiswas really a fantastic piece of work, and Ithink it’s something that we can all greatlyadmire. Dahlquist is one of my personalheros, not only because of his brilliance as

a scientist, but as a human being he just wasvery concerned about other people, and hehad many very good ideas. He was a veryoriginal man and a great scientist. Anotherperson I admire greatly is Walter Gautschi,because he is such a serious scientist, such aserious scholar; he has done a lot of wonder-ful work. Those are just a few of the namesoff the top of my head; there are all kindsof other names.

NJH: One other thing that just comes tomind, because I was reading a book aboutthis last week from SIAM, is the Mathe-matics Research Center at Madison. Didyou have much contact with that?

GHG: Yes, in fact they even offered mea position at one time. I went to Chicagoto see my mother and then I was going togo on to MRC for a few weeks. When Iwas in Chicago I had some spare time andI went to see a man called Victor Barcilon.He was telling me about his work on inverseeigenvalue problems, and he said to me hewanted to solve not the tridiagonal case butthe five diagonal case. After about a weekin Chicago I went up to Madison and thereI started working with Carl de Boor. Weworked on the tridiagonal case and while wewere working on it I recognised the use ofthe Lanczos algorithm for inverse eigenvalueproblems. We wrote a paper which sort ofgot things right in the inverse eigenvalueproblem for tridiagonal matrices. Then Iwent back to Stanford and I started workingon the five diagonal and the general case.Ironically, I sent Victor Barcilon a copy ofthe paper, but, as it turns out, althoughyou more or less get a unique solution forthe tridiagonal case when choosing the setsof eigenvalues, you don’t get a unique solu-tion if you have three sets of eigenvalues andwant to build up the five diagonal case. In-deed, Gil Strang at a meeting in Moscow re-cently was talking about the problems aris-ing when you have n(n+1)/2 eigenvalues—

6http://compmath50.stanford.edu/

11

the eigenvalues of all the leading principalsubmatrices; you are far from uniqueness inany of these cases. So Barcilon said, “Well,I’m not interested and if you can’t give me aunique solution I don’t want the solution”,and he wrote some other paper. By the way,I now have a book with Moody Chu thatjust came out on inverse eigenvalue prob-lems.

So I love that subject. I did work onGauss quadrature and then I did some workon QR-like algorithms for tridiagonal ma-trices and stuff on moments, and then sud-denly one day in the Lanczos algorithm itall came together and it was just wonder-ful. It was a big thrill to see that they allrelated to one another.

The other thing I have been so proud ofI guess are the students that we’ve had atStanford. Some have been my students andsome have not been my students. We’vejust been very fortunate in the kinds of peo-ple that have come to Stanford. It’s reallya great period that we’ve had. My personallife hasn’t been as happy as I would haveliked, but when I look back at this aspectof life it’s something that’s going to give megreat pleasure.

So when I think about the influences ofmy life, I think being a student at the Uni-versity of Illinois, that really had an enor-mous impact on my life. Being at Cam-bridge for a year—although I didn’t do verymuch—in terms of friendships I still havemany friends even though it’s over 45 yearssince I first came here, and not all of themare connected with the profession, but theyall had a great impact on my life. And alsobeing at Stanford of course; I’ve been theresince 1960, so 45 years. The way we meetpeople is serendipitous: we bump into peo-ple, we talk to people, so that’s really beena great satisfaction.

Bibliography

[1] Richard Bellman. Introduction to Matrix

Analysis. McGraw-Hill, New York, secondedition, 1970. xxiii+403 pp. Reprinted bySociety for Industrial and Applied Math-ematics, Philadelphia, PA, USA, 1997.ISBN 0-89871-399-4.

[2] John M. Chambers. Computational Meth-

ods for Data Analysis. Wiley, New York,1977. ISBN 0-471-02772-3. xiii+268 pp.

[3] Raymond H. Chan, Chen Greif, and Di-anne P. O’Leary, editors. Milestones in

Matrix Computation: The Selected Works

of Gene H. Golub, with Commentaries.Oxford University Press, 2007. ISBN 978-0-19-920681-0. xi+565 pp.

[4] Jagdish Chandra and Stephen M. Robin-son. An Uneasy Alliance: The Mathemat-

ics Research Center at the University of

Wisconsin, 1956–1987. Society for Indus-trial and Applied Mathematics, Philadel-phia, PA, USA, 2005. ISBN 0-89871-535-0. xi+89 pp.

[5] Moody T. Chu and Gene H. Golub. In-

verse Eigenvalue Problems: Theory, Al-

gorithms, and Applications. Oxford Uni-versity Press, 2005. ISBN 0-19-856664-6.xvii+387 pp.

[6] Jack J. Dongarra, Gene H. Golub, CleveMoler, and Keith Moore. Netlib and NA-Net: Building a scientific computing com-munity. MIMS EPrint 2007.123, Manch-ester Institute for Mathematical Sciences,The University of Manchester, UK, 2007.18 pp. To appear in IEEE Annals of theHistory of Computing.

[7] G. H. Golub. Numerical methods for solv-ing linear least squares problems. Numer.

Math., 7:206–216, 1965.

[8] G. H. Golub and J. H. Wilkinson. Noteon the iterative refinement of least squaressolution. Numer. Math., 9:139–148, 1966.

[9] Gene H. Golub. Matrix decompositionsand statistical calculations. In Roy C.

12

Milton and John A. Nelder, editors, Sta-

tistical Computation, pages 365–397. Aca-demic Press, New York, 1969.

[10] Gene H. Golub. Some modified matrixeigenvalue problems. SIAM Rev., 15(2):318–334, 1973.

[11] Gene H. Golub and Gerard Meurant. Ma-trices, moments and quadrature. In D. F.Griffiths and G. A. Watson, editors, Nu-

merical Analysis 1993, Proceedings of the

15th Dundee Conference, volume 303 ofPitman Research Notes in Mathematics,pages 105–156. Addison Wesley Longman,Harlow, Essex, UK, 1994. Reprinted in [3].

[12] Gene H. Golub and James M. Ortega. Sci-

entific Computing and Differential Equa-

tions: An Introduction to Numerical

Methods. Academic Press, New York,1992. ISBN 0-12-289255-0. viii+337 pp.

[13] Gene H. Golub and James M. Ortega.Scientific Computing: An Introduction

with Parallel Computing. Academic Press,New York, 1993. ISBN 0-12-289253-4.viii+442 pp.

[14] Gene H. Golub and Charles F. Van Loan.Matrix Computations. Johns HopkinsUniversity Press, Baltimore, MD, USA,third edition, 1996. ISBN 0-8018-5413-X (hardback), 0-8018-5414-8 (paperback).xxvii+694 pp.

[15] Gene H. Golub and John H. Welsch. Cal-culation of Gauss quadrature rules. Math.

Comp., 23(106):221–230, 1969.

[16] Trevor Hastie, Robert Tibshirani, andJerome Friedman. The Elements of Sta-

tistical Learning: Data Mining, Inference,

and Prediction. Springer-Verlag, NewYork, 2001. ISBN 0-387-95284-5. xvi+533pp.

[17] Cornelius Lanczos. Linear Differential

Operators. Van Nostrand, New York,1961. ISBN 0-486-68035-5. xvi+564 pp.Reprinted by Dover, New York, 1997.

[18] N. J. Mackintosh, editor. Cyril Burt:

Fraud or Framed? Oxford Univer-sity Press, 1995. ISBN 0-19-852336-X.vii+156 pp.

[19] H. W. Turnbull and A. C. Aitken. An

Introduction to the Theory of Canonical

Matrices. Blackie, London and Glasgow,1932. xiii+200 pp. Reprinted with ap-pendix, 1952.

[20] Maurice V. Wilkes, David J. Wheeler, andStanley Gill. The Preparation of Programs

for an Electronic Digital Computer. With

Special Reference to the EDSAC and the

Use of a Library of Subroutines. Addison-Wesley, Cambridge, Massachusetts, 1951.Reprinted by Tomash Publishers, Los An-geles, 1982, Volume 1 in the Charles Bab-bage Institute Reprint Series for the His-tory of Computing, with a new Introduc-tion by Martin Campbell-Kelly.

13