John Reid’s influence in

informatics and mathematical modelling

Kaj MadsenTechnical University of Denmark

Numerical Analysis GroupComputer Science and Systems DivisionA.E.R.E. Harwell

Allan CurtisRoger FletcherMike PowellJohn Reid

August 1973

Space Mapping

Physical problem

Rf fine model

Rc

coarse model

Connect similar residuals

. xf*. xc*

P

( ) ( ( ))f f c fR x R P xJohn Bandler, 1993

Fortran programming

Polynomial zeros

Spring 1974:

John was Visiting Professor at

Institute for Numerical Analysis

Technical University of Denmark

Owe Axelson:Solution of linear systems of equations: iterative methods

J. Alan George: Solution of linear systems of equations: direct methods for finite element problems

John K. Reid:Solution of linear systems of equations: direct methods (general)

Axel Ruhe:Computation of eigenvalues and eigenvectors

Numerical Analysis

Per Christian Hansen

T. K. Jensen and P. C. Hansen, Iterative regularization with minimum-residual methods, BIT, 47 (2007), pp. 103–120.

Scientific Computing

Image Deblurring

deblurring

Io (moon of Jupiter)

blurring

John Reid & Conjugate Gradients

It took a few years for researchers to realize that it was more fruitful to consider the conjugate gradient method truly iterative. In 1972, John Reid was one of the first to point in this direction.

Henk A. van der VorstKrylov Subspace Iterations

Computing in Science and Engineering, IEEE, 2000

J. K. Reid, The Use of Conjugate Gradients for Systems of Equations Possessing ’Property A’, SIAM J. Numerical Analysis, 9 (1972), pp. 325–332.

T. K. Jensen and P. C. Hansen, Iterative regularization with minimum-residual methods, BIT, 47 (2007), pp. 103–120.

Example (2953903 = 345150 unknowns)

Scientific Computing

Computer Science

Wireless Networks for Smart Energy

Devices join and leave a secure wireless network as described by a Markov Chain.

When devices leave there is a risk that the security is compromised.

ZigBee devices contain tiny microprocessors have limited memory, and

are deployed in home and industrial settings.

What is the trade-off between installing

new security keys and the risk of security flaws?

Wireless Networks for Smart Energy

The question:

The model:

The matrix:ZigBee devices

contain tiny microprocessors have limited memory, and

are deployed in home and industrial settings.

Scientific ComputingOperations ResearchStatisticsImage Processing

Computer Science

Principal Component Analysis (PCA)[Karl Persson (1901)]

Brain Morphometry

Image from temagami.carleton.ca

The corpus callosum is the nerve fiber bundle that connects the two hemispheres of the brain.

Local atrophy correlates to loss of particular ability, e.g walking speed, verbal fluency (age-related degeneracy)

F

M

S A

P/T

V

In a study of 600 elderly the CC outline was extracted using automated image analysis on MRI brain images

Each outline is represented by a list of corresponding ”landmark” coordinates sampled along the outline

),...,,,,...,,( 2121 pp yyyxxxx

We want to find local (sparse) variations from the mean CC shape to be used in predicition of cognitive and clinical parameters such as max. walking speed and verbal fluency

The shape coordinates are projected onto the first few (sparse) principal components before regression

Reconstruction error

Transformation to PCA space and back

Sparse principal components

kkT

k

jjj

k

jj

n

ii

Ti

IAA

llxALxLALA

subject to

minarg)ˆ,ˆ(1

11

2

1

2

,

Transformation to k-D PCA-space

Elastic net type regularization

Keep loading matrix L near orthogonal

For k = 0, A = L is the ordinary principal component loadings. For positive ’s L is sparse

mean

slowerWalking Speed

Regression of walking speed on the sparse eigen modes identifies two significant modes representing atrophy in the nerve fibers connect the motor control centres and cognitive centres of the brain, respectively

Sparse Decomposition and Modeling of Anatomical Shape Variation Sjöstrand, Karl ; Rostrup, Egill ; Ryberg, Charlotte ; Larsen, Rasmus ; Studholme, Colin ; Baezner, Hansjoerg ; Ferro, Jose ; Fazekas, Franz ;

Pantoni, Leonardo ; Inzitari, Domenico ; Waldemar, Gunhildin journal: IEEE Transactions on Medical Imaging (ISSN: 0278-0062) , vol: 26, issue: 12, pages: 1625-1635, 2007

Scientific ComputingOperations ResearchStatisticsImage ProcessingSignal AnalysisComputer Science

…

A1,1A2,1A1,2

The CocKtail Party Problem

References

[1] P. Comon, Independent component analysis, A new concept?, Signal Processing (36)287-314,1994

[2] T. Bell, T. Sejnowski, An information maximisation approach to blind separation and blind deconvolution, Neural Computation (7) 1129-1159, 1995

[3]  L. K. Hansen, J. Larsen and T. Kolenda, On Independent Component Analysis for Multimedia Signals, Multimedia Image and Video Processing, 175-199, 2000

IC1

IC2IC3

The CocKtail Party Problem

…

A1,1A2,1A1,2

Solution: As the distribution of unmixed speech signals are sparse optimizing for such that becomes sparse solves the above ambiguity up to scale and permutation of the sources. This solution can be obtained through a method named Independent Component Analysis (ICA) [1-3], i.e:

ICA solution for SMixture XTrue sources S

Problem: From mixture recover mixing matrix and underlying sources . There are infinitely many potential solutions, i.e.

where is an invertible matrix.

Mixed signals X are not in general as sparse as true underlying sources S.

Happy Birthday, John !