lihong feng - max planck society...moment-matching method lihong feng outline • orthogonality of...

Moment-matching method

Lihong Feng

Outline

• Orthogonality of two vectors

• Gram-Schmidt (modified Gram-Schmidt) process

• Orthogonality of a vector to a group of orthogonal vectors

• Arnoldi algorithm

• Preliminaries

• Method based on Pade approximation, explicit moment-matching (AWE)

• Method based on Pade, Pade-type approximation, implicit moment-matching

• Method based on rational interpolation

Angle between two vectors:

Two vectors u, v:

Inner products in : uvvuT

Preliminaries

Orthogonality of two vectors: 00||||||||

0)cos(22

vuvu

vu TT

22 ||||||||)cos( vuvu

nR

a

b

Pb

acc :

• Orthogonalization of two vectors

If Pb is the projection of b onto a,

then c=b-Pb is orthogonal to a.

How to compute c?

maPb (m is a scalar)

amabPbbc

0)( mabaT

aa

bam

T

T

Finally: aaa

babc

T

T

nRba ,

},{},{ acspanabspan

An important information:

Preliminaries

• Orthogonalization of a vector b to a group of orthogonal vectors laaa ,,, 21

b

Pb

la2a

1a

c=b-Pb

c=b-Pb

llamamamPb 2211 ac

0)( 2211 llTi amamamba

0jTi aa

and

iTi

Ti

iaa

bam

l

lTl

Tl

T

T

T

T

aaa

baa

aa

baa

aa

babc 2

22

21

11

1

},,,,{},,,,{ 2121 ll aaacspanaaabspan

Preliminaries

Step 1:

It is used to orthogonalize a group of vectors mbbb ,,, 21

• Gram-Schmidt process:

21, bb 111

2122

~b

bb

bbbb

T

T

direction in

b1

Step i:

321 ,~

, bbb2

22

321

11

3133

~~~

~~

bbb

bbb

bb

bbbb

T

T

T

T

direction in

Step 2:

ibbbb ,~

,~

, 321

2

~b

1

11

12

22

21

11

1 ~~~

~~

~~

~~

ii

Ti

iT

i

T

iT

T

iT

ii bbb

bbb

bb

bbb

bb

bbbb

direction in

1

~ib

Preliminaries

Gram-Schmidt process:

for i=2,3, ..., m

1

11

12

22

21

11

1 ~~~

~~

~~

~~

ii

Ti

iT

i

T

iT

T

iT

ii bbb

bbb

bb

bbb

bb

bbbb

end

What is the relation between and ? mbbb ,,, 21 mbbb~

,,~

, 21

}~

,,~

,{},,,{ 2121 mm bbbspanbbbspan

Preliminaries

Gram-Schmidt process:

for i=2,3, ..., m

1

11

12

22

21

11

1 ~~~

~~

~~

~~

ii

T

i

i

T

i

T

i

T

T

i

T

ii bbb

bbb

bb

bbb

bb

bbbb

end

It is accurate in

accurate arithmetic,

brings errors in finite

arithmetic, not quite

orthogonal

Computation with computers is finite arithmetic!

for i=2,3, ..., m

for j=1,2, ..., i-1

j

j

T

j

i

T

j

ii bbb

bbbb

~~~

~~~~

end

end

Any difference, and

what difference?

Numerically stable.

Preliminaries

Modified Gram-Schmidt process:

ii bb ~

Usually the vectors are required to be orthonormalized, so that there will be no

overflow in the computation with computers.

Modified Gram-Schmidt process :

for i=2,3, ..., m

for j=1,2, ..., i-1

j

jTj

iTj

ii bbb

bbbb

end

end

|||| 1

11

b

bb

|||| i

ii

b

bb

What if is zero or

close to zero? What does

it mean?

|||| ib

Preliminaries

Modified Gram-Schmidt process with deflation :

||||/ 111 bbb

for i=2,3, ..., m

for j=1,2, ..., i-1

j

jTj

iTj

ii bbb

bbbb

end

end

bii bb /

If

|||| ib b

else

delete ib

end

deflation

Preliminaries

tolb

},,,,{span),( 12 rArAArrrAK pp

• Arnoldi algorithm:

It computes an orthonormal basis for the Krylov subspace: qvvv ,, 21

The core in Arnoldi algorithm is Modified Gram-Schmidt process.

i.e. ),(},,,{span 21 rAKvvv qp

Preliminaries

Arnoldi algorithm

||||/1 rrv

},,,,{),( 12 rArAArrrAK pp

for i=2,3, ..., p

for j=1,2, ..., i-1

j

jTj

Tj

vvv

wvww

end

1 iAvw

|||| ww

If

wi wv /

else

stop

end

end

Why?

pqrAKvvv pq ),,(},,,{span 21

It is clear:

Preliminaries

tolw

Transfer function

The transfer function is a function of s.

)(ˆ)( sHsH )(ˆ sHDoes there exist a *easier to compute* such that ?

Motivation of AWE method

.0)0(),()(

),()()(

xtCxty

tButAxdt

tdxE

Original large-scale system

BAsECsH 1)()(

AWE method[Pillage,Rohrer ‘90] : Asymptotic waveform evaluation method.

Padé approximation

• Rational function:

A rational function is the quotient of two polynomials PN(x) and QM(x) of degree N and M respectively:

bxaforxQ

xPxR

M

NMN ,

)(

)()(,

• Padé approximation:

Approximates a function f(x) (analytic) by a rational function, and requires that f(x) and its derivatives be continuous at x=0.

The transfer function can be approximated by Padé approximation!

• PN(x) and QM(x):

.1)(

)(

221

2210

MMM

NNN

xqxqxqxQ

xpxpxppxP

• Notice that in QM(x), q0=1, which is without loss of generality. Because, RN,M(x) is not changed when both PN(x) and QM(x) are divided by the same constant.

)(

)()(,

xQ

xPxR

M

NMN

Padé approximation

Padé approximation:

.1)(

)(

21

2210

MMM

NNN

xqxqxqxQ

xpxpxppxP

Maclaurin expansion: f (x) = f0 + f1x + f2x2 +· · ·+fk x

k +· · · ,

The coefficients in PN(x) and QM(x) can be computed by requiring : f(x) and RN,M (x) agree at x=0 and at their derivatives (at x=0 ) up to N+M degree.

This implicates:

1

, )(:)()(MNj

j

jMN xexexfxR

Maclaurin expansion: RN,M(x) = r0 + r1x + r2x2 +· · ·+rk x

k +· · · ,

)(

)()(,

xQ

xPxR

M

NMN

How to compute Padé approximation

)()(/)()()(, xfxQxPxfxR MNMN

1

, )()(MNj

j

jMN xexfxR

11

~)()()()(MNj

j

j

MNj

j

jMMN xexexQxfxQxP

:0x 000 pf

:1x 01101 pffq

:Nx

011 NNMNMMNM pffqfq

(1)


:1Nx 011211 NNMNMMNM ffqfqfq

:2Nx 0211312 NNMNMMNM ffqfqfq

:MNx 01111 MNMNNMNM ffqfqfq

(2)

M unknowns and M equations in (2), qis can be obtained by solving (2),

pis can be immediately obtained from (1) without solving equations.


An example:

10,1)( xxxf

xq

xpp

xQ

xPxR

1

10

1

11,1

1)(

)()(

10 000 ppf

02/11)2/1(

0

1

1101

p

pffq

2/1/0 121211 ffqffq

.8/3)0( ;4/1)0(

;2/1)0( ;1)0(

)3(

3

"

2

'

10

ffff

ffff

1)

2)

3) )(1,1 xR

221

10

2

12,1

1)(

)()(

xqxq

xpp

xQ

xPxR

0

0

32112

21102

ffqfq

ffqfq1)

3

2

2

1

12

01

f

f

q

q

ff

ff

25.0

1

2

1

q

q

000 pf

01101 pffq

2)

5.0

2/11

1

1

0

p

p

3) )(2,1 xR


0 0.2 0.4 0.6 0.8 11

1.1

1.2

1.3

1.4

1.5

1.6

1.7

1.8

1.9

2

x

R11

f(x)

R12

0 0.2 0.4 0.6 0.8 11

1.1

1.2

1.3

1.4

1.5

1.6

1.7

1.8

1.9

2

x

R11

R12

R22

f(x)


0 0.5 1 1.5 2 2.5 30

5

10

15

20

25

30

35

40

x

f(x)=exp(x)

R11

R22

R33

0 1 2 3 40

10

20

30

40

50

60

70

80

x

f(x)=exp(x)

R33

Padé approximation is only accurate around 0


N, M ?

MOR: AWE based on Padé approximation

MOR: AWE tries to find a Padé approximation of H(s). )(ˆ sH

)(

)()(,

xQ

xPxR

M

NMN

• How to choose N and M in PN(x) and QM(x)?:

Proposition*: For a fixed value of N+M, the error is smallest when PN(x) and QM(x) have the same degree: N=M or when PN(x) has degree one higher than QM(x): N=M+1.

*from the book: John H. Mathews and Kurtis K. Fink, Numerical Methods Using Matlab, 4th Edition, Prentice-Hall Inc. Upper Saddle River, New Jersey, USA 2004.


Assuming is diagonalizable: ),,,(,~

21

1

ndiagZZA

bsIc

bZsIZc

bAAsIcsH

T

T

T

~)(~

~)(

)()~

()(

1

11

11

n

j j

jj

s

blsH

11

~~

)(

• H(s) is a rational function. • Numerator polynomial is of degree at most n-1, denominator polynomial is of degree at most n.

)()()()( 1111 bAIEsAcbAsEcsH TT

EAA 1~

)(

)()(

tcxy

tButAxdt

dxE

Given a system

The transfer function is


Therefore, it is natural to take M=r, and N=r-1

)(

)(

)(

)()( 1,1

xQ

xP

xQ

xPxR

r

r

M

Nrr

rr

rr

r

r

sqsq

spspp

sQ

sPsH

1

11101

1)(

)()(ˆ

Computing the coefficients:

Pade approximation requires: The values of H(s) at s=0, and the derivatives of H(s) at s=0 till r-1+r degree should be the same as those of . )(ˆ sH


2

210

00

~~)( smsmmsmsbAlsH

i

i

i

i

i

iT

Derivatives of H(s) at s=0 are the coefficients of Maclaurin series of H(s):

2

210

1

1110 ˆˆˆ

1)(ˆ smsmm

sqsq

spsppsH

rr

rr

12

122

2

1

11102

2101

)( rrr

rrr

rr sesesqsq

spsppsmsmm

i

i

iTTT sbAcbAEsAIcbAsEcsH

0

1111 ~~)()()()(

,1,0,~~

ibAcm iTi are the moments of the transfer function.

A~

b~


011110 rrrr mmqmqmq

011211 rrrr mmqmqmq

01222111 rrrrrr mmqmqmq

As has been introduced above, qis can be obtained by solving :

pis can be immediately obtained from:

000 pm

01101 pmmq

0111201 rrrr pmmqmq

(3)

rr

rr

r

r

sqsq

spspp

sQ

sPsH

1

11101

1)(

)()(ˆ

Ok ?

(4)


Any other possible way? Yes!: Parial Fractions Decomposition:

rr

rr

r

r

sqsq

spspp

sQ

sPsH

1

11101

1)(

)()(ˆ

)(ˆ sH is inaccurate at high frequency due to floating point overflow.

r

r

rr

rr

as

k

as

k

as

k

sqsq

spsppsH

ˆ

ˆ

ˆ

ˆ

ˆ

ˆ

1)(ˆ

2

2

1

1

11

1110

If we know (approximate poles) and (approximate residues) then is known and is easily computed.

raaa ˆ,,ˆ,ˆ 21

rkkkˆ,,ˆ,ˆ 21 )(

ˆ sH

How to compute the poles and the residues?

and we have known how to compute qis !

raaa ˆ,,ˆ,ˆ 21 are nothing but the roots of r

r sqsq 11


What is left? computation of the residues:

11

22

21

11

1 )ˆ

1(ˆ

ˆ)

ˆ1(

ˆ

ˆ)

ˆ1(

ˆ

ˆ

rr

r

a

s

a

k

a

s

a

k

a

s

a

k

From Pade approximation:

1212

22

22

121210

1122

111 ][)ˆ(

ˆ)ˆ(ˆ)ˆ(ˆ

rr

rr

rr

rrrr

sese

smsmsmmaskaskask

)ˆˆ

1(ˆ

ˆ)

ˆˆ1(

ˆ

ˆ

2

2

21

2

11

1 rrr

r

a

s

a

s

a

k

a

s

a

s

a

k

rkkkˆ,,ˆ,ˆ 21


0

2

2

1

1

ˆ

ˆ

ˆ

ˆ

ˆ

ˆm

a

k

a

k

a

k

r

r

1221

2

21

1

ˆ

ˆ

ˆ

ˆ

ˆ

ˆm

a

k

a

k

a

k

r

r

1

2

2

1

1

ˆ

ˆ

ˆ

ˆ

ˆ

ˆ

rrr

r

rrm

a

k

a

k

a

k

rr

rr

r

r

aaa

aaa

aaa

ˆˆˆ

ˆˆˆ

ˆˆˆ

21

222

21

112

11

1

1

0

2

1

ˆ

ˆ

ˆ

rrm

m

m

k

k

k

the residues can be obtained by solving the above euqations!

r

r

as

k

as

k

as

ksH

ˆ

ˆ

ˆ

ˆ

ˆ

ˆ)(ˆ

2

2

1

1

Once raaa ˆ,,ˆ,ˆ 21 and rkkkˆ,,ˆ,ˆ 21 are computed, we have:

(5)

• Why not approximation by *truncated* Taylor expansion?


• The Padé approximant often gives better approximation of the function than truncating its Taylor series, and it may still work where the Taylor series does not converge.

• Round off error or overflow of in truncated Taylor expansion is avoided .

• Poles and residues of H(s) can be computed more easily by Pade approximation. • H(s) itself is a rational function.


Implementation of AWE:

2. Compute the roots of the polynomial:

1. Solve (3) to get qis.

rr sqsq 11

3. Solve (5) to get rkkkˆ,,ˆ,ˆ 21

the roots: raaa ˆ,,ˆ,ˆ 21

r

r

as

k

as

k

as

ksH

ˆ

ˆ

ˆ

ˆ

ˆ

ˆ)(ˆ

2

2

1

1

4. Form the reduced (simpler) transfer function:

Much Easier to be computed than H(s)!

In MATLAB step 2. and 3. are implemented in the function: residue.m


In MATLAB:


2. Get pis from (4).

3. Use *residue(p,q)* in matlab to get the poles and residues:

rkkkˆ,,ˆ,ˆ 21

4. Form the Pade approximation (approximate transfer function):

r

r

as

k

as

k

as

ksH

ˆ

ˆ

ˆ

ˆ

ˆ

ˆ)(ˆ

2

2

1

1

;ˆ,,ˆ,ˆ 21 raaa


How to compute the output response y(t) in time domain from ?

Definition of transfer function:

)(ˆ sH

)(/)()( sUsYsH

If the input is the unit impulse function:

0,0

0,1)()(

t

tttu

1

t

)()(/)()( sYsUsYsH

then

0

1)()( dtetsU st

Therefore with impulse input:

j

j

ststj

jdsesH

jdsesY

jty

)(

2

1)(

2

1)(


r

r

as

k

as

k

as

ksH

ˆ

ˆ

ˆ

ˆ

ˆ

ˆ)(ˆ

2

2

1

1

We get:

Replace with: )(ˆ sH

We obtain:

j

j

ststj

jdsesH

jdsesH

jty

)(ˆ

2

1)(

2

1)(

r

i

tai

iekty1

ˆˆ)(

Because:

ta

i

i

istj

ji

i iekas

kLdse

as

k

j

ˆ1 ˆ)ˆ

ˆ(

ˆ

ˆ

2

1

Impulse output response in time domain

Numerical instability of AWE

Eigenvalue and eigenvectors of a matrix A

5.05.0

011A

1

01a

1

2b11aA

bA1

An eigenvector either does not change direction by A or is reversed by A.

1

02a

21aA

11aA

21aA

bA1

http://en.wikipedia.org/wiki/File:Beam_mode_1.gif


Eigenvalue and eigenvectors of a matrix :

niA iii ,2,1,

nii ,2,1, are eigenvectors of A.

nii ,2,1, are eigenvalues of A.

Applications to Engineering: • Vibration of a beam. • Stability of a system. •

nnRA


011110 rrrr mmqmqmq


011211 rrrr mmqmqmq

01222111 rrrrrr mmqmqmq

(3)

221

21

110

rrr

r

r

mmm

mmm

mmm

12

1

1

1

r

r

r

r

r

m

m

m

q

q

q

bA

bA

bA

bA

b

c

r

r

T

~

~~

~~

~~

~

12

1

2

1

12

2

1

0

r

r

m

m

m

m

m

bAbEAA 11~

,~

12

2

1

0

r

r

m

m

m

m

m

bA

bA

bA

bA

b

r

r

~~

~~

~~

~~

~

12

2

1


)~

( 1

A

bAx~~

1

bAx~~2

2

bAx~~6

6 bAx

~~77

bAx~~8

8

bA~~9

bA~~10

, i=1,2,... run parallel to soon !

Usually, after i=8, all will in the same direction with .

bAx ii~~

bAx ii~


nncccb 2211~

n

m

n

mmm AcAcAcbA ~~~~~

2211

nmnn

mm ccc 222111

n

m

nn

m

m

c

c

c

cc

11

2

1

2

1

2111

m

1111 )(~~

mm cbA

)( 21 n

121 to~~

changeserror off-round close,not are and If bAm


1

~~bAm

1

bAm~~

1

bAm~~

0

0


Notice: bAcm iTi~~

n

j

n

j j

j

j

jjn

j j

jj

as

k

s

bl

s

blsH

1 11

~~~~

)(

However, the original transfer function contains the information of all the eigenvalues:

.~

, ofn informatio econtain thonly ~~

means this

, with line same on thealmost be will~~

all ,8 when examples,many For

1111

1

AbA

bAi

i

i

. ofn informatio econtain thonly also 8, Therefore 1imi

moments! more usingby improved becannot

)(ˆ ofaccuracy they,numericall ),(ˆ accurate more tolead willand moments, more

match will more compute tomoments more employing ly,heoreticalAlthough t

sHsH

qi

Conclusion:


L. T. Pillage, R. A. Rohrer, Asymptotic Waveform Evaluation for Timing Analysis, IEEE Transactions on computer-aided design, Vol. 9, No. 4, 1990.

)(ˆ

)()(/

tCVzy

tBuWtAVzWdtEVdzW TTT

Vzx

)(/)()(/)()( susCxsusysH

i

i

sssBsAc

BAEsIEAEsssc

BAEsEssEc

BAsEcsH

)()(~

)(~

)()))(((

)(

)()(

0

0

00

1

0

11

00

1

00

1

Implicit moment-matching(Pade, Pade-type approximation)

)(

)()(

tCxy

tButAxdt

dxE

Recall that for projection based MOR:

By definition:

Taylor expansion at s0:

BAEssBEAEssA 1001

00 )()(~

,)()(~

where

Definition ( moments are defined for any expansion point )

function. transfer theof

moments thecalled are ,...,0 system), SISOfor )(~

)(~

( )(~

)(~

)( 00000 isbsAcsBsACsMii

i

0s


Recall that in AWE method, the moments

are computed explicitly. It is numerically unstable.

12 ,...,0 ,)0(~

)0(~

ribAcM ii

:ofeither composed being asseen becan ,...,0 ,)(~

)(~

that Observe 00 tsbsAcMt

t

(5) ,...1,0 ),(~

and

(4) ...1,0 ),(~

)(~

0

00

jsAc

isbsA

j

i

Instead of explicitly computing the terms in (4)(5) or in (6)(7) as is done by AWE method, one can compute the basis of

ofor

(7) and

(6) ...1,0 ),(~

)(~

00

c

tsbsAt

(9) }))(~

(,,)(~

,span{)W~

( range

(8) })(~

)(~

,),(~

)(~

),(~

span{range(V)

1

00

00

1

000

TpTTT

p

csAcsAc

sbsAsbsAsb

Implicit moment-matching(Pade approximation)

How to compute W, V? [Feldmann, Freund ’95]

IVW T ~

(9) )),(~

(}))(~

(,,)(~

,span{)~

( range

(8) ))(~

,)(~

(})(~

)(~

,),(~

)(~

),(~

span{)range(

0

1

00

0000

1

000

TT

p

TpTTT

p

p

csAKcsAcsAcW

sbsAKsbsAsbsAsbV

W, V span two Krylov subspaces, so that they can be simultaneously computed by (band)Lanczos algorithm, such that .

Recall that

TT

T

T

cV

bEsAW

EVEsAWT

10

1

0

)(~

)(~

are algorithm Lanczos of outputs The

Implicit moment-matching (Pade approximation)

Applying Petrov-Galerkin projection (using ) to the transformed system

(10) ),(

),()()()()( 101

0

1

0

tcxy

tbuEsAtAxEsAdt

dxEEsA

One gets the reduced order model (ROM)

(11) ),(

),()(~

)()(~

)(~ 1

0

1

0

1

0

tcVzy

tbuEsAWtAVzEsAWdt

dzEVEsAW TTT

By studying the ROM in (11) [Freund ’03]

. and ,~

)( with system original

the toprojectionGalerkin -Petrov applying toequivalent is (11)-(10) that seecan one

0 VWEsAWT

VW ,~

Implicit moment-matching (Pade approximation)

.)( ofion approximat Pade a is )(ˆ Therefore,

.12,...,1,0 ),(ˆ)(

i.e. , )( of moments 2first match the (11)in ROM theof

function transfer theof moments 2first the(10),in system SISO afor then

,~

satisfy and (8)(9),in subspace theof basis theare V ,W~

If

00

sHsH

pisMsM

sHp

p

IVW

ii

T

Theorem 1 [Feldmann, Freund ’95]

Drawbacks of Lanczos method of computing the projection matrices: • The ROM computed by W, V may be unstable, there are eigenvalues with positive real parts. • The Lanczos method does not maintain precise biorthogonality given limited numerical precision.

Actually the ROM in (11) can be implicitly derived using the outputs of the Lanczos

algorithm[Freund ’03] :

(12) ),(

),()()( 0

tzy

tutzTsIdt

dzT

T

outputs.

and inputs are There .or in columns theofnumber thely,equivalent

or model, reduced theoforder theis Here .)( oft approximan Padematrix

a isit and ,)( of moments r/n r/nfirst least theat matches )(ˆthen

,~

satisfy and ,algorithm) Lanczosby generatedy necessarilnot are(they

(8)(9)in subspace theof basisany are V ,W~

if shown that isit '00], [FreundIn

IO

OI

T

nnWV

rsH

sHsH

IVW

Number of moments matched for MIMO systems

Implicit moment-matching(Pade approximation)

It is immediately seen from the above statement that for SISO systems, there are

at least 2r moments matched. Moreover, for SISO systems, r = p. Therefore, there

is no contradiction with Theorem 1.

Implicit moment-matching(Pade-type approximation)

Pade-type approximation [Odabasioglu,et.al ’97, Freund ‘03]

A passive (therefore stable) ROM can be obtained by using W=V.

))(~

,)(~

(})(~

)(~

,),(~

)(~

),(~

span{)range( 00001

000 sbsAKsbsAsbsAsbV pp

algorithm. Arnoldiby computed becan it Therefore . satisfies IVVV T

.1,...,1,0 ),(ˆ)(

,)(ˆby matched are )( of moments first the(10),in system SISO afor then

(8),in subspace Krylov theof basis orthogonalan constitute in columns theIf

00 pisMsM

sHsHp

V

ii

Theorem 2 [Odabasioglu, et.al ’97, Freund ‘03]

).( ofion approximat type-Pade a is (s)ˆ sHH

Recall:

0

00

0

000 )(~

)(~

)()())((~

)(~

)()()()(i

i

i

i

ii sbsAsgsusssbsAsxsusHscx

1

0

00 )(~

)(~

)()(p

i

i

i sbsAsgsx

1

0

00 )(~

)(~

)()(p

i

i

i sbsAsgsxVzx


?n rather tha , usingWhy WzxVzx

i

i

sssbsAcbAsEcsH )()(~

)(~

)()( 00

00

1

))(~

,)(~

(} )(~

)(~

,),(~

)(~

),(~

span{)range( 00100000 sbsAKsbsAsbsAsbV pp

The ROM is obtained by Galerkin projection onto the original system.

(11) )(ˆ

)()(/

tcVzy

tbuVtAVzVdtEVdzV TTT

nonzero.or zero aschosen becan :point expansion The 00 sss


Expansion point

preferred. is point expansion

nonzero a then zero, fromaway far is rangefrequency ginterestin theIf

0s

Implicit moment-matching(rational interpolation)

• A single expansion point is used in the method based on Pade approximation.

Multiple expansion points are used in rational interpolation method.

• Rational interpolation views computing the transfer function from the viewpoint

of solving linear systems.

b

T

c xAsExbAsEAsEAsEcbAsEcsH )())(()()()(111

where

bxAsEcxAsE bT

c

T )( ,)(

Applying a preconditioner to each of the linear systems,

bEsAxAsEEsAcEsAxAsEEsA bTT

c

TT 1

0

1

000 )()()( ,)()()(

If using Krylov-subspace iterative methods to solve the preconditioned linear

systems, we have (this is the property of Krylov-subpace iterative methods, e.g.

CG, GMERS, MINRES. etc..)


))(),()((ˆ 101

0 bEsAAsEEsAKxx qbb

))(,)()((ˆ 00TTTT

qcc cEsAAsEEsAKxx

))(,)(())(),()(( 101

0

1

0

1

0 bEsAEEsAKbEsAAsEEsAK qq

EEsAssIsEAEsA 1001

0 ))(()()( Since

).,()),((

, nonzero and g vector G,matrix any For

gGKgIGK qq

))(,)(())(,)()(( 0000TTTT

q

TTT

q cEsAEEsAKcEsAAsEEsAK

Lemma 2.2 [Grimme ’97] Krylov subspace shift-invariance


(12) ))(,)(()range(

such that , Compute

1

0

1

0 bEsAEEsAKV

V

p

(13) ))(,)(()range(

such that , Compute

00

TTTT

p cEsAEEsAKW

W

Then cccbbb WzxxVzxx ˆ ,ˆ

b

TTT

cb

TT

cb

T

c zAVWEVsWzVzAsEWzxAsExsHsH )()(ˆ)(ˆ)(ˆ)(

Therefore the reduced matrices are: ,ˆ ,ˆ AVWAEVWE TT

.12,,1,0 ),(ˆ)( i.e.),(ˆby matched are )( of moments 2then

(12)(13),satisfy , matrices projection theif systems, MIMO and SISOboth For

00 pisMsMsHsHp

WV

ii

Theorem ([Grimme ’97])


Instead of using a single expansion point, multiple expansion points are used in

rational interpolation method, such that

(14) ))(,)(()range( 11

0

bEsAEEsAKV iip

k

ii

(15) ))(,)(()range(0

TT

i

TT

ip

k

i

cEsAEEsAKWi

.,,0;12,,1,0 ,)(ˆ)(

i.e.,point expansion each at )(ˆby matched are )( of moments 2then

(14)(15),satisfy , matrices projection theif systems, MIMO and SISOboth For

kjpisMsM

ssHsHp

WV

jjiji

jj

Theorem ([Grimme ’97])

. that requirednot isit

method,ion interpolat rational theofproperty matching-moment For the

IVW T

Remark


Computation of V, W in (14)(15)

• Rational Arnoldi method or rational Lanczos method in [Grimme ’97]

• Repeated modified Gram-Schmidt algorithm (Repeated Arnoldi algorithm).

How to decide the expansion points?

• Some heuristic methods

• Using error estimation and a greedy algorithm.

• Locally optimal algorithm: IRKA.



Using error estimation and a greedy algorithm

Error estimation, e.g. :

)(/||||||||||ˆ|| :ˆ and between Error

)(/||||||)(ˆ)(|| :ˆ and between Error

||)(ˆ)(|||||| Residual

min222

min22

22

AsErryyyy

AsErsxsxxx

sxAsEBr

dupr

2min22

1

2

1

2

11

2

1

2

)(/||||||||||)(||

||ˆ)(()(||

||ˆ)()()(||||ˆ)(||||ˆ||

AsErrAsE

xAsEBAsE

xAsEAsEBAsExBAsExx

)(s

)( , TTTdupr AsECrrr (Proof in [Feng, Benner, Antoulas ’14])



A greedy algorithm: Selection of expansion points

WHILEEND

);ˆ(

);(maxargˆ

];,[ ];,[

))(,)(()range( );)(,)(()range(

;ˆ

;1

WHILE

of samples ofset large a :

;1 ;ˆ :pointexpansion Initial

11

0

s

ss

WWWVVV

CAEsEAEsKWBAEsEAEsKV

ss

ii

s

iss

trains

ii

TT

i

TT

ipiiipi

i

tol

train



• Locally optimal algorithm: IRKA for both SISO and MIMO system:

r

T

rr

T

rr

T

r

rr

T

rr

r

T

r

TT

r

rrrr

rr

TT

rii

iiii

r

T

rr

T

r

j

old

jj

rr

T

rrr

T

r

TT

r

rrrr

rr

i

CVCBWBAVWAEVWE

WVWW

CCAECCAEW

BBAIBBAEVWV

CCCBBBYCCYBB

yyYri

riyyAE

AVWAEVWE

VVWWCCAECCAEW

BBAEBBAEVWV

CCBB

ri

ˆ,ˆ,ˆˆ .4

)( (f)

.}~

)(,,~

)span{()Ran(

},~

)( , ,~

)span{()Ran( so and Update(e)

)~

,,~

(~

),~

,~

(~

,ˆ~

,ˆ~

(d)

),,( ;,,1for Assign (c)

,,1,)ˆˆ( Solve (b)

ˆ,ˆ (a)

tol)max( WHILE.3

)( and .}~

)(,,~

)span{()Ran(

},~

)( , ,~

)span{()Ran( that so and Choose 2.

.~

,~

,~

,~

directions initial Choose tol.

afix n,conjugatiounder closed,,,1for , ofselection initialan Make 1.

(IRKA) Algorithm Krylov Rational IterativeAn

1

1-

1

1-

1

1-

r1

1-

1

11

1

r,1,j

11-

1

1-

1

1-

r1

1-

1

11



• Locally optimal algorithm: IRKA for both SISO and MIMO system:

r

T

rr

T

rr

T

r

rr

T

rr

r

T

r

TT

r

rrrr

rr

T

rii

iii

TT

iiiii

r

T

rr

T

r

j

old

jj

rr

T

rrr

T

r

TT

r

rrrr

rr

i

CVCBWBAVWAEVWE

WVWW

CCAECCAEW

BBAIBBAEVWV

CCCBBBYCCYBB

yyYri

riyyAEyyAE

AVWAEVWE

VVWWCCAECCAEW

BBAEBBAEVWV

CCBB

ri

ˆ,ˆ,ˆˆ .4

)( (f)

.}~

)(,,~

)span{()Ran(

},~

)( , ,~

)span{()Ran( so and Update(e)

)~

,,~

(~

),~

,~

(~

,ˆ~

,~ˆ~ (d)

),,( ;,,1for Assign (c)

,,1~~)ˆˆ( ,)ˆˆ( Solve (b)

ˆ,ˆ (a)

tol)max( WHILE.3

)( and .}~

)(,,~

)span{()Ran(

},~

)( , ,~

)span{()Ran( that so and Choose 2.

.~

,~

,~

,~

directions initial Choose tol.

afix n,conjugatiounder closed,,,1for , ofselection initialan Make 1.

(IRKA) Algorithm Krylov Rational IterativeAn

1

1-

1

1-

1

1-

r1

1-

1

11

1

r,1,j

11-

1

1-

1

1-

r1

1-

1

11


}~

)(,],~

)Im[(],~

)Re[(,,~

)span{(

}~

)(,,~

)(,~

)(,,~

)span{(

thatSo

]}.~

)Im[(],~

)span{Re[(}~

)(,~

)span{(

Therefore

parts.imaginary and real same thehave ~

)( and ~

)( ,)()( Since

)()~

()(~

)(

)~

()(~

)(

have we, riablecomplex vaany For

matrices as taken becan , then n,conjugatiounder closed are Residual

1111

1

1*1*11

1

11*1*1

*1*1**

*

0

*11*1*

0

111

bAEbAEbAEbAE

bAEbAEbAEbAE

bAEbAEbAEbAE

bAEbAE

bAEAbAE

bAEAbAE

.why? realVW

rii

rii

iiii

ii

k

i

k

i

k

i

k

k

i

k

i

k

k

i

i

i

. toscorrespond and

in is then r,eigenvecto ingcorrespond theis in complex, is if

)~

,,~

(~

),~

,~

(~

,ˆ~

,~ˆ~ (d)In 11

i

iii

rr

T

YyYy

CCCBBBYCCYBB


• Locally optimal algorithm: IRKA for SISO system

Upon convergence, Algorithm IRKA leads to:

.,,1for )ˆ()ˆ(ˆ and )ˆ()ˆ(ˆ '' riHHHH iiii

.,,1for )ˆ()ˆ(ˆ and )ˆ()ˆ(ˆ

:,,1,ˆat derivativefirst its and

)(both esinterpolat )(ˆThen .,,1,ˆat poles simple has )(ˆ that suppose and

||~

||min||ˆ||

problem

reduction model optimal for the dimension ofminimizer local a beˆ)ˆ(ˆ)(ˆ

let ,)()( system SISO stable aGiven '08] alet [Gugercin 3.4. Theorem

''

)~

dim(

2

1

1

2

stable :~

2

riHHHH

ri

sHsHrisH

HHHH

HrbAsIcsH

bAsIcsH

iiii

i

i

HrH

H

H

Therefore IRKA obtains a reduced model satisfies the local optimal necessary

conditions in Theorem 3.4 in [Gugercin et al. ’08].


References

Pade- and Pade-type approximation;

[Freund ‘00] R. W. Freund, Krylov-subspace methods for reduced-order modeling in circuit simulation,

Journal of Computational and Applied Mathematics, 123: 395-421, 2000.

[Freund ‘03] R. W. Freund, Model reduction methods based on Krylov subspaces, Acta Numerica, 12:

267-319, 2003.

[Feldmann, Freund ‘95] P. Feldmann and R. W. Freund, Efficient linear circuit analysis by Pade

approximation via the Lanczos process. IEEE Transactions on Computer-aided design of Integrated

Circuits and Systems. 14(5), 1995.

[Odabasioglu et al ‘97] Odabasioglu, M. Celik, L. T. Pileggi, PRIMA: passive reduced-order interconnect

macromodeling algorithm. IEEE/ACM ICCAD, 58-65, 1997.

Rational interpolation:

[Grimme ‘97] E. J. Grimme, Krylov projection methods for model reduction, PhD thesis, University of

Illinois at Urbana-Champaign, 1994.

IRKA:

[Gugercin et al ‘08] S. Gugercin, A. C. Antoulas, C. Beattie, H_2 Model reduction for large-scale linear

dynamical systems. SIAM J. Matrix Analysis, 30(2): 609-638, 2008.

lihong feng - max planck society...moment-matching method lihong feng outline • orthogonality of...

Documents