krylov subspace methods in model order reduction

KrylovMethods in

Introduction

Moments &MarkovParameters

KrylovSubspace

MomentMatching

Issues withKrylovMethods

Orthogonalization

StoppingCriteria

Krylov Subspace Methods inModel Order Reduction

Mohammad Umar Rehman

PhD Candidate, EE Department, IIT Delhiumar.ee.iitd@gmail.com

March 8, 2016

1 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

Outline

1 Introduction

2 Moments & Markov Parameters

3 Krylov Subspace

4 Moment Matching

5 Issues with Krylov MethodsOrthogonalizationStopping Criteria

2 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

IntroductionModel Reduction Problem Revisited

Given a MIMO state space model

Ex = Ax+Bu

y = Cx(1)

where, E,A ∈ Rn×n,B ∈ Rn×m,C ∈ Rp×n,u ∈ Rm,y ∈ Rp,x ∈ Rn and n is sufficiently large.

It is required to obtain the following reduced order model

Erz = Arz+Bru

y = Crz(2)

where, Er,Ar ∈ Rq×q,Br ∈ Rq×m,Cr ∈ Rp×q,u ∈ Rm,y ∈ Rp, z ∈ Rq q << nEr = WTEV,Ar = WTAV,Br = WTB,Cr

T = CTVW,V are suitable Krylov subspace based projection matrices.

3 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

Ex = Ax+Bu

y = Cx(1)

where, E,A ∈ Rn×n,B ∈ Rn×m,C ∈ Rp×n,u ∈ Rm,y ∈ Rp,x ∈ Rn and n is sufficiently large.It is required to obtain the following reduced order model

Erz = Arz+Bru

y = Crz(2)

T = CTV

W,V are suitable Krylov subspace based projection matrices.

3 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

Ex = Ax+Bu

y = Cx(1)

where, E,A ∈ Rn×n,B ∈ Rn×m,C ∈ Rp×n,u ∈ Rm,y ∈ Rp,x ∈ Rn and n is sufficiently large.It is required to obtain the following reduced order model

Erz = Arz+Bru

y = Crz(2)

T = CTVW,V are suitable Krylov subspace based projection matrices.

3 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

Moments and Markov Parameters

The transfer function of the system in (1) is

G(s) = C(sE−A)−1B

By assuming that A is nonsingular, the Taylor series of this transferfunction around zero is:

G(s) = −CA−1B−C(A−1E)A−1Bs− · · · −C(A−1E)iA−1Bsi − · · ·

Coefficients of powers of s are known as momentsi-th moment:

M0i = C(A−1E)iA−1B, i = 0, 1, . . .

M0i = −

diG(s)

∣∣∣s=0

is the value of subsequent derivatives of the transfer function G(s) at thepoint s = 0

4 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

Contd...

A different series in terms of negative powers of s is obtained whenexpanded about s→∞

G(s) = CE−1Bs−1+C(E−1A)E−1Bs−2+· · ·+C(E−1A)iE−1Bs−i+· · ·

and the coefficients are known as Markov parameters.

Model reduction is achieved by the means of matching of Moments(Markov parameters)Explicit moment matching becomes numerically cumbersome for largesystem order

Go for implicit moment matching: Krylov subspace based Projection

5 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

Contd...

A different series in terms of negative powers of s is obtained whenexpanded about s→∞

G(s) = CE−1Bs−1+C(E−1A)E−1Bs−2+· · ·+C(E−1A)iE−1Bs−i+· · ·

and the coefficients are known as Markov parameters.

Model reduction is achieved by the means of matching of Moments(Markov parameters)Explicit moment matching becomes numerically cumbersome for largesystem orderGo for implicit moment matching: Krylov subspace based Projection

5 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

Remarks 1

Asymptotic Waveform Evaluation (AWE) method is basedon explicit moment matching

Matching at s = 0 is known as Pade Approximation, andsteady state response (low frequency) is reflected in thereduced order model.

Matching at s→∞ is known as Partial Realization, andthe reduced order model is a good approximation of theHF response.

Matching at s = s0, i. e. at some arbitrary value of s isknown as Rational Interpolation and is aimed atapproximating system response at specific frequency bandof interest.

6 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

Defining Krylov Subspace

Kq(A,b) = span{b,Ab, . . . ,Aq−1b},

A ∈ Rn×n and b ∈ Rn is called the starting vector. q is some givenpositive integer called index of the Krylov sequence.

The vectors b,Ab, . . . , constructing the subspace are called basicvectors.The Krylov subspace is also known as controllability subspace incontrol community.For each state space, there are two Krylov subspaces that are dual toeach other, input Krylov subspace and output Krylov subspace.Either or both of subspaces can be used as projection matrices formodel reduction.The respective method is then called One-Sided/Two-sided

7 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

A ∈ Rn×n and b ∈ Rn is called the starting vector. q is some givenpositive integer called index of the Krylov sequence.The vectors b,Ab, . . . , constructing the subspace are called basicvectors.

The Krylov subspace is also known as controllability subspace incontrol community.For each state space, there are two Krylov subspaces that are dual toeach other, input Krylov subspace and output Krylov subspace.Either or both of subspaces can be used as projection matrices formodel reduction.The respective method is then called One-Sided/Two-sided

7 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

A ∈ Rn×n and b ∈ Rn is called the starting vector. q is some givenpositive integer called index of the Krylov sequence.The vectors b,Ab, . . . , constructing the subspace are called basicvectors.The Krylov subspace is also known as controllability subspace incontrol community.

For each state space, there are two Krylov subspaces that are dual toeach other, input Krylov subspace and output Krylov subspace.Either or both of subspaces can be used as projection matrices formodel reduction.The respective method is then called One-Sided/Two-sided

7 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

A ∈ Rn×n and b ∈ Rn is called the starting vector. q is some givenpositive integer called index of the Krylov sequence.The vectors b,Ab, . . . , constructing the subspace are called basicvectors.The Krylov subspace is also known as controllability subspace incontrol community.For each state space, there are two Krylov subspaces that are dual toeach other, input Krylov subspace and output Krylov subspace.Either or both of subspaces can be used as projection matrices formodel reduction.

The respective method is then called One-Sided/Two-sided

7 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

A ∈ Rn×n and b ∈ Rn is called the starting vector. q is some givenpositive integer called index of the Krylov sequence.The vectors b,Ab, . . . , constructing the subspace are called basicvectors.The Krylov subspace is also known as controllability subspace incontrol community.For each state space, there are two Krylov subspaces that are dual toeach other, input Krylov subspace and output Krylov subspace.Either or both of subspaces can be used as projection matrices formodel reduction.The respective method is then called One-Sided/Two-sided

7 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

Input and Output Krylov Subspaces

Input Krylov subspace

(A−1E,A−1b

)= span

{A−1b, . . . ,

(A−1E

)q1−1A−1b}Output Krylov Subspace

(A−TET,A−Tc

)= span

{A−Tc, . . . ,

(A−TET

)q2−1A−Tc

V is any basis of Input Krylov SubspaceW is any basis of Output Krylov Subspace

8 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

Input and Output Krylov Subspaces

Input Krylov subspace

(A−1E,A−1b

)= span

{A−1b, . . . ,

(A−1E

)q1−1A−1b}Output Krylov Subspace

(A−TET,A−Tc

)= span

{A−Tc, . . . ,

(A−TET

)q2−1A−Tc

}V is any basis of Input Krylov SubspaceW is any basis of Output Krylov Subspace

8 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

Moment Matching: SISO

Theorem If the matrix V used in (2), is a basis of Krylovsubspace Kq1

(A−1E,A−1b

)with rank q and matrix W is

chosen such that the matrix Ar is nonsingular, then the first qmoments (around zero) of the original and reduced ordersystems match.

9 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

Proof: The zero-th moment of the reduced system is

mr0 = cTr A−1r br = cTV

)−1WTb

The vector A−1b is in the Krylov subspace and it can bewritten as a linear combination of the columns of the matrix V,

∃r0 ∈ Rq : A−1b = Vr0

Therefore,(WTAV

)−1WTb =

)−1WT

(AA−1

=(WTAV

)−1WTAVr0

10 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

)−1WTb

∃r0 ∈ Rq : A−1b = Vr0

Therefore,(WTAV

)−1WTb =

)−1WT

(AA−1

=(WTAV

)−1WTAVr0

10 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

)−1WTb

∃r0 ∈ Rq : A−1b = Vr0

Therefore,(WTAV

)−1WTb =

)−1WT

(AA−1

=(WTAV

)−1WTAVr0

10 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

Contd...

With this, mr0 becomes

mr0 = cTV(WTAV

)−1WTb = cTVr0 = cTA−1b = m0

For the next moment (first moment) consider the following result:(WTAV

)−1WTEV

)−1 (WTb

)=(WTAV

)−1WTEVr0

=(WTAV

)−1WTEA−1b

and the fact that A−1EA−1b is also in the Krylov subspace can bewritten as A−1EA−1b = Vr1

11 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

Contd...

Thus,(WTAV

)−1WT

(AA−1

)EA−1b =

)−1WTAVr1

mr1 = cTV(WTAV

)−1WTEV

)−1WTb

= cTVr1 = cTA−1EA−1b

12 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

Contd...

Thus,(WTAV

)−1WT

(AA−1

)EA−1b =

)−1WTAVr1

mr1 = cTV(WTAV

)−1WTEV

)−1WTb

12 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

Contd...

Thus,(WTAV

)−1WT

(AA−1

)EA−1b =

)−1WTAVr1

mr1 = cTV(WTAV

)−1WTEV

)−1WTb

12 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

Contd...

Thus,(WTAV

)−1WT

(AA−1

)EA−1b =

)−1WTAVr1

mr1 = cTV(WTAV

)−1WTEV

)−1WTb

12 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

Contd...

Thus,(WTAV

)−1WT

(AA−1

)EA−1b =

)−1WTAVr1

mr1 = cTV(WTAV

)−1WTEV

)−1WTb

12 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

Remarks 2

For the second moment, the results of first moment canbe used and the fact that

(A−1E

)2A−1b can be written

as a linear combination of columns of matrix V

The proof can be continued by repeating these steps(Induction) until mr(q−1) = m(q−1) i.e. q moments match.

The method discussed above was one-sided as we did notgo for computing W. Usually, W = V is chosen

In two-sided method W is chosen to be the basis of outputKrylov subspace, then 2q moments can be matched.

Proof is similar for matching Markov parameters and theMIMO case [3,4].

13 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

Issues with Krylov Methods

Major issues with Krylov Subspace based MOR Methods:

1 Orthogonalization

2 Stopping Point of Iterative Scheme

14 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

Orthogonalization

The Krylov vectors are known to lose independence readilyand tend to align towards the dominant vector, even formoderate values of n and q.

The remedy lies in constructing an orthogonal basis usingGram-Schmidt process.

However, classical GS is also known to be unstable

Go for Modified GS methods —Arnoldi (Unsymmetric A) / Lanczos (Symmetric A)

15 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

Orthogonalization

15 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

Orthogonalization

15 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

Orthogonalization

Go for Modified GS methods —

Arnoldi (Unsymmetric A) / Lanczos (Symmetric A)

15 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

Orthogonalization

Go for Modified GS methods —Arnoldi (Unsymmetric A)

/ Lanczos (Symmetric A)

15 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

Orthogonalization

15 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

Arnoldi AlgorithmUsing Modified Gram-Schmidt Orthogonalization

Algorithm 1 Arnoldi

1: Start: Choose initial starting vector b,v = b‖b‖

2: Calculate the next vector: vi = Avi−1Orthogonalization:

3: for j = 1 to i− 1 do4: h = vT

i vj , vi = vi − hvj

Normalization:5: i-th column of V is vi =

vi‖vi‖ stop if vi = 0

6: end for

Output of Arnoldi Iteration:

1 Orthonormal Projection matrix V,

2 Hessenberg Matrix H = VTAV

16 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

Stopping Criterion

When to stop the iterative scheme?is another question to be answered

This also decides the size of the ROM

TU-M: Singular values based stopping criterion.1

IIT-D: A more efficient criterion based on a index knownas CNRI2 is proposed.3

1B. Salimbahrami and Lohmann, B., “Stopping Criterion in OrderReduction of Large Scale Systems Using Krylov Subspace Methods”, Proc.Appl. Math. Mech., 4: 682–683, 2004.

2Coefficent of Numerical Rank Improvement3M. A. Bazaz, M. Nabi and S. Janardhanan. “A stopping criterion for

Krylov-subspace based model order reduction techniques”. Proc. Int.Conf. Modelling, Identification & Control (ICMIC), pp. 921 - 925, 2012

17 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

Comparison with Balanced Truncation

Parameter BT Krylov

No. of Flops O(n3)

O(q2n)

Numerical Reliability for large n No Yes

Accuracy of the reduced system More Accurate Less Accurate

Range of Applicability ∼ 103 ∼ 104 or higher

Stability Preservation Yes No

Iterative Method No Yes

Reliable Stopping Criterion Yes No*

18 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

Selected References

1 A. C. Antoulas, Approximation of Large Scale DynamicalSystems, SIAM, 2005.

2 Behnam Salimbehrami, Structure Preserving Order Reduction ofLarge Scale Second Order Models, PhD Thesis, TU Munich,2005.

3 Rudy Eid, Time Domain Model Reduction By Moment Matching,PhD Thesis, TU Munich, 2008.

4 B. Salimbehrami, Boris Lohmann, Krylov Subspace Methods inLinear Model Order Reduction: Introduction and InvarianceProperties. Scientific Report, Univ. of Bremen, 2002.

5 Zhaojun Bai, Krylov subspace techniques for reduced-ordermodeling of large-scale dynamical systems, Applied NumericalMathematics, 43 (2002), pp 9-44.

19 / 20

KrylovMethods in

Introduction

KrylovSubspace

MomentMatching

Orthogonalization

StoppingCriteria

Thanks!,

20 / 20

krylov subspace methods in model order reduction

Engineering

rational krylov subspace methods for phi-functions in

krylov-subspace-based order reduction methods applied to

structure preserving model order reduction via krylov...

krylov subspace methods for projected lyapunov equations...

krylov subspace methods for nonlinear model reduction ·...

krylov subspace methods for model reduction of mimo...

tutorial: krylov subspace methods - people at dtu compute

model reduction of nonlinear circuit equations - birs.ca ·...

krylov subspace methods for functions of...krylov subspace...

computing approximate (block) rational ......krylov...

krylov subspace methods for eigenvalue problems · krylov...

krylov subspace methods for simultaneous primal …krylov...

krylov subspace methods on supercomputers...krylov subspace...

krylov subspace methods for computing hydrodynamic...

structure-preserving krylov subspace methods for...

communication-avoiding krylov subspace...

model reduction of linear time-varying systems using...

stability of krylov subspace spectral methods · stability...

deﬂation and augmentation techniques in krylov...

a short course on: preconditioned krylov subspace...