on efficiently updating singular value decomposition based ... fileon efficiently updating singular...

On Efficiently Updating Singular Value Decomposition Based Reduced Order Models

Ralf Zimmermann

GAMM Workshop Applied and Numerical Linear Algebra with Special Emphasis on Model Reduction

Bremen, Sep. 22.-23. 2011

2

1. Input: CFD snaphots

Flow solutions at different flow conditions

The POD-based ROM approach

m

3



2. POD BasisOrthonomal basis ordered by information content spanning the same space


m

4

3. Order Reduction Select POD components with largest information content






mm ~

m

5



4. Prediction step Determine POD-ROM coefficients by interpolation / solving low-order PDEs / least-squares optimization

4. Prediction step Determine POD-ROM coefficients by interpolation / solving low-order PDEs/ least-squares optimization





mm ~

m

6



5. Output: approximated flow field

Untried flow condition





mm ~4. Prediction step Determine POD-ROM coefficients by interpolation / solving low-order PDEs / least-squares optimization

4. Prediction step Determine POD-ROM coefficients by interpolation / solving low-order PDEs/ least-squares optimization

m

7

POD based reduced order models are a powerful tool…

Industrial aircraft configurationROM Speed-up factor > 300

Grid size ~9Mio, subsonic Ma = 0.2

Snapshot data at AoA = -1°, 0°, 1°, 2°

Prediction at AoA = 7° (Extrapolation!)

8

How to compute POD/SVD of augmented data set efficiently?

… but treating large snapshots may become a challenge!

POD/SVD representation known

Incoming new snapshots

9

Nomenclature

Tm

mm

mm

Tmmm

m

j

jmm

mnmm

VUY

WWAWAWAYY

WA

RWWY

:SVD

),,...,(:),...,(1 :Centering

:averageSnapshot

),...,( :matrixSnapshot

11

1

1

1

1,)( :contentn Informatio Relative1

21

2

mrrric mm m

i i

mr

i i

10

Discard columns corresponding to small singular values:

Definition: The data setis called reduced-order model of order of The ratio is called the compression rate.

m

mmmm

r

mrmr

mrnr

m

Tmmmm

RVVVRUUU

VUY

000000

,,...,,,...,

,

111

mmmmm rmnAVU ,,,,,,mr .mY

mrm

Reduced-order representation

11

Given: ROM of

p new snapshot observations

Task: Compute ROMof

Requirement:use only the previous stage ROM and the incoming snapshots!

! Keep computational costs depending on as low as possible

mmmmm rmnAVU ,,,,,, .mnm RY

.pmY

pmm WW ,...,1

pmpmpmpmpm rpmnAVU ,,,,,,

The SVD basis update problem

mn n

12

Objective

POD

MO

DES

=SVD

SNA

PSH

OTS

Efficient SVD basis update• Update SVD basis without having to store the initial snapshots

13

Objective


POD

MO

DES

=SVD

SNA

PSH

OTS

=

SNA

PSH

OTS

SVD+ +PO

D M

OD

ES

14

Objective

POD

MO

DES

=SVD

SNA

PSH

OTS

SNA

PSH

OTS

+

SNA

PSH

OTS

=SVDPO

D M

OD

ES


15

Updating the snapshot mean

Shift vector:

Shift update snapshots to the previous-stage center:

To do: Decompose

SVD basis update strategies

pmmpm

pm

mi

impmpm

TAA

WpAT

1

1:

piAWW mimim ,...,1,:

,1,!

Tpmpmpm

Tpmpmmpm VUTWYY

16

Updating the snapshot mean

Shift vector:

Shift update snapshots to the previous-stage center:

To do: Decompose

SVD basis update strategies

pmmpm

pm

mi

impmpm

TAA

WpAT

1

1:

piAWW mimim ,...,1,:

,1,!

Tpmpmpm

Tpmpmmpm VUTWYY

start here

17

Setting

it holds

SVD basis update strategies (A): two-steps SVD1,2

ppmpTpnm IBYX ,0,0,

prTmmm

Tm

mVUXBWXWY 0, and ,

1 M. Brand: “Fast low-rank modifications of the thin SVD”, Lin. Alg. and its Appl. 415, 20062 P. Hall et al.: “Merging and splitting eigenspace models”, IEEE Trans. Pattern analysis

and Machine Intelligence, 22(9), 2000

18

Setting

it holds


ppmpTpnm IBYX ,0,0,

prTmmm

Tm

mVUXBWXWY 0, and ,



rank-p SVD update problem

19

Setting

it holds

Factoring out orthogonal components:


ppmpTpnm IBYX ,0,0,

prTmmm

Tm

mVUXBWXWY 0, and ,



),0()0,(

00

, ppmp

prTm

ppm

mT

IV

IWUBWX

m

20

Setting

it holds


where e.g. via Gram Schmidt


ppmpTpnm IBYX ,0,0,

prTmmm

Tm

mVUXBWXWY 0, and ,



),0()0,(

0,

ppmp

prTm

Torth

Tmm

orthmT

IV

PPWU

PUBWXm

)(orth),( PPWUUWP orthTm

21

Setting

it holds



ppmpTpnm IBYX ,0,0,

prTmmm

Tm

mVUXBWXWY 0, and ,



),0()0,(

0,

ppmp

prTm

Torth

Tmm

orthmT

IV

PPWU

PUBWXm

orthonormal columns

22

Setting

it holds



ppmpTpnm IBYX ,0,0,

prTmmm

Tm

mVUXBWXWY 0, and ,



),0()0,(

0,

ppmp

prTm

Torth

Tmm

orthmT

IV

PPWU

PUBWXm

orthonormal columns

23

Setting

it holds



ppmpTpnm IBYX ,0,0,

prTmmm

Tm

mVUXBWXWY 0, and ,



),0()0,(

0,

ppmp

prTm

Torth

Tmm

orthmT

IV

PPWU

PUBWXm

size )()( prpr mm

24

Setting

it holds



ppmpTpnm IBYX ,0,0,

prTmmm

Tm

mVUXBWXWY 0, and ,



),0()0,(~~~,

ppmp

prTmT

orthmT

IV

VUPUBWXm

25

Setting

it holds



ppmpTpnm IBYX ,0,0,

prTmmm

Tm

mVUXBWXWY 0, and ,



),0()0,(~~~,

ppmp

prTmT

orthmT

IV

VUPUBWXm

26

Repeat for shifting to new center:




,1,!

Tpmpmpm

Tpmpmmpm VUTWYY

SVD now known

27

Comments: ‘re-orthogonalization’ expensive, additional (large-scale) SVD requiredless robust via Gram-Schmidt

Parallelization is more involved




)(orth),( PPWUUWP orthTm

28

Objective:

First step: Compute SVD of

Reduce to symmetric EVD

SVD basis update strategies (B): EVD3+SVD

WWYWWYYYWYWY

Tm

T

Tmm

Tm

mT

m ,,

3 Generalize ideas from: J.R. Bunch, C.P. Nielsen: “Updating the singular value decomposition”, Numer. Math. 31, 1978

,1,!

Tpmpmpm

Tpmpmmpm VUTWYY

WYm ,

29

Objective:




WWVUW

WUVVVWYWY TT

mmmT

Tmmm

Tmmm

mT

m

2

,,


,1,!

Tpmpmpm

Tpmpmmpm VUTWYY

WYm ,

Exploit previous stage SVD

30

Objective:




pp

Tm

Tmm

T

Tmmm

ppm

mT

m IV

WWUWWU

IV

WYWY0

00

0,,

2


,1,!

Tpmpmpm

Tpmpmmpm VUTWYY

WYm ,

factor out

31

Objective:




pp

Tm

Tmm

T

Tmmm

ppm

mT

m IV

WWUWWU

IV

WYWY0

00

0,,

2


,1,!

Tpmpmpm

Tpmpmmpm VUTWYY

WYm ,

size )()( prpr mm

32

Objective:




pp

TmT

ppm

mT

m IV

QQI

VWYWY

00~~~

00

,,


,1,!

Tpmpmpm

Tpmpmmpm VUTWYY

WYm ,

33

Objective:



Compute left singular vectors via


pp

TmT

ppm

mT

m IV

QQI

VWYWY

00~~~

00

,,


,1,!

Tpmpmpm

Tpmpmmpm VUTWYY

WYm ,

1~~,ˆ QWUU mm

34

Use Brand’s method for shifting to new center:

SVD basis update strategies (B): EVD3 + SVD

,1,!

Tpmpmpm

Tpmpmmpm VUTWYY

SVD now known


35

Objective:

Let

Reduce to symmetric EVD, exploit previous stage SVD in matrix products

Compute left singular vectors via

SVD basis update strategies (C): one-step EVD

)()(~~~ pmpmTT RQQXX

,1,!

Tpmpmpm

Tpmpmmpm VUTWYY

)(1,: pmnTpmpmm RTWYX

1~~ QXU pm

36

Comments:Straight forward parallelization! (only standard matrix products required in parallel)

SVD basis update strategies (B) and (C)

37

Count flops for matrix product

Strategy (B) is more efficient than strategy (A):

Analysis of Computational costs

nmp pmmn RBRAAB ,,

))((

))(()(flops(B)flops(A)2

221

npmpO

PorthOnppprm

38

The ranking of Strategy (B) vs. (C) depends on the compression rate!

Assumption:

Then the computational costs differ by


2))(32()2(flops(B)flops(C) 222 ppmmpxmxxn

rate)on (compressi , mr

mpmmxpxmprr

39

The ranking of Strategy (B) vs. (C) depends on the compression rate!

Solving the quadratic equation shows thatStrategy (B) is more efficient than Strategy (C) if

In practical implementations, use switch to select the most efficient update strategy.


710)1(10)1(30 2241 ppmpmpmx m

40

Rules of thumb:For highly compressed models, Strategy (B) is more efficient than Strategy (C)For weakly compressed models, the opposite holds true

More preciselyStrategy (B) is more efficient than Strategy (C), if and

Strategy (C) is more efficient than Strategy (B), if


21x

.32x

.12 pm

41

Uncompressed models

Example: Updating SVDs of Random matrices

100,500,000,100,, pmnRWRY pnmn

Method Reconstruction error time (sec)

Strategy (A), SVD-orth 2.504e-12 7.88

Strategy (A), QR-orth 2.333e-12 8.15

Strategy (B) 2.342e-12 6.32

Strategy (C) 2.279e-12 4.42

599,4991 pmm rmr

42

Compression level

Example: Updating SVDs of Random matrices

100,500,000,100,, pmnRWRY pnmn

Method Reconstruction error time (sec)

Strategy (A), SVD-orth 92.21 4.18

Strategy (A), QR-orth 92.21 4.23

Strategy (B) 92.21 1.93

Strategy (C) 93.41 3.54

4.0,200,200 mr

pmmmxrr

43

The update strategies following the symmetric EVD approach are more efficient than the SVD update known from literature (Brand, Hall et al.)(Assumption: n>>m)

Industrial point of view: SVD/EVD performed by ‘black box function’

Most efficient: choose method depending on the compression rate

The examples suggest that all methods share a similar level of accuracy(SVD-approach may suffer from orthogonal out-factoring,EVD-approaches may suffer from squaring the condition number)

Summary

44

For details and additional references, see:“A comprehensive comparison of various algorithms for efficiently updating singular value decomposition based reduced order models”, DLR IB 124-2011/3

Freely available at DLR’s electronic library:http://elib.dlr.de/70251

Summary

45

Thank you for your attention!

on efficiently updating singular value decomposition based ... fileon efficiently updating singular...

Documents