randomized model order reduction · randomized model order reduction alessandro alla work in...

Randomized Model Order Reduction

Alessandro Allawork in collaboration with J. Nathan Kutz

Data-Driven Methods for Reduced-Order Modelingand Stochastic Partial Differential Equations

Banff, January 30, 2016

A. Alla (Florida State University) Randomized MOR 1 / 42

Problem Settings

Dynamical SystemM y(t) = Ay(t) + f(t ,y(t)), t ∈ (0,T ],

y(0) = y0.

Assumptions

y0 ∈ Rn is a given initial data,M,A ∈ Rn×n given matrices,f : [0,T ]× Rn → Rn a continuous function in both arguments andlocally Lipschitz-type with respect to the second variable.

WARNING: High dimensional problems are computationally expensive.


Introduction

Low-rank approximation requires:Solutions of the original high-dimensional system (snapshots),Dimensionality-reduction produced by SVD,Galerkin projection of the dynamics on the low-rank subspace.

WARNING:Offline stages are exceptionally expensive, but enable the (cheap)online stage to potentially run in real time.

GOAL: To improve the efficiency of the offline stageRandomized techniques attempt to construct low-rank matrixdecompositions, fast and accurate approximations of QR and SVD.


Outline

Outline

1 Model Order ReductionProper Orthogonal DecompositionDiscrete Empirical Interpolation MethodDynamic Mode DecompositionCoupling POD and DMD methods

2 Randomized Linear Algebra in Model Order ReductionCompressed Model Order Reduction TechniquesCompressed PODCompressive Sampling DMD

3 Numerical Tests


Model Order Reduction

Outline



3 Numerical Tests


Model Order Reduction Proper Orthogonal Decomposition

Proper Orthogonal Decomposition and SVD

Proper Orthogonal Decomposition (POD), [L. Sirovich ’87]

Simulate at different time instances,Take snapshots of the state,Perform POD (Proper Orthogonal Decomposition) using SVD(Singular Value Decomposition),Use the POD basis functions as (non local) FEM ansatz functions.



Proper Orthogonal Decomposition and SVD

Given snapshots (y(t0), . . . , y(tn)) ∈ Rm

We look for an orthonormal basis ψi`i=1 in Rm with ` minn,m s.t.

J(ψ1, . . . , ψ`) =n∑

j=1

αj

∥∥∥∥∥yj −∑i=1

〈yj , ψi〉ψi

∥∥∥∥∥2

=d∑

i=`+1

σ2i

reaches a minimum where αjnj=1 ∈ R+.

min J(ψ1, . . . , ψ`) s.t .〈ψi , ψj〉 = δij

Singular Value Decomposition: Y = ΨΣV T .

For ` ∈ 1, . . . ,d = rank(Y ), ψi`i=1 are called POD basis of rank `.

ERROR INDICATOR: E(`) =

∑i=1

σi

d∑i=1

σi

with σi singular values of the SVD.



Reduced Order System

POD-Galerkin ansatz

y(t) ≈ ΨPODy`(t), ΨPOD ∈ Rn×`.

POD dynamical systemM`y`(t) = A`y`(t) + (ΨPOD)T f (t ,ΨPODy`(t)),

y`(0) = y`0.

Dimension of the entries

(M`)ij = 〈Mψi ,ψj〉 ∈ R`×`,(A`)ij = 〈Aψi ,ψj〉 ∈ R`×` ,

y`0 = (ΨPOD)T y0 ∈ R`.


Model Order Reduction Discrete Empirical Interpolation Method

Discrete Empirical Interpolation Method

Problem:Reduction of the nonlinearity is NOT independent from n:

F(t ,y`(t)) := (ΨPOD)T f(t ,ΨPODy`(t)) = 〈f(t ,y(t)),ΨPOD〉.

IDEA:Do not evaluate the nonlinearity everywhere, but select the mostimportant points via the greedy procedure.

ReferencesM. Barrault, Y. Maday, N.C. Nguyen, A.T. Patera, An empirical interpolationmethod: application to efficient reduced-basis discretization of partial differentialequations, 2004.

S. Chatarantabut, D. Sorensen, Nonlinear Model Reduction via DiscreteEmpirical Interpolation, 2010.




Compute y(tj) from the dynamical system,Evaluate f(tj ,y(tj)),U ∈ Rn×k the POD basis function of rank k of the nonlinear part.The DEIM approximation of f(t ,y(t)) is as follows

fDEIM(t ,yDEIM(t)) := U(ST U)−1f(t ,yDEIM(t))

where S ∈ Rn×k and yDEIM(t) = STΨPODy`(t) ∈ Rk .

Interpolation Points: Matrix S tells where evaluate the nonlinearity.LU decomposition with pivoting (Chatarantabut, Sorensen, 2010),QR decomposition with pivoting (Drmac, Gugercin, 2015).




Let us define ΨDEIM := U(ST U)−1 ∈ Rn×k .The reduced nonlinearity may be expressed as:

(ΨPOD)T fDEIM(t ,yDEIM(t)) = (ΨPOD)TΨDEIMf(t ,yDEIM)

where we only select a small (sparse) number of rows of ΨPODy`(t).

Computational expenses

STΨPOD ∈ Rk×`, (ST U)−1 ∈ Rk×k and (ΨPOD)TΨDEIM ∈ R`×k

RemarkPrecomputed quantities are independent of the full dimension n.




POD dynamical systemM`y`(t) = A`y`(t) + (ΨPOD)T f (t ,ΨPODy`(t)),

y`(0) = y`0

POD-DEIM dynamical systemM`y`(t) = A`y`(t) + (ΨPOD)TΨDEIMf(t ,yDEIM)y`(0) = y`0.

low-rank approximation of the nonlinear term.

Error Estimation

‖f− fDEIM‖2 ≤ c‖(I− UUT )f‖2, c = ‖(ST U)−1‖2with different performances depending on the matrix S.


Model Order Reduction Dynamic Mode Decomposition

Introduction to DMD

DMD is an equation-free, data-driven method capable of providingaccurate model for complex system, and short-time future estimates ofsuch a systems.

It traces its origins to pioneering work of Bernard Koopman in 1931.Koopman theory is a dynamical systems tool that provides informationabout a nonlinear dynamical system via an associatedinfinite-dimensional linear system.

DMD method was proposed as a data-driven algorithm for modelingcomplex flows as a special case of Koopman theory.(Brunton, Kutz, Mezic, Noack, Rowley, Schmid, Tu, . . . )



Dynamic Mode Decomposition

Dynamic Mode DecompositionSuppose we have a dynamical system and compute snapshots(y(t0), . . . , y(tm) and two sets of data

Y=

y(t0) y(t1) · · · y(tm−1)

, Y′=

y(t1) y(t2) · · · y(tm)

with y(tj) an initial condition of the dynamical system and y(tj+1) itscorresponding output⇒ Y′ = AYY with AY ∈ Rn×n unknown.

The DMD modes are eigenvectors of

Ay = Y′Y†

where † denotes the Moore-Penrose pseudoinverse.A. Alla (Florida State University) Randomized MOR 14 / 42



Ay is a finite dimensional approximation of the Koopman operatorfor a linear observable.The definition of DMD produces a regression procedure wherebythe data snapshots in time are used to produce the best-fit lineardynamical system for the data Y.

The DMD procedure constructs the proxy, approximate linear evolution

d ydt

= Ayy

with y(0) = y0 and whose solution is: y(t) =∑n

i=1 biψi exp(ωi t),ψi and ωi are the eigenfunctions and eigenvalues of the matrix Ay.

SOLUTIONDMD circumvents the eigendecomposition of Ay by considering arank-reduced representation in terms of a POD-projected matrix Ay.




DMD algorithm

Require: Snapshots y(t0), . . . ,y(tm),1: Set Y = [y(t0), . . . ,y(tm−1)] and Y′ = [y(t1), . . . ,y(tm)],2: Compute the (reduced) SVD of Y, Y = UΣVT

3: Define Ay := U∗Y′VΣ−1

4: Compute eigenvalues and eigenvectors of AyW = WΛ.5: Set ΨDMD = Y′VΣ−1W



Example: DMD-Galerkin approximation

yt (x , t) + yx (x , t) = 0 (x , t) ∈ [0,4]× [0,3],

y(x ,0) = y0(x) x ∈ [0,4],

y(0, t) = 0 = y(4, t) t ∈ [0,T ],

where y0(x) = sin(πx) if 0 ≤ x ≤ 1 and 0 elsewhere.

DMD Ansatz: y(t) ≈ ΨDMDyDMD(t).

50 100 150 200 250 300

10−15

10−10

10−5

100



Figure: Reduced approximation with rank=5,10,15. POD approximation(top), DMD-Galerkin (middle), DMD (data-driven) (bottom)



Example: DMD-Galerkin approximation

0 10 20 30 40 50 6010

−14

10−12

10−10

10−8

10−6

10−4

10−2

100

Number of Basis Functions

POD

DMD−Galerkin

DMD

0 0.5 1 1.5 2 2.5 3 3.5 4−0.4

−0.3

−0.2

−0.1

0

0.1

0.2

0.3

POD

DMD−Galerkin

DMD

0 0.5 1 1.5 2 2.5 3 3.5 4−0.2

−0.15

−0.1

−0.05

0

0.05

0.1

0.15

POD

DMD−Galerkin

DMD

Figure: Error analysis with respect to the Frobenius norm (left), first mode(middle), second mode (right).


Model Order Reduction Coupling POD and DMD methods

POD-DMD method

MAIN IDEAThe evaluation of the nonlinearity is the most expensive part in modelorder reduction. We aim faster approximation of the nonlinear term.

We need snapshots!

y(t0), . . . ,y(tm), to compute the POD basis functions,f(t0,y(t0)), . . . , f(tm,y(tm)) to compute the DMD basis functions.

POD-DMD method (NO EVALUATION OF THE NONLINEARITY)(A., Kutz, 2016) M`y`(t) = A`y`(t) + (ΨPOD)TΨDMDdiag(eω

DMDt )b,

y`(0) = y0`.



Example: Semi-Linear Parabolic Equation

yt − θ∆y + µ(y − y3) = 0 (x , t) ∈ Ω× [0,T ],

y(x ,0) = y0(x) x ∈ Ω,

y(·, t) = 0 x ∈ ∂Ω, t ∈ [0,T ],

Parameters:Ω = [0,1]× [0,1],T = 3,y0(x) = 0.1 if 0.1 ≤ x1x2 ≤ 0.6 and 0 elsewhere.




0

0.5

1

0

0.5

1

0

0.02

0.04

0.06

0.08

0.1

0

0.5

1

0

0.5

1

0

0.05

0.1

0.15

0.2

0.25

0

0.5

1

0

0.5

1

0

0.2

0.4

0.6

0.8

1

0

0.5

1

0

0.5

1

0

0.2

0.4

0.6

0.8

1

Figure: Solution at time t = 0,0.1 (top) and t = 1.5,3 (bottom)

.A. Alla (Florida State University) Randomized MOR 22 / 42



0 5 10 15 20 25 30 35 4010

−4

10−3

10−2

10−1

100

101

FULL

POD

POD−DEIM

POD−DMD

DMD

0 5 10 15 20 25 30 35 4010

−10

10−8

10−6

10−4

10−2

100

POD

POD−DEIM

POD−DMD

DMD

Figure: CPU-time online stage (left) and Relative Error wrt Frobenius norm.Number of POD modes and DEIM/DMD points are the same




0 5 10 15 20 25 30 35 4010

−3

10−2

10−1

100

POD−DMD

POD−DEIM

0 5 10 15 20 25 30 35 4010

−4

10−3

10−2

10−1

100

POD−DMD

POD−DEIM

0 5 10 15 20 25 30 35 4010

−5

10−4

10−3

10−2

10−1

100

POD−DMD

POD−DEIM

Figure: Relative Error for 5 POD basis functions (left), 10 POD basis (middle),15 POD basis (right)




A fair comparison

10−6

10−5

10−4

10−3

10−2

10−1

100

10−3

10−2

10−1

100

RELATIVE ERROR

CP

U T

IME

POD

POD−DEIM

POD−DMD

DMD


Randomized Linear Algebra in Model Order Reduction

Outline



3 Numerical Tests



Randomized SVD (Haiko, Martinsson, Tropp, 2011)

Computational Costs for Y ∈ Rm×n:

SVD: O(mn2)

Randomized SVD: O(mn`),

First Steps:

Choose desired target rank ` m,n,Create a random (gaussian) sampling matrix Ω ∈ Rn×`,Sampled matrix X ∈ Rn×` is computed as: X = YΩ.

Remarks:If the matrix Y has exact rank `, then the sampled matrix X spans,with high probability, a basis for the column space.In practice, it is favorable to slightly oversample ` = `+ p, were pdenotes the number of additional samples.



Randomized SVD

Second Steps: Obtain low-rank SVD from a compressed matrix

Compute X = QR, Q ∈ Rn×` orthonormal,Y is projected into this low-dimensional space B ∈ R`×m:

B = QᵀY,

Compute the (cheap) SVD B = UΣVT ,

Set U = QU

Remark: if ` is large enough:

Y ≈QQᵀY≈QB

≈QUΣVT

≈UΣVT



Randomized SVD

0 20 40 60 80 100 12010

−14

10−12

10−10

10−8

10−6

10−4

10−2

100

102

104

Singular Values

FULL

randomized l=r/2

randomized l=r

randomized l=2r


Randomized Linear Algebra in Model Order Reduction Compressed Model Order Reduction Techniques

How to use RSVD in model reduction?

Model order reduction techniques:are based on snapshots of the dynamical system.SVD decomposition of the snapshots matrix provides alow-dimension projector operator that allows one to obtainsurrogate models.WARNING: SVD may be computationally expensive to reducedthe offline cost of the method.

IDEA:Compute basis functions from a few spatially incoherentmeasurements (not from the full set of measurements)


Randomized Linear Algebra in Model Order Reduction Compressed POD

Compressed POD

we collect the snapshot set,we solve the POD optimization problem,optimality conditions provide eigenvalue problems,RSVD computes compressed POD basis functions in asignificantly faster way.

Algorithm

Require: Snapshot Matrix Y ∈ Rn×m, ` number of basis functions., pnumber of measurements.

1: Compute the Randomized SVD [U,Σ,V] = rsvd(Y,p)2: Set Ψi = Ui for i = 1, . . . , `.


Randomized Linear Algebra in Model Order Reduction Compressed POD

Compressed POD

Error (d rank of Y, p number of samples)

E

m∑j=1

αj

∥∥∥∥∥y(tj)−∑i=1

〈y(tj),ψi〉ψi

∥∥∥∥∥2 =

(1 +

√`

p − 1

)σ2`+1 +

√`+ pp

d∑j=`+1

σ2j .

Remarks:we consider the expectation value of the errorerror depends on the computation of the set of snapshots and p,if the singular values of Y decay rapidly a few samples drives theerror close to the theoretically minimum value.if the singular values do not decay rapidly we can lose accuracy.we suppose Y ∈ Rm×n, such that m ≈ n(eigenvalue problem is also expensive)


Randomized Linear Algebra in Model Order Reduction Compressive Sampling DMD

Compressive DMD (Brunton, Proctor, Kutz, 2015)

GOAL:Compute DMD and apply as Galerkin projection method.

Algorithm

Require: Snapshots y(t0), . . . ,y(tm), C ∈ Rp×m

1: Set Y = [y(t0), . . . ,y(tm−1)] and Y ′ = [y(t1), . . . ,y(tm)],2: X = CY, X′ = CY′

3: Compute the SVD of X, X = UΣVT

4: Define Ax := U∗Y′VΣ−1

5: Compute eigenvalues and eigenvectors of AxW = WΛ.6: Set ΨDMD = X′VΣ−1W


Numerical Tests

Outline



3 Numerical Tests


Numerical Tests

Test 1: Semi-Linear Parabolic Equation

yt − θ∆y + µ(y − y3) = 0 (x , t) ∈ Ω× [0,T ],

y(x ,0) = y0(x) x ∈ Ω,

y(·, t) = 0 x ∈ ∂Ω, t ∈ [0,T ],

Parameters:Ω = [0,1]× [0,1],T = 3,y0(x) = 0.1 if 0.1 ≤ x1x2 ≤ 0.6 and 0 elsewhere.


Numerical Tests


0 5 10 15 20 2510

−1

100

101

102

CPU TIME

Number Of Basis Functions

POD

rPOD

POD−DEIM

rPOD−rDEIM

DMD

cDMD

0 5 10 15 20 2510

−8

10−6

10−4

10−2

100

RELATIVE ERROR


POD

cPOD

POD−DEIM

rPOD−rDEIM

DMD

cDMD



Numerical Tests


1000 2000 3000 4000 5000 6000

10−2

100

102

104

CPU TIME

Dimension of the full problem

POD

cPOD

POD−DEIM

rPOD−rDEIM

DMD

cDMD

1000 2000 3000 4000 5000 600010

−4

10−3

10−2

10−1

100

RELATIVE ERROR

Dimension of the full problem

PODcPODPOD−DEIMrPOD−rDEIMDMDcDMD



Numerical Tests

Test 2: Parametric Elliptic Equation

−∆u(x , y) + s(u(x , y);µ) = f (x , y) (x , y) ∈ Ω

u(x , y) = 0 (x , y) ∈ ∂Ω

Parameters:

Ω = [0,1]× [0,1], µ = (µ1, µ2) ∈ D = [0.01,10]2

s(u, µ) =µ1

µ2(eµ2u − 1), f (x , y) = 100 sin(2πx) sin(2πy).


Numerical Tests

Test 2: Parametric Elliptic Equations

0

0.2

0.4

0.6

0.8

1

0

0.2

0.4

0.6

0.8

1

−1.5

−1

−0.5

0

0.5

1

1.5

0

0.2

0.4

0.6

0.8

1

0

0.2

0.4

0.6

0.8

1

−1.5

−1

−0.5

0

0.5

1

1.5

0

0.2

0.4

0.6

0.8

1

0

0.2

0.4

0.6

0.8

1

−1.5

−1

−0.5

0

0.5

1

1.5

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

−1

−0.5

0

0.5

1

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

−1.2

−1

−0.8

−0.6

−0.4

−0.2

0

0.2

0.4

0.6

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 10

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

−1.4

−1.2

−1

−0.8

−0.6

−0.4

−0.2

0

0.2

Figure: Test 2: Solution of problem for different parametric configurations(top) and countour lines (bottom)


Numerical Tests

Test 2: Parametric Elliptic Equations

0 5 10 15 2010

−1

100

101

102

CPU TIME


POD

cPOD

POD−DEIM

cPOD−cDEIM

DMD

cDMD

0 5 10 15 2010

−8

10−6

10−4

10−2

100

RELATIVE ERROR


POD

cPOD

POD−DEIM

cPOD−cDEIM

DMD

cDMD

Figure: Test 2: CPU-time of the offline-online stages (left) and Relative Errorin Frobenius norm (right). We compare the following methods: POD (red),cPOD (magenta), POD-DEIM (yellow), cPOD-cDEIM (blue, DMD (black),cDMD (green). Number of model are always the same for all the methods.


Numerical Tests

Conclusions

Model order reduction is a successful technique that projectsnonlinear high dimensional dynamical systems and PDEs into lowdimensional surrogate modelsCompressed (randomized) techniques are a promising approachto circumventing expensive offline stages in model orderreduction.DMD works successfully in a Galerkin projection framework


Numerical Tests

Thank you for your attention

A. Alla, J.N. Kutz, Randomized Model Order Reduction, 2016.A. Alla, J.N. Kutz, Nonlinear Model Reduction via DMD, 2016.M. Barrault, Y. Maday, N.C. Nguyen, A.T. Patera, An empirical interpolationmethod: application to efficient reduced-basis discretization of partial differentialequations, 2004.P. Benner, S. Gugercin and K. Willcox, A Survey of Projection-Based ModelReduction Methods for Parametric Dynamical Systems, 2015.S. L. Brunton, J. L. Proctor, and J. N. Kutz, Compressive sampling and dynamicmode decomposition. 2015.S. Chatarantabut, D. Sorensen, Nonlinear Model Reduction via DiscreteEmpirical Interpolation. 2010.Z. Drmac, S. Gugercin, A new selection operator for the discrete empiricalinterpolation method - improved a priori error bound and extensions, 2016.N. Halko, P.-G. Martinsson and J. Tropp, Finding Structure with Randomness:Probabilistic Algorithms for Constructing Approximate Matrix Decompositions,2011.P. Schmid, Dynamic mode decomposition of numerical and experimental data,2010.J. Tu, C. Rowley, D. Luchtenberg, S. Brunton, and J. N. Kutz, On Dynamic ModeDecomposition: Theory and Applications, 2014.S. Volkwein, Model Reduction using Proper Orthogonal Decomposition, 2013.


randomized model order reduction · randomized model order reduction alessandro alla work in...

Documents