adi for tensor structured equations
TRANSCRIPT
![Page 1: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/1.jpg)
MAX PLANCK INSTITUTE
FOR DYNAMICS OF COMPLEX
TECHNICAL SYSTEMS
MAGDEBURG
83rd GAMM Annual Scientific ConferenceDarmstadt, 28 March 2012
ADI for Tensor Structured Equations
Thomas Mach and Jens Saak
Max Planck Institute for Dynamics of Complex Technical SystemsComputational Methods in Systems and Control Theory
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 1/24
![Page 2: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/2.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Classic ADI [Peaceman/Rachford ’55]
Developed to solve linear systems related to Poisson problems
−∆u = f in Ω ⊂ Rd , d = 1, 2
u = 0 on ∂Ω.
uniform grid size h, centered differences, d = 1,
⇒ ∆1,hu = h2 f
∆1,h =
2 −1−1 2 −1
. . .. . .
. . .
−1 2 −1−1 2
.
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 2/24
![Page 3: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/3.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Classic ADI [Peaceman/Rachford ’55]
Developed to solve linear systems related to Poisson problems
−∆u = f in Ω ⊂ Rd , d = 1, 2
u = 0 on ∂Ω.
uniform grid size h, 5-point difference star, d = 2,
⇒ ∆2,hu = h2 f
∆2,h =
K −I−I K −I
. . .. . .
. . .
−I K −I−I K
and K =
4 −1−1 4 −1
. . .. . .
. . .
−1 4 −1−1 4
.
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 2/24
![Page 4: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/4.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Classic ADI [Peaceman/Rachford ’55]
Observation
∆2,h = (∆1,h ⊗ I )︸ ︷︷ ︸=:H
+ (I ⊗∆1,h)︸ ︷︷ ︸=:V
.
Solve ∆2,hu = h2 f =: f exploiting structure in H and V .
For certain shift parameters perform
(H + pi I ) ui+ 12
= (pi I − V ) ui + f ,
(V + pi I ) ui+1 = (pi I − H) ui+ 12
+ f ,
until ui is good enough.
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 3/24
![Page 5: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/5.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Classic ADI [Peaceman/Rachford ’55]
Observation
∆2,h = (∆1,h ⊗ I )︸ ︷︷ ︸=:H
+ (I ⊗∆1,h)︸ ︷︷ ︸=:V
.
Solve ∆2,hu = h2 f =: f exploiting structure in H and V .
For certain shift parameters perform
(H + pi I ) ui+ 12
= (pi I − V ) ui + f ,
(V + pi I ) ui+1 = (pi I − H) ui+ 12
+ f ,
until ui is good enough.
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 3/24
![Page 6: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/6.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Classic ADI [Peaceman/Rachford ’55]
Observation
∆2,h = (∆1,h ⊗ I )︸ ︷︷ ︸=:H
+ (I ⊗∆1,h)︸ ︷︷ ︸=:V
.
Solve ∆2,hu = h2 f =: f exploiting structure in H and V .
For certain shift parameters perform
(H + pi I ) ui+ 12
= (pi I − V ) ui + f ,
(V + pi I ) ui+1 = (pi I − H) ui+ 12
+ f ,
until ui is good enough.
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 3/24
![Page 7: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/7.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
ADI and Lyapunov Equations [Wachspress ’88]
Lyapunov Equation
FX + XFT = −GGT
Vectorized Lyapunov Equation[(I ⊗ F )︸ ︷︷ ︸
=:HF
+ (F ⊗ I )︸ ︷︷ ︸=:VF
]vec(X ) = −vec(GGT )
Same structure ⇒ apply ADI
(F + pi I )Xi+ 12
= −GGT − Xi
(FT − pi I
)(F + pi I )Xi+1 = −GGT − XT
i+ 12
(FT − pi I
)
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 4/24
![Page 8: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/8.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
ADI and Lyapunov Equations [Wachspress ’88]
Lyapunov Equation
FX + XFT = −GGT
Vectorized Lyapunov Equation[(I ⊗ F )︸ ︷︷ ︸
=:HF
+ (F ⊗ I )︸ ︷︷ ︸=:VF
]vec(X ) = −vec(GGT )
Same structure ⇒ apply ADI
(F + pi I )Xi+ 12
= −GGT − Xi
(FT − pi I
)(F + pi I )Xi+1 = −GGT − XT
i+ 12
(FT − pi I
)
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 4/24
![Page 9: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/9.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
ADI and Lyapunov Equations [Wachspress ’88]
Lyapunov Equation
FX + XFT = −GGT
Vectorized Lyapunov Equation[(I ⊗ F )︸ ︷︷ ︸
=:HF
+ (F ⊗ I )︸ ︷︷ ︸=:VF
]vec(X ) = −vec(GGT )
Same structure ⇒ apply ADI
(F + pi I )Xi+ 12
= −GGT − Xi
(FT − pi I
)(F + pi I )Xi+1 = −GGT − XT
i+ 12
(FT − pi I
)Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 4/24
![Page 10: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/10.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Generalizing Matrix Equations
∆2,hvec(X ) = vec(B)(
I ⊗ I ⊗
I ⊗ ∆1,h︸ ︷︷ ︸=H
+
I ⊗ I ⊗
∆1,h ⊗ I︸ ︷︷ ︸=V
+ I ⊗ ∆1,h ⊗ I ⊗ I︸ ︷︷ ︸=R
+ ∆1,h ⊗ I ⊗ I ⊗ I︸ ︷︷ ︸=Q
)vec(X )︸ ︷︷ ︸
=u
= vec(B)︸ ︷︷ ︸=f
Xa
b
c
d Xa
b
c
d
Xa
b
c
d Xabcd
a∆µa
b∆µb
c ∆µc
d ∆µd
+
+
+ = Ba
b
c
d
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 5/24
![Page 11: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/11.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Generalizing Matrix Equations
∆4,hvec(X ) = vec(B)(I ⊗ I ⊗ I ⊗ ∆1,h︸ ︷︷ ︸
=H
+ I ⊗ I ⊗ ∆1,h ⊗ I︸ ︷︷ ︸=V
+ I ⊗ ∆1,h ⊗ I ⊗ I︸ ︷︷ ︸=R
+ ∆1,h ⊗ I ⊗ I ⊗ I︸ ︷︷ ︸=Q
)vec(X )︸ ︷︷ ︸
=u
= vec(B)︸ ︷︷ ︸=f
Xabcd Xabcd
Xabcd Xabcd
a∆µa
b∆µb
c ∆µc
d ∆µd
+
+
+ = Babcd
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 5/24
![Page 12: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/12.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Generalizing ADI
(I ⊗ ∆1,h︸ ︷︷ ︸
=H
+ ∆1,h ⊗ I︸ ︷︷ ︸=V
)vec(X )︸ ︷︷ ︸
=u
= vec(B)︸ ︷︷ ︸=f
(H + I ⊗ pi ,1I )Xi+ 12
= (pi ,1I − V )Xi + B
(V + pi ,2I ⊗ I )Xi+ 12
= (pi ,2I − H)Xi+ 12
+ B
(I ⊗ I ⊗ I ⊗ ∆1,h︸ ︷︷ ︸
=H
+ I ⊗ I ⊗ ∆1,h ⊗ I︸ ︷︷ ︸=V
+ I ⊗ ∆1,h ⊗ I ⊗ I︸ ︷︷ ︸=R
+ ∆1,h ⊗ I ⊗ I ⊗ I︸ ︷︷ ︸=Q
)vec(X )︸ ︷︷ ︸
=u
= vec(B)︸ ︷︷ ︸=f
(H + I ⊗ I ⊗ I ⊗ pi ,1I )Xi+ 14
= (pi ,1I − V − R − Q)Xi + B
(V + I ⊗ I ⊗ pi ,2I ⊗ I )Xi+ 12
= (pi ,2I − H − R − Q)Xi+ 14
+ B
(R + I ⊗ pi ,3I ⊗ I ⊗ I )Xi+ 34
= (pi ,3I − H − V − Q)Xi+ 12
+ B
(Q + pi ,4I ⊗ I ⊗ I ⊗ I )Xi+1 = (pi ,4I − H − V − R)Xi+ 34
+ B
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 6/24
![Page 13: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/13.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Generalizing ADI
(I ⊗ ∆1,h︸ ︷︷ ︸
=H
+ ∆1,h ⊗ I︸ ︷︷ ︸=V
)vec(X )︸ ︷︷ ︸
=u
= vec(B)︸ ︷︷ ︸=f
(H + I ⊗ pi ,1I )Xi+ 12
= (pi ,1I − V )Xi + B
(V + pi ,2I ⊗ I )Xi+ 12
= (pi ,2I − H)Xi+ 12
+ B
(I ⊗ I ⊗ I ⊗ ∆1,h︸ ︷︷ ︸
=H
+ I ⊗ I ⊗ ∆1,h ⊗ I︸ ︷︷ ︸=V
+ I ⊗ ∆1,h ⊗ I ⊗ I︸ ︷︷ ︸=R
+ ∆1,h ⊗ I ⊗ I ⊗ I︸ ︷︷ ︸=Q
)vec(X )︸ ︷︷ ︸
=u
= vec(B)︸ ︷︷ ︸=f
(H + I ⊗ I ⊗ I ⊗ pi ,1I )Xi+ 14
= (pi ,1I − V − R − Q)Xi + B
(V + I ⊗ I ⊗ pi ,2I ⊗ I )Xi+ 12
= (pi ,2I − H − R − Q)Xi+ 14
+ B
(R + I ⊗ pi ,3I ⊗ I ⊗ I )Xi+ 34
= (pi ,3I − H − V − Q)Xi+ 12
+ B
(Q + pi ,4I ⊗ I ⊗ I ⊗ I )Xi+1 = (pi ,4I − H − V − R)Xi+ 34
+ B
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 6/24
![Page 14: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/14.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Goal
Solve AX = B
A = I ⊗ I ⊗ · · · ⊗ I ⊗ I ⊗ A1 +
I ⊗ I ⊗ · · · ⊗ I ⊗ A2 ⊗ I +
. . . +
Ad ⊗ I ⊗ · · · ⊗ I ⊗ I ⊗ I
B is given in tensor train decomposition⇒ X is sought in tensor train decomposition.
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 7/24
![Page 15: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/15.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Tensor Trains [Oseledets, Tyrtyshnikov ’09]
T (i1, i2, . . . , id) =
r1,...,rd−1∑α1,...,αd−1=1
G1(i1, α1)G2(α1, i2, α2)
· · ·Gj(αj−1, ij , αj) · · ·Gd−1(αd−2, id−1, αd−1)Gd(αd−1, id).
G1(i1, α1) α1 G2(α1, i2, α2) α2 · · · Gd(αd−1, id)
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 8/24
![Page 16: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/16.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Tensor Trains [Oseledets, Tyrtyshnikov ’09]
Tensor trains are
computable, and
require only O(dnr2) storage, with TT-rank r and T ∈ Rnd .
Canonical representation
T (i1, i2, . . . , id) =∑α
G1(i1, α) · · ·Gd(id , α)
Tucker decomposition
T (i1, i2, . . . , id) =∑
α1,...,αd
C (α1, . . . , αd)G1(i1, α1) · · ·Gd(id , αd)
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 9/24
![Page 17: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/17.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Tensor Trains [Oseledets, Tyrtyshnikov ’09]
(I ⊗ · · · ⊗ I ⊗ A1)
−1
T
G1(i1, α1)
i1
A1(β, i1)
α1 G2(α1, i2, α2) α2 · · · Gd(αd−1, id)
= G1(β, α1) = A1G1
T (i1, i2, . . . , id)×1 A1
−1
=∑
α1,...,αd−1
A1
−1
∣∣β,i1G1(i1, α1)G2(α1, i2, α2)
· · ·Gd−1(αd−2, id−1, αd−1)Gd(αd−1, id)
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 10/24
![Page 18: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/18.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Tensor Trains [Oseledets, Tyrtyshnikov ’09]
(I ⊗ · · · ⊗ I ⊗ A1)
−1
T
G1(i1, α1)
i1
A1(β, i1)
α1 G2(α1, i2, α2) α2 · · · Gd(αd−1, id)
= G1(β, α1) = A1G1
T (i1, i2, . . . , id)×1 A1
−1
=∑
α1,...,αd−1
A1
−1
∣∣β,i1G1(i1, α1)G2(α1, i2, α2)
· · ·Gd−1(αd−2, id−1, αd−1)Gd(αd−1, id)
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 10/24
![Page 19: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/19.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Tensor Trains [Oseledets, Tyrtyshnikov ’09]
(I ⊗ · · · ⊗ I ⊗ A1)
−1
T
G1(i1, α1)
i1
A1(β, i1)
α1 G2(α1, i2, α2) α2 · · · Gd(αd−1, id)
= G1(β, α1) = A1G1
T (i1, i2, . . . , id)×1 A1
−1
=∑
α1,...,αd−1
A1
−1
∣∣β,i1G1(i1, α1)G2(α1, i2, α2)
· · ·Gd−1(αd−2, id−1, αd−1)Gd(αd−1, id)
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 10/24
![Page 20: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/20.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Tensor Trains [Oseledets, Tyrtyshnikov ’09]
(I ⊗ · · · ⊗ I ⊗ A1)
−1
T
G1(i1, α1)
i1
A1(β, i1)
α1 G2(α1, i2, α2) α2 · · · Gd(αd−1, id)
= G1(β, α1) = A1G1
T (i1, i2, . . . , id)×1 A1
−1
=∑
α1,...,αd−1
A1
−1
∣∣β,i1G1(i1, α1)G2(α1, i2, α2)
· · ·Gd−1(αd−2, id−1, αd−1)Gd(αd−1, id)
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 10/24
![Page 21: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/21.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Tensor Trains [Oseledets, Tyrtyshnikov ’09]
(I ⊗ · · · ⊗ I ⊗ A1)−1T
G1(i1, α1)
i1
A1(β, i1)
α1 G2(α1, i2, α2) α2 · · · Gd(αd−1, id)
= G1(β, α1) = A1\G1
T (i1, i2, . . . , id)×1 A1−1 =
∑α1,...,αd−1
A1−1∣∣β,i1G1(i1, α1)G2(α1, i2, α2)
· · ·Gd−1(αd−2, id−1, αd−1)Gd(αd−1, id)
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 10/24
![Page 22: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/22.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Algorithm
Input: A1, . . . ,Ad, tensor train B, accuracy εOutput: tensor train X , with AX = Bforall j ∈ 1, . . . , d do
X(0)j := zeros(n, 1, 1)
end
while∥∥r (i)
∥∥ > ε do
Choose shift piforall k ∈ 1, . . . , d do
X (i+ kd ) :=
(B+piX
(i+ k−1d )−
d∑j=1j 6=k
X (i+ k−1d ) ×j Aj
)×k (Ak + pi I )
−1
end
end
r (i) := Bforall j ∈ 1, . . . , d do
r (i) := r (i) − Xi ×j Aj
end
(I ⊗ I ⊗ · · · ⊗ I ⊗ Aj ⊗ I ⊗ · · · ⊗ I )Xi+ k−1d
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 11/24
![Page 23: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/23.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Algorithm
Input: A1, . . . ,Ad, tensor train B, accuracy εOutput: tensor train X , with AX = Bforall j ∈ 1, . . . , d do
X(0)j := zeros(n, 1, 1)
end
while∥∥r (i)
∥∥ > ε do
Choose shift piforall k ∈ 1, . . . , d do
X (i+ kd ) :=
(B+piX
(i+ k−1d )−
d∑j=1j 6=k
X (i+ k−1d ) ×j Aj
)×k (Ak + pi I )
−1
end
end
r (i) := Bforall j ∈ 1, . . . , d do
r (i) := r (i) − Xi ×j Aj
end
(I ⊗ I ⊗ · · · ⊗ I ⊗ Aj ⊗ I ⊗ · · · ⊗ I )Xi+ k−1d
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 11/24
![Page 24: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/24.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Algorithm
Input: A1, . . . ,Ad, tensor train B, accuracy εOutput: tensor train X , with AX = Bforall j ∈ 1, . . . , d do
X(0)j := zeros(n, 1, 1)
end
while∥∥r (i)
∥∥ > ε do
Choose shift piforall k ∈ 1, . . . , d do
X (i+ kd ) :=
(B+piX
(i+ k−1d )−
d∑j=1j 6=k
X (i+ k−1d ) ×j Aj
)×k (Ak + pi I )
−1
end
end
r (i) := Bforall j ∈ 1, . . . , d do
r (i) := r (i) − Xi ×j Aj
end
(I ⊗ I ⊗ · · · ⊗ I ⊗ Aj ⊗ I ⊗ · · · ⊗ I )Xi+ k−1d
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 11/24
![Page 25: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/25.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Eigenvalues
A = I ⊗ · · · ⊗ I ⊗A1 + I ⊗ · · · ⊗ I ⊗A2 ⊗ I + . . .+ Ad ⊗ I ⊗ · · · ⊗ I
Stephanos’ theorem:
⇒ λi (A) = λi1(A1) + λi2(A2) + · · ·+ λid (Ad),
with i = i1 + i2n1 + · · ·+ idd−1∏j=1
nj .
AX = B ⇔d∑
j=1X ×j Aj = B
A is regular ⇔ λi (A) 6= 0 ∀i ⇐ Ai Hurwitz ∀i
mk = argmaxj∈1,...,nk : Im (λi (Ak ))≥0
|λj(Ak)|
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 12/24
![Page 26: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/26.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Eigenvalues
A = I ⊗ · · · ⊗ I ⊗A1 + I ⊗ · · · ⊗ I ⊗A2 ⊗ I + . . .+ Ad ⊗ I ⊗ · · · ⊗ I
Stephanos’ theorem:
⇒ λi (A) = λi1(A1) + λi2(A2) + · · ·+ λid (Ad),
with i = i1 + i2n1 + · · ·+ idd−1∏j=1
nj .
AX = B ⇔d∑
j=1X ×j Aj = B
A is regular ⇔ λi (A) 6= 0 ∀i ⇐ Ai Hurwitz ∀i
mk = argmaxj∈1,...,nk : Im (λi (Ak ))≥0
|λj(Ak)|
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 12/24
![Page 27: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/27.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Lemma
Lemma [Grasedyck ’04]
The tensor equation ∑dj=1 X ×j Aj = B
with Ak Hurwitz ∀k has the solution
X = −∫∞
0 B ×1 exp(A1t)×2 · · · ×d exp(Ad t)dt
Z (t) = B ×1 exp(A1t)×2 · · · ×d exp(Ad t)
Z (t) =d∑
j=1
Z (t)×j Aj Z (∞)− Z (0) =
∫ ∞0
Z (t)dt,
0− B =d∑
j=1
∫ ∞0
Z (t)dt ×j Aj
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 13/24
![Page 28: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/28.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Theorem
Theorem
A1, . . . ,Ad ⇒ A, Λ(A) ⊂ [−λmax,−λmin]⊕ ı [−µ, µ] ⊂ C−.Let k ∈ N and use the quadrature points and weights:
hst := π√k, tj := log
(e jhst +
√1 + e2jhst
), wj := hst√
1+e−2jhst.
Then the solution X can be approximated by
X (i1, i2, . . . , id) = −r1,...,rd−1∑
α1,...,αd−1=1H1(i1, α1) · · ·Hd(αd−1, id),
with Hp(αp−1, ip, αp) :=∑k
j=−k2wj
λmin
∑βpe
2tjλmin
Ap∣∣ip,βp
Gp (αp−1, βp, αp)
with the approximation error
‖X − X‖2 ≤ Cstπλmin
e2µλ−1
min+1
π−π√k∮
Γ
∥∥∥(λI − 2A/λmin)−1∥∥∥
2dΓλ ‖B‖2 .
extending [Grasedyck ’04] (X and B of low Kronecker rank) to low TT-rank
B (i1, i2, . . . , id) =∑
α1,...,αd−1
G1(i1, α1)G2(α1, i2, α2)
· · ·Gj(αj−1, ij , αj) · · ·Gd−1(αd−2, id−1, αd−1)Gd(αd−1, id).
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 14/24
![Page 29: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/29.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Approximation Accuracy
0 5 10 15 20 25 30
2
4
6
8
Iteration
Sto
rage
in10
4·D
oub
le constant truncation errortightened truncation error
10−20
10−14
10−8
10−2
Tru
nca
tion
Err
orε i
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 15/24
![Page 30: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/30.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Example: Laplace – Ai = ∆1, 111
Ai = ∆1, 111
B =[0 0 . . . 0 1
]
Shifts:pi := e1(∗1) + . . .+ ed(∗d) — random chosen eigenvalue
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 16/24
![Page 31: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/31.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Numerical Results – Ai = ∆1, 111
d t in s residual mean(#it)
2 3.887 e−01 7.015 e−10 112.85 5.398 e+00 7.467 e−10 45.88 6.007 e+00 6.936 e−10 12.8
10 3.662 e+00 7.685 e−10 6.825 3.142 e+01 2.437 e−10 5.050 2.268 e+02 2.049 e−10 5.075 7.192 e+02 4.036 e−10 5.0
100 1.700 e+03 1.864 e−10 5.0150 5.538 e+03 1.801 e−10 5.0200 1.280 e+04 1.472 e−10 5.0250 2.499 e+04 1.816 e−10 5.0300 4.298 e+04 2.535 e−10 5.0500 1.952 e+05 2.039 e−10 5.0
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 17/24
![Page 32: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/32.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Numerical Results – Ai = ∆1, 111
sparse densed TADI \ MESS Penzl’s sh. \ lyap
2 0.310 0.0006 0.024 0.003 0.0003 0.00054 3.130 0.1695 0.011 0.049 6.331 0.0126 8.147 — 0.076 0.094 — 7.178 5.458 — 5.863 1.097 — 13 698.2
10 5.306 — 3 445.523 249.464 — —
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 18/24
![Page 33: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/33.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Numerical Results – Ai = ∆1, 111
10 100 30010−2
10−1
100
101
102
103
104
105
Dimension d
Com
pu
tati
onT
ime
ins
Tensor ADI
sparse \MESSPenzl’s shifts
dense \lyap
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 19/24
![Page 34: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/34.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Numerical Results – Ai = ∆1, 111
10 100 30010−2
10−1
100
101
102
103
104
105
Dimension d
Com
pu
tati
onT
ime
ins
Tensor ADI
sparse \MESSPenzl’s shifts
dense \lyap
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 19/24
![Page 35: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/35.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Single Shift and Convergence
A = I ⊗ · · · ⊗ I ⊗A1 + I ⊗ · · · ⊗ I ⊗A2 ⊗ I + . . .+ Ad ⊗ I ⊗ · · · ⊗ I
We assume Λ(Ak) ⊂ R−.
Error Propagation, Single Shift
‖G1‖2 ≤ maxλk∈Λ(Ak ),k=1,...,d
∣∣∣∣∣∣d∏
l=0
p −∑k
λk + λl
p + λl
∣∣∣∣∣∣ =
∣∣∣∣∣∣d∏
l=0
1−
∑k
λk
p + λl
∣∣∣∣∣∣ .If ‖G1‖2 < 1, then the ADI iteration converges.
p < 0 and p > −∞
p < λi (A) =∑d
k=1 λk(Ak) ∀iLyapunov case (Ak = A0 ∀k): p < d−2
2 λmin(A0)
= 0
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 20/24
![Page 36: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/36.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Single Shift and Convergence
A = I ⊗ · · · ⊗ I ⊗A1 + I ⊗ · · · ⊗ I ⊗A2 ⊗ I + . . .+ Ad ⊗ I ⊗ · · · ⊗ I
We assume Λ(Ak) ⊂ R−.
Error Propagation, Single Shift
‖G1‖2 ≤ maxλk∈Λ(Ak ),k=1,...,d
∣∣∣∣∣∣d∏
l=0
p −∑k
λk + λl
p + λl
∣∣∣∣∣∣ =
∣∣∣∣∣∣d∏
l=0
1−
∑k
λk
p + λl
∣∣∣∣∣∣ .If ‖G1‖2 < 1, then the ADI iteration converges.
p < 0 and p > −∞p < λi (A) =
∑dk=1 λk(Ak) ∀i
Lyapunov case (Ak = A0 ∀k): p < d−22 λmin(A0)
= 0
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 20/24
![Page 37: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/37.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Single Shift and Convergence
A = I ⊗ · · · ⊗ I ⊗A1 + I ⊗ · · · ⊗ I ⊗A2 ⊗ I + . . .+ Ad ⊗ I ⊗ · · · ⊗ I
We assume Λ(Ak) ⊂ R−.
Error Propagation, Single Shift
‖G1‖2 ≤ maxλk∈Λ(Ak ),k=1,...,d
∣∣∣∣∣∣d∏
l=0
p −∑k
λk + λl
p + λl
∣∣∣∣∣∣ =
∣∣∣∣∣∣d∏
l=0
1−
∑k
λk
p + λl
∣∣∣∣∣∣ .If ‖G1‖2 < 1, then the ADI iteration converges.
p < 0 and p > −∞p < λi (A) =
∑dk=1 λk(Ak) ∀i
Lyapunov case (Ak = A0 ∀k): p < d−22 λmin(A0)
= 0
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 20/24
![Page 38: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/38.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Single Shift and Convergence
A = I ⊗ · · · ⊗ I ⊗A1 + I ⊗ · · · ⊗ I ⊗A2 ⊗ I + . . .+ Ad ⊗ I ⊗ · · · ⊗ I
We assume Λ(Ak) ⊂ R−.
Error Propagation, Single Shift
‖G1‖2 ≤ maxλk∈Λ(Ak ),k=1,...,d
∣∣∣∣∣∣d∏
l=0
p −∑k
λk + λl
p + λl
∣∣∣∣∣∣ =
∣∣∣∣∣∣d∏
l=0
1−
∑k
λk
p + λl
∣∣∣∣∣∣ .If ‖G1‖2 < 1, then the ADI iteration converges.
p < 0 and p > −∞p < λi (A) =
∑dk=1 λk(Ak) ∀i
Lyapunov case (Ak = A0 ∀k): p < 2−22 λmin(A0) = 0
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 20/24
![Page 39: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/39.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Shifts
Min-Max-Problem
minp1,1,...,p`,d⊂C
maxλk∈Λ(Ak ) ∀k
∣∣∣∣∣∏i=0
d∏k=0
pi ,k −∑
j 6=k λj
pi ,k + λk
∣∣∣∣∣
Min-Max-Problem, Lyapunov case (Ak = A0 ∀k , A0 Hurwitz)
maxλk∈Λ(A0) ∀k
∣∣∣∣∣∏i=0
d∏k=0
pi
,k
−∑
j 6=k λj
pi
,k
+ λk
∣∣∣∣∣λk = λ0 ∀kPenzl’s idea: p1, . . . , p` ⊂ (d − 1)Λ(A0)
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 21/24
![Page 40: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/40.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Shifts
Min-Max-Problem
minp1,1,...,p`,d⊂C
maxλk∈Λ(Ak ) ∀k
∣∣∣∣∣∏i=0
d∏k=0
pi ,k −∑
j 6=k λj
pi ,k + λk
∣∣∣∣∣Min-Max-Problem, Lyapunov case (Ak = A0 ∀k , A0 Hurwitz)
minp1,1,...,p`,d⊂C
maxλk∈Λ(A0) ∀k
∣∣∣∣∣∏i=0
d∏k=0
pi ,k −∑
j 6=k λj
pi ,k + λk
∣∣∣∣∣
λk = λ0 ∀kPenzl’s idea: p1, . . . , p` ⊂ (d − 1)Λ(A0)
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 21/24
![Page 41: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/41.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Shifts
Min-Max-Problem
minp1,1,...,p`,d⊂C
maxλk∈Λ(Ak ) ∀k
∣∣∣∣∣∏i=0
d∏k=0
pi ,k −∑
j 6=k λj
pi ,k + λk
∣∣∣∣∣Min-Max-Problem, Lyapunov case (Ak = A0 ∀k , A0 Hurwitz)
minp1,...,p`⊂C
maxλk∈Λ(A0) ∀k
∣∣∣∣∣∏i=0
d∏k=0
pi
,k
−∑
j 6=k λj
pi
,k
+ λk
∣∣∣∣∣
λk = λ0 ∀kPenzl’s idea: p1, . . . , p` ⊂ (d − 1)Λ(A0)
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 21/24
![Page 42: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/42.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Shifts
Min-Max-Problem
minp1,1,...,p`,d⊂C
maxλk∈Λ(Ak ) ∀k
∣∣∣∣∣∏i=0
d∏k=0
pi ,k −∑
j 6=k λj
pi ,k + λk
∣∣∣∣∣Min-Max-Problem, Lyapunov case (Ak = A0 ∀k , A0 Hurwitz)
minp1,...,p`⊂C
maxλk∈Λ(A0) ∀k
∣∣∣∣∣∏i=0
d∏k=0
pi
,k
−∑
j 6=k λj
pi
,k
+ λk
∣∣∣∣∣λk = λ0 ∀kPenzl’s idea: p1, . . . , p` ⊂ (d − 1)Λ(A0)
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 21/24
![Page 43: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/43.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Random Example
seed := 1;
R := rand(10);
R := R + R ′;
R := R − λmin + 0.1;
A0 = −R;
Λ(A0) = −0.1000,−0.2250,−1.1024,−1.7496,−2.0355,
−2.4402,−3.1330,−3.3961,−3.9347,−11.9713
⇒ The random shifts do not lead to convergence.
p0 = λ10(A0)(d − 1)
p1 = λ9(A0)(d − 1)
p2 = λ8(A0)(d − 1)
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 22/24
![Page 44: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/44.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Numerical Results – Ai = −R
d t in s residual #it
2 2.7673 9.1353 e−09 219.05 7.8942 9.6503 e−09 98.08 18.9964 9.8650 e−09 84.0
10 18.4739 7.5746 e−09 58.015 27.5661 5.0619 e−09 40.020 32.2409 4.9971 e−09 32.025 40.2462 5.1732 e−09 29.050 76.3225 7.4093 e−09 14.075 159.6627 3.2629 e−09 10.0
100 436.6120 9.1137 e−09 11.0
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 23/24
![Page 45: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/45.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Numerical Results – Ai = −R
d t in s tdmrg in s #it
2 2.7673 0.0148 219.05 7.8942 2.5576 98.08 18.9964 5.4536 84.0
10 18.4739 5.5852 58.015 27.5661 6.3068 40.020 32.2409 7.4044 32.025 40.2462 8.3371 29.050 76.3225 11.8840 14.075 159.6627 18.0581 10.0
100 436.6120 28.8515 11.0
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 23/24
![Page 46: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/46.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Conclusions and Outlook
We have seen
a generalization of the ADI method,
capable of solving tensor Lyapunov and Sylvester equations,
producing solutions of low TT-rank.
Open questions:
more sophisticated shift strategies and
why is the dmrg solver so much faster?
Thank you for your attention.
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 24/24
![Page 47: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/47.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Conclusions and Outlook
We have seen
a generalization of the ADI method,
capable of solving tensor Lyapunov and Sylvester equations,
producing solutions of low TT-rank.
Open questions:
more sophisticated shift strategies and
why is the dmrg solver so much faster?
Thank you for your attention.
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 24/24
![Page 48: ADI for Tensor Structured Equations](https://reader030.vdocuments.net/reader030/viewer/2022032617/55ae48bf1a28ab00188b461c/html5/thumbnails/48.jpg)
ADI ADI for Tensors Numerical Results and Shifts Conclusions
Conclusions and Outlook
We have seen
a generalization of the ADI method,
capable of solving tensor Lyapunov and Sylvester equations,
producing solutions of low TT-rank.
Open questions:
more sophisticated shift strategies and
why is the dmrg solver so much faster?
Thank you for your attention.
Max Planck Institute Magdeburg Thomas Mach, Jens Saak, Tensor-ADI 24/24