primer: 3d grid-based hartree-fock solver lect. 12. tensor ... · tensor-structured hf solver [vk...

15 Lectures on Tensor Numerical Methods forMulti-dimensional PDEs

Lect. 12. Tensor-structured calculation of 3D integrals in 1D complexity.

Primer: 3D grid-based Hartree-Fock solver

Boris Khoromskij & Venera KhoromskaiaShanghai Jiaotong University, Institute of Natural Sciences

April 2017

Max-Planck-Institute for Mathematics in the Sciences, Leipzig, Germany

Boris Khoromskij & Venera Khoromskaia Shanghai Jiaotong University, Institute of Natural Sciences April 2017Tensor numerical methods 12 1 / 24

Outline of the lecture

1 3D integro-differential Hartree-Fock equationElectronic Schrodinger equationStandard Galerkin schemeComputation of the ground state energy

2 Computation of multidimensional integrals in 1D complexityExample: calculation of 3D Hartree potentialMultidimensional tensor-product convolutionTensor-based 3D Laplace operatorNuclear potential operatorTensor factorization of two-electron integrals (TEI)Tensor-based Hartree-Fock solver


Electronic Schrodinger equation

The Hartree-Fock model originates from the electronic Schrodinger equation,

He Ψ = EΨ, (1)

with the Hamiltonian

He = −1

2

N∑i=1

∆i −N∑

i=1

M∑A=1

ZA

xi − xA+

N∑i<j≤N

1

|xi − xj |, xA, xi , xj ∈ R3, (2)

describing energy of a molecule in the framework of the so-called Born-Oppenheimerapproximation (fixed nuclei). M is the number of nuclei, ZA – nuclei charges, N – number ofelectrons.The electronic Schrodinger equation is practically numerically nontractable multidimensionalproblem in R3N (except for Hydrogen atom).

The Hartree-Fock equation is a 3D problem in space variables (solvable!) obtained as a result ofthe minimization of the energy functional for (1).The underlying condition for the wavefunction Ψ for electrons (fermions) Ψ is that it should beantisymmetric, therefore it is parametrized using a single Slater determinant containing theproducts of electronic orbitals.

Ψ =1

N!

∣∣∣∣∣∣∣∣ϕ1(x1) ϕ2(x1) . . . ϕN (x1)ϕ1(x2) ϕ2(x2) . . . ϕN (x2)· · · · · · · · · · · ·

ϕ1(xN ) ϕ2(xN ) . . . ϕN (xN )

∣∣∣∣∣∣∣∣ ,where ϕi (xj ) are the one-electron wavefunctions, i , j = 1, . . .N.


The Hartree-Fock equation

Hartree-Fock equation is a basic model for “ab-initio” calculation of the groundstate energy of molecular systems (first step in calc. of excited states).It is a nonlinear eigenvalue problem ( we consider closed shell molecular systems, Norb = N/2)

Fϕi (x) = λi ϕi (x),

∫R3

ϕiϕj = δij , i = 1, ...,Norb.

where the Fock operator F depends on the density matrix τ(x , y) = 2Norb∑i=1

ϕi (x)ϕi (y), and

electron density ρ(y) = τ(y , y),

Fϕ := [−1

2∆−

M∑ν=1

Zν‖x − aν‖

+

∫R3

ρ(y)

‖x − y‖dy ]ϕ− 1

2

∫R3

τ(x , y)

‖x − y‖ϕ(y)dy .

Numerical solution of this equation is a challenging numerical task due to nonlocal3D and 6D integral transforms and presence of strong cusps in the electron density.Numerical challenges: high accuracy, 3D and 6D singular integrals, strongnonlinearity.


Standard Galerkin scheme

Standart computational scheme developed in quantum chemical community is based onexpansion of the molecular orbitals in Gaussian type basis function, {gµ}1≤µ≤Nb

,

ϕi (x) =

Nb∑µ=1

ciµgµ(x), i = 1, ...,Norb,

that yields the Galerkin system of nonlinear equations for coefficients matrixC = {ciµ} ∈ RNorb×Nb , (and density matrix D = 2CC∗ ∈ RNb×Nb )

F (C)C = SCΛ, Λ = diag(λ1, ..., λNb), C T SC = INb

,

where F (C) = H + J(C) + K(C).For a given basis set the core Hamiltonian H = {hµν}

hµν =1

2

∫R3∇gµ · ∇gνdx +

∫R3

Vc (x)gµgνdx 1 ≤ µ, ν ≤ Nb.

and two-electron integrals (TEI)

bµνκλ =

∫R3

∫R3

gµ(x)gν(x)gκ(y)gλ(y)

‖x − y‖dxdy .

are precomputed analytically using erf-functions for Gaussian-type separable basis functions.


Ground state energy

With precomputed TEI and core Hamiltonian H, the Hartree-Fock EVP

F (C)C = SCΛ

is solved iteratively, using DIIS scheme for providing convergence [Pulay ’80] and updating theGalerkin matrix of the Fock operator, F (C) = H + J(C) + K(C), at every iteration, byrecalculating the Coulomb (Hartree) potential and exchange operators

J(C)µν =

Nb∑κ,λ=1

bµν,κλDκλ, K(C) = −1

2

Nb∑κ,λ=1

bµλ,νκDκλ.

Then the ground state energy of a molecule

EHF = 2

Norb∑i=1

λi −Norb∑i=1

(Ji − Ki

), Ji = (ϕi ,VHϕi ), Ki = (ϕi ,Kϕi ).

Benchmark packages: MOLPRO, GAUSSIAN, CRYSTAL, ...All are based on analytical integration of multidimensional operators usingseparable GTO basis.Require a large number of sharp Gaussians for approximating Slater-type cusps inelectron density of molecules.There are problems for larger systems and heavier atoms.


Tensor-structured HF solver

[VK & Khoromskij, 2008-2014]

Several years ago the idea to solve the Hartree-Fock equationby fully 3D grid based numerical approach seemed to be a fantasy,and the tensor-structured methods did not exist.

In fact, the tensor numerical methods evolved during the work on thischallenging problem.

Glycine, Alanine amino acids, finite lattices in a box.


Grid-based Hartree-Fock solver

Grid-based Hartree-Fock solver [Khoromskaia, Khoromskij ’08 - ’14]Discretization of basis functions:

−b +bx

g

g(1)

gk k

k

ii+1

i−1

gk

(x )1

(1) (1)

xx1,i1,i−1 1,i+1

gk(x )1

x1

x x x x

Computational box [−b, b]3. For single molecules, b = 20au, and n × n × n 3D Cartesiangrid, with n3 ∼ 1015, and step-size h ∼ 10−4 A.

GTO basis is presented on 3D grids, using p.w.c. interpolation,

gµ(x) = pµ(x1, x2, x3)eαµ(x21 +x2

2 +x23 )

≈ I0gµ := gk (x) =3∏`=1

g(`)k (x`) =

3∏`=1

n∑i`=1

g(`)µ (x`,i` )ζ

(`)i`

(x`), ` = 1, 2, 3.

Thus we obtain rank-1 tensors: Gµ = g(1)µ ⊗ g

(2)µ ⊗ g

(3)µ , with

g(`)µ = {g (`)

µ (x`,i` )}ni`=1 ∈ Rn.


Example: 3D integral in 1D complexity (Hartree-Fock and Kohn-Sham eqn.)

Let us consider calculation of the Hartree potential operator in the Hartree-Fock equation.

Jµν :=

∫R3

gµ(x)gν(x)VH (x)dx , µ, ν = 1, . . .Nb x ∈ R3,

It is a 3D convolution operator

VH (x) :=

∫R3

ρ(y)

‖x − y‖dy

with the electron density (a function with multiple strong cusps),

ρ(x) = 2

Norb∑i=1

(ϕi )2, ϕi (x) =

Nb∑µ=1

ciµgµ(x), i = 1, ...,Norb.

In tensor-structured approach for calculating the Hartree potential operator we use grid-basedbasis functions (approximated by 1D p.w.c.finite elements),

gµ(x) ≈ Gµ = g(1)µ ⊗ g

(2)µ ⊗ g

(3)µ , Gµ ∈ Rn×n×n, gµ ∈ Rn

The tensor n × n × n 3D grids, should be fine enough to resolve singularities in ρ(x), x ∈ R3.


Calculation of the 3D Hartree potential in 1D complexity

[Khoromskij ’08], [Khoromskij, Khoromskaia ’08 (SISC 2009)]Products of functions (orbitals squared) are substituted by Hadamard products of vectors

ρ ≈ Θ =

Norb∑i=1

Nb∑µ=1

Nb∑ν=1

Ci,νCi,µ(g(1)µ � g

(1)ν )⊗ (g

(2)µ � g

(2)ν )⊗ (g

(3)µ � g

(3)ν ).

Canonical-to-Tucker transform reduces the rank N2b/2→ Rρ (∼ from 104 ↓ to 102)

Θ→ Θ′ :=

Rρ∑t=1

u(1)t ⊗ u

(2)t ⊗ u

(3)t .

Tensor product convolution (1D FFT) is calculated in O(n log n) complexity instead of requiredO(n3 log n) operations for 3D FFT.

VH ≈ VH = Θ′ ∗ PN =

Rρ∑t=1

RN∑q=1

cmbk

(u

(1)t ∗ p

(1)q

)⊗(u

(2)t ∗ p

(2)q

)⊗(u

(3)t ∗ p

(3)q

),

where Newton kernel is approximated by a canonical tensor of low rank,

PN =

RN∑q=1

p(1)q ⊗ p

(2)q ⊗ p

(3)q .


Tensor-product convolution vs. 3D FFT

Then projection of the 3D potential to 3D basis functions is computed by the scalar products

Jµν ≈ 〈Gµ � Gν ,VH〉 µ, ν = 1, . . .Nb.

n3 5123 10243 20483 40963 81923 163843

FFT3 582.8 ∼ 6000 – – – ∼ 1 yearC ∗ C 1.5 8.8 20.0 61.0 157.5 299.2C2T 5.6 6.9 10.9 20.0 37.9 86.0

Table shows CPU time (in sec) for the computation of VH for H2O.(3D FFT time for n ≥ 1024 is obtained by extrapolation).

[Stenger], [Braess], [Gavrilyuk, Hackbusch, Khoromskij ’08],Canonical tensor for the Newton kernel: [Bertoglio, Khoromskij ’08]Tensor approximation of the Newton kernel using Laplace transform and and (2M + 1)-termsinc-quadrature approximation

1

‖x‖=

∫ ∞0

e−t2‖x‖2dt ≈

M∑k=−M

ck e−t2k‖x‖2

=M∑

k=−M

ck

∏3

`=1e−t2

k x2` 7→ PN .

PN ∈ Rn×n×n, can. rank of PN RN ≤ 30.


Laplacian in a Gaussian basis

Core Hamiltonian part of the Fock operator

Hc = [−1

2∆−

M∑ν=1

Zν

‖x − aν‖

in a Gaussian separable basis,

hµν =1

2

∫R3∇gµ · ∇gνdx +

∫R3

Vc (x)gµgνdx 1 ≤ µ, ν ≤ Nb.

For 3D grid-based tensor-structured calculation of the Laplace operator, we define a set ofpiecewise linear basis functions gk := I1gk , k = 1, ...,Nb, by linear tensor-product interpolationvia the set of product hat functions, {ξi} = ξi1 (x1)ξi2 (x2)ξi3 (x3), i ∈ I, associated with therespective grid-cells (voxels).

−b +b

ξ

x

g

g(1)

gk k

k

ii+1

i−1

(x )

ξ(x )

ξ1

1

(x )1

i

i+1

i−1g

k(x )1

(1) (1)

xx1,i1,i−1 1,i+1

gk(x )1

x1


Tensor-based Laplacian

[Khoromskaia, Andrae, Khoromskij, CPC’12]We approximate the exact Galerkin matrix Ag ∈ RNb×Nb ,

Ag = {akm} := {〈−∆(3)gk , gm〉} ≡ {〈∇(3)gk ,∇(3)gm〉}, k,m = 1, . . .Nb,

using the piecewise linear representation of the basis functions, gk (x) ∈ R3 constructed onN × N × N Cartesian grid. Here ∇(3) denotes the 3D gradient operator. The approximatingmatrix AG is now defined by

Ag ≈ AG = {akm} := {〈−∆(3)gk , gm〉} ≡ {〈∇(3)gk ,∇(3)gm〉}, AG ∈ RNb×Nb .

The Laplace operator applies to a separable function η(x), x = (x1, x2, x3) ∈ R3,η(x) = η1(x1)η2(x2)η3(x3), as follows

∆(3)η(x) =d2η1(x1)

dx21

η2(x2)η3(x3) + η1(x1)d2η2(x2)

dx22

η3(x3) + η1(x1)η2(x2)d2η3(x3)

dx23

,

corresponding to the rank-3 tensor representation of the respective Galerkin stiffness matrix A3

in a tensor basis {ξi (x1)ξj (x2)ξk (x3)}, i , j , k = 1, . . .N,

A3 := A(1) ⊗ S(2) ⊗ S(3) + S(1) ⊗ A(2) ⊗ S(3) + S(1) ⊗ S(2) ⊗ A(3) ∈ RN⊗3×N⊗3.



[Khoromskaia, Andrae, Khoromskij, CPC’12]Here the 1D stiffnesss and mass matrices A(`),S(`) ∈ RN×N , ` = 1, 2, 3, are given byA(`) := {〈∇(1)ξi (x`),∇(1)ξj (x`)〉}N

i,j=1 = 1h

tridiag{−1, 2,−1},,S(`) = {〈ξi , ξj 〉}N

i,j=1 = h6

tridiag{1, 4, 1}, respectively, and ∇(1) = ddx`

.

Since {ξi}Ni=1 are the same for all modes ` = 1, 2, 3, we further assume, A(`) = A1, and

S(`) = S1.

LemmaAssume that the basis functions {gk (x)}, x ∈ R3, k = 1, . . .Nb, are rank-1 separable, i.e.,

gk (x) = g(1)k (x1)g

(2)k (x2)g

(3)k (x3). Then the matrix entries of AG have the representation,

akm = 〈A1g(1)k , g

(1)m 〉〈S1g

(2)k , g

(2)m 〉〈S1g

(3)k , g

(3)m 〉 (3)

+ 〈S1g(1)k , g

(1)m 〉〈A1g

(2)k , g

(2)m 〉〈S1g

(3)k , g

(3)m 〉

+ 〈S1g(1)k , g

(1)m 〉〈S1g

(2)k , g

(2)m 〉〈A1g

(3)k , g

(3)m 〉

= 〈A3Gk ,Gm〉,

where g(`)k , g

(`)m ∈ RN (k,m = 1, . . . ,Nb), are the vectors of collocation coefficients of

{g (`)k (x`)}, ` = 1, 2, 3, and Gk are the corresponding rank-1 3-tensors Gk = g

(1)k ⊗ g

(2)k ⊗ g

(3)k .



Representation for rank-1 tensor Laplace can be simplified by the standardlumping procedure preserving the same approximation error O(h2),

akm = 〈A3,FDGk ,Gm〉,

where A3,FD denotes the finite difference (FD) discrete Laplacian,

A3,FD := A(1) ⊗ I (2) ⊗ I (3) + I (1) ⊗ A(2) ⊗ I (3) + I (1) ⊗ I (2) ⊗ A(3),

with I (`) being the n × n identity matrix and

A(`) := {〈∇(1)ξi (x`),∇(1)ξj (x`)〉}Ni,j=1 =

1

htridiag{−1, 2,−1},

is a 1D stiffnesss mass matrix, ` = 1, 2, 3.


Nuclear potential operator

[Khoromskaia, ’13, CMAM]

Vc (x) = −M∑α=1

Zα‖x − aα‖

, Zα > 0, x , aα ∈ R3

Using the canonical tensor representation of the Newton potential

PR =R∑

q=1

p(1)q ⊗ p(2)

q ⊗ p(3)q ∈ R2n×2n×2n,

and shifting/windowing operator

Wα =W(1)α ⊗W(2)

α ⊗W(3)α .

Pc =M∑α=1

ZαWαPR =M∑α=1

Zα

R∑q=1

W(1)α p(1)

q ⊗W(2)α p(2)

q ⊗W(3)α p(3)

q ∈ Rn×n×n,

vkm =

∫R3

Vc (x)gk (x)gm(x)dx ≈ 〈Gk � Gm,Pc〉, 1 ≤ k,m ≤ Nb.


Tensor-structured core Hamiltonian

−20 −15 −10 −5 0 5 10 15 200

0.01

0.02

0.03

0.04

0.05

0.06

0.07

0.08

0.09

0.1

x−axis (au)

Vc for ethanol molecule (C2H5OH) at levels: z = 0 and z = 0.75 au, n3 = 1015, mesh-sizeh ∼ 2.2 · 10−4 au = 1.164 · 10−4 A = 11.64 fm (10−15 m) ∼ size of atomic radii.

p 13 14 15 16 17N3 = 23p 81923 163843 327683 655363 1310723

Er(AG ) 0.032 0.0083 0.0021 5.2 · 10−4 1.3 · 10−4

|a11 − a11| 208 52 13 3.3 0.82Er(VG ) 0.024 0.0083 0.0011 3.1 · 10−4

|v11 − v11| 14 5.4 0.8 0.3

Ethanol (C2H5OH): abs. accuracy of the Galerkin matrices corresponding to the Laplace,Er(AG ), and the nuclear potential operators, Er(VG ), using the discretized basis of 123Gaussians (abs. values of a11 and v11 are of the order of 104 and 103, respectively).


Grid-based two-electron integrals (TEI)

[Khoromskaia, Khoromskij, Schneider, ’12]

bµνκλ =

∫R3

∫R3

gµ(x)gν(x)gκ(y)gλ(y)

‖x − y‖dxdy = 〈Gµ � Gν ,PN ∗ (Gκ � Gλ)〉n⊗3 ,

Gµ = g(1)µ ⊗ g(2)

µ ⊗ g(3)µ ∈ Rn×n×n.

G (`) =[g(`)µ � g(`)

ν

]1≤µ,ν≤Nb

∈ Rn×N2b ` = 1, 2, 3.

Factorization (“1D density fitting”) by Cholesky decomposition of G (`)G (`)T:

G (`) ∼= U(`)V (`)T, G (`) ∈ Rn×N2

b ,U(`) ∈ Rn×R` , V (`) ∈ RN2b×R` ,

⇒ convolutions are reduced N2b to R` ∼ Nb, (131072× 40000)


Grid-based two-electron integrals (TEI)

[Khoromskaia, Khoromskij, Schneider, ’12]

SVD: G (`) ∼= U(`)V (`)T, ` = 1, 2, 3, with U(`) ∈ Rn×R` and V (`) ∈ RN2

b×R`

The Newton kernel: P(`) ∈ Rn×RN are the factor matrices in the rank-RNcanonical tensor PN ∈ Rn×n×n.

B ∼= Bε :=

RN∑k=1

�3`=1V

(`)M(`)k V (`)T

,

(storage: RN∑3`=1 R

2` + N2

b

∑3`=1 R`), with the Galerkin convolution matrix

M(`)k = U(`)T

(P(`)k ∗n U

(`)) ∈ RR`×R` , k = 1, ...,RN .

storage: O((RG + RN )n), complexity: O(RNR2Gn + RGRN n log n).


Factorization of two-electron integrals (TEI)

0 20 40 60 80 10010

−8

10−6

10−4

10−2

100

0 50 100 150 200 25010

−8

10−6

10−4

10−2

100

glycin, 4k, Nrank=170

0 50 100 150 20010

−8

10−7

10−6

10−5

10−4

10−3

10−2

10−1

100

C2H5OH, 4k, Nrank=123

ε-ranks R`: NH3, glycin, C2H5OH.


Factorization of two-electron integrals (TEI)

The column and diagonal elements in the TEI matrix B :

B(:, j∗) =RN∑k=1�3`=1V (`)M

(`)k V (`)(j∗)

T, B(i , i) =

RN∑k=1�3`=1V (`)(i)M

(`)k V (`)(i)

T,

with i = vec(µ, ν) := (µ− 1)Nb + ν, j = vec(κ, λ), i , j ∈ IN := {1, ...,N}.

Cholesky decomposition (ε-approximation)

B := mat(B) = [bµν,κλ] ≈ LLT .

B Coulomb matrix: given D = vec(D),

vec(J) = BD ≈ L(LT D).

B HF exchange: employing the permuted tensor B = permute(B, [2, 3, 1, 4]),

vec(K) = BD, B = mat(B).

Representation complexity of B using the quantized tensor format can be reduced to O(NbN2orb)

(instead of O(N3b )). (Nb ∼ 10Norb).


Fast convolution via tensor approximation of Green’skernel

Tensor approximation of the Newton kernel using Laplace transform and sinc-quadratures:[Gavrilyuk, Hackbusch, Khoromskij ’08][Bertoglio, Khoromskij ’10]Green’s function for ∆ in R3, via (2M + 1)-term sinc-quadrature approximation

1

‖x‖=

∫ ∞0

e−t2‖x‖2dt ≈

M∑k=−M

ck e−t2k‖x‖2

=M∑

k=−M

ck

∏3

`=1e−t2

k x2` 7→ PN .

PN ∈ Rn×n×n, can. rank of PN RN ≤ 30.

Tensor-product convolution, O(n log n):

[Khoromskij ’08], [Khoromskij, Khoromskaia ’09]

U ∗ PN =

RF∑k=1

RN∑m=1

ck bm(u(1)m ∗ p

(1)k )⊗ (u

(2)m ∗ p

(2)k )⊗ (u

(3)m ∗ p

(3)k )

n3 5123 10243 20483 40963 81923 163843 327683

FFT3 37.5 350.6 ∼ 3500 – – – ∼ 1.2 yearsCRF∗ CRN 2.4 6.7 14.6 44 107 236 535

CPU time (in sec) for TEI: 1‖x‖ ∗ gµgν , µ, ν = 1, ...,Nb, ε = 10−7,

H2O, Nb = 41, Nb(Nb+1)2

7→ RF = 71, RN = 27.Boris Khoromskij & Venera Khoromskaia Shanghai Jiaotong University, Institute of Natural Sciences April 2017Tensor numerical methods 12 22 / 24

Self-consistent iteration for nonlinear EVP

[Khoromskaia, CMAM’13][Khoromskaia, Khoromskij CPC’13]EVP algorithm for black-box solver:

F (C)C = SCΛ, F = H0 + J(C)− K(C),

Initial guess for J = 0, K = 0, F (0) = H0.

solve EVP [H0 + J(C)− K(C)]C = SCΛ .

Update of J(C) and K(C):

B Coulomb matrix: given D = vec(D),

vec(J) = BD ≈ L(LT D).

B HF exchange: using D = 2CC T and B = LLT ,

K(D)µν = −Norb∑i=1

RB∑k=1

(∑λ

Lµλk Cλi )(∑κ

Cκi Lκνk ),

[Lµνk ] = reshape(L, [Nb,Nb,RB ]) is the Nb × Nb × RB -folding of the Cholesky fact. L.

DIIS for providing convergence.


Tensor-based Hartree-Fock solver

[Khoromskaia, CMAM’13], [Khoromskaia, Khoromskij PCCP’15]

Tensor-based Electronic Structure Calculations (TESC) package[Khoromskaia, Khoromskij, 2008-2014]

SCF DIIS iteration for amino acids glycine (C2 H5N O2) with TEI on n3 = 1310723, N2b = 28000

SCF iteration10 20 30 40 50 60

10-10

10-5

100

residual∆ E

0,g

iteration

40 45 50 55

Energ

y (

Hart

ree)

-282.8656

-282.8654

-282.8652

-282.865

-282.8648

-282.8646

SCF iteration0 20 40 60

energ

y (

hart

ree)

-100

-90

-80

-70

-60

-50

last SCF iterations30 35 40

-76.0309

-76.0308

-76.0308

-76.0307

-76.0307

E0 tensor EVP

E0 Molpro

H2O: convergence in energy; last k + 27 iterations.Boris Khoromskij & Venera Khoromskaia Shanghai Jiaotong University, Institute of Natural Sciences April 2017Tensor numerical methods 12 24 / 24

primer: 3d grid-based hartree-fock solver lect. 12. tensor ... · tensor-structured hf solver [vk...

Documents