learning potential energy lanscapes with localized graph kernels … · 2016. 11. 18. · ab initio...

www.enpc.fr

Learning Potential Energy

Lanscapes with Localized Graph

Kernels

IPAM Workshop

Grégoire Ferré – Terry Haut – Kipton Barros

CERMICS - ENPC & Los Alamos National Laboratory

Thursday, November 17th, 2016

Outline

1. Introduction

2. Density kernels and graphs

3. GRAPE

4. Numerical results

Grégoire Ferré, Terry Haut & Kipton Barros CERMICS - ENPC & Los Alamos National Laboratory

Learning Potential Energy Lanscapes with Localized Graph Kernels 2 / 20

1. Introduction

Molecular Dynamics

Langevin Dynamics dqt = ptdt ,

dpt = −∇V(qt )dt −γptdt +√

2γβ dWt ,

Schrödinger equation:

Hψ = Eψ



Goal of Machine Learning Potentials

Md requires the computation of forces for many configurations

Atomic environment

Objective

Estimate:

• the global energy E(x)

• the forces −∇E(x)



Goal of Machine Learning Potentials

Potential at an intermediate level

Ab Initio Calculations

Accurate, long to compute, depend on the number of electrons

Machine Learning Potentials and Forces

Ab Initio accuracy with lower computational cost

Analytical Potentials and Forces

Unable to reproduce all the properties of a complex material



Kernel Ridge Regression and localization

Approximated energy E

From a database (xi ,E(xi ))Ni=1 and a kernel K we have

E(x) =N∑i=1

αiK(xi ,x), with α =(K +NλI

)−1EN and Ki ,j = K(xi ,xj ).



Kernel Ridge Regression and localization

Approximated energy E

From a database (xi ,E(xi ))Ni=1 and a kernel K we have

E(x) =N∑i=1

αiK(xi ,x), with α =(K +NλI

)−1EN and Ki ,j = K(xi ,xj ).

Localization

Assume that the energy decomposes as E(xi ) =∑ni

j=1 ε(x ji ), then

E(x) =N∑i=1

βi

nx∑j ′=1

ni∑j=1

K(x j′, x ji ), β = (LKLT +NλI)−1EN ,

where the entries of K are the localized correlations K(x ji , xj ′

i ′ ).



Difficulties

Physical properties:

• the number of atoms may vary,

• invariance with respect to ordering of atoms,

• rotation and translation invariance,

• stability, differentiability, control of the approximation,

• multiscale features.



Difficulties

Physical properties:

• the number of atoms may vary,

• invariance with respect to ordering of atoms,

• rotation and translation invariance,

• stability, differentiability, control of the approximation,

• multiscale features.

Extensive literature on this problem:

• symmetry functions (Behler & Parrinello, 2007),

• Smooth Overlap of Atomic Position (Soap, Csányi, Bartók, Kondor,2010),

• Coulomb (Rupp, Tkatchenko, Müller, von Lilienfeld, 2012),

• Internal Vector coordinates (Li, Kermode, De Vita, 2015),

• scattering transform (Hirn, Poilvert, Mallat, 2015),

• Moment Tensor Polynomials (MTP, Shapeev, 2016).



2. Density kernels and graphs

Functional Representation of Atomic Position

−1 −0.5 0 0.5 1

0

0.25

0.5

0.75

1

x

ϕ(x)

(a)√

2πσe− ‖r‖

2

2σ2

−1 −0.5 0 0.5 1

0

0.5

1

1.5

·10−2

xϕ(x)

(b) e− 1σ2−‖r‖2 1{‖r‖<σ }

−1 −0.5 0 0.5 1

0

0.25

0.5

0.75

1

1.25

x

ϕ(x)

(c) σ−11{‖r‖<σ }(r)

−1 −0.5 0 0.5 1

0

0.2

0.4

0.6

x

ϕ(x)

(d) Hat function

Figure: Functional representation of one atom

Function ϕσ : R3 7→R



Functional Representation of Atomic Position

−1 −0.5 0 0.5 1

0

0.25

0.5

0.75

1

x

ϕ(x)

(a)√

2πσe− ‖r‖

2

2σ2

−1 −0.5 0 0.5 1

0

0.5

1

1.5

·10−2

xϕ(x)

(b) e− 1σ2−‖r‖2 1{‖r‖<σ }

−1 −0.5 0 0.5 1

0

0.25

0.5

0.75

1

1.25

x

ϕ(x)

(c) σ−11{‖r‖<σ }(r)

−1 −0.5 0 0.5 1

0

0.2

0.4

0.6

x

ϕ(x)

(d) Hat function

Figure: Functional representation of one atom

Function ϕσ : R3 7→R

«Smoothed» atomic position



Functional Representation of Atomic Configuration

Configuration x = (ri ,zi )ni=1 ∈ (R3 ×R+)n .

Associated density : ρ(r) =n∑

i=1

wziϕσ (r − ri ), ϕσ (r) = e− ‖r‖

2

2σ2



Density kernel and graphs

Similarity and kernel between two densities ρ, ρ′ :

S(ρ,ρ′) =

∫R

3ρρ′ , K(ρ,ρ′) =

∫SO(3)

S(ρ,Rρ′)pdR , for some p ≥ 2.





S(ρ,ρ′) =

∫R

3ρρ′ , K(ρ,ρ′) =

∫SO(3)


Actually,

S(ρ,ρ′) =n∑

i=1

n ′∑j=1

wziwzj exp

− (ri − r ′j )2

4σ2

= Tr(AAT ), where

Ai ,j =√wziwzjϕ2σ

(|ri − r ′j |

), is the

adjacency matrix of a bipartite graph.





S(ρ,ρ′) =

∫R

3ρρ′ , K(ρ,ρ′) =

∫SO(3)


Actually,

S(ρ,ρ′) =n∑

i=1

n ′∑j=1

wziwzj exp

− (ri − r ′j )2

4σ2

= Tr(AAT ), where

Ai ,j =√wziwzjϕ2σ

(|ri − r ′j |

), is the

adjacency matrix of a bipartite graph. Figure: Associated bipartite graph

.Grégoire Ferré, Terry Haut & Kipton Barros CERMICS - ENPC & Los Alamos National Laboratory


3. GRAPE

Graph Representation

Define the adjacency matrix of a configuration as:

Ai ,j =ωziωzjϕσ (|ri − rj |)fcut(|ri |)fcut(|rj |)

Graph kernels since 2002 (Gärtner):

• spectral properties (Kondor et. al,2002),

• shortest paths (Borgwadt et al.,2005),

• graphlets (Shervashidze &Vishwanathan, 2009),

• random walks (Vishwanathan,Schraudolph, Kondor & Borgwardt,2010),

• functional embedding (Shrivastava &Li, 2014).

Figure: Graph representation of amethane molecule, CH4.



Random walk graph kernels

Adjacency matrices A , A ′ as generators of Markov processes.

Figure: Two systems constituted of 3 and 4particles represented as graphs with weights wikand w′jl . The direct product graph representedbelow has weights wij ,kl = wikw

′jl .



Random walk graph kernels

Adjacency matrices A , A ′ as generators of Markov processes.

Figure: Two systems constituted of 3 and 4particles represented as graphs with weights wikand w′jl . The direct product graph representedbelow has weights wij ,kl = wikw

′jl .

Kronecker product:

A ⊗A ′ =

A1,1A ′ . . . A1,nA ′...

...An ,1A ′ . . . An ,nA ′

,Idea:

• starting distributions p , p ′ ,

• stopping distributions q , q ′ ,

• compare the random walksthrough the power iterationsAk , A ′k .

Choose p , p ′ , q , q ′ uniform.



GRaphe APproximated Energy (GRAPE)

Sum over walks of all lengths with penalization µk :

k(A ,A ′) = (q ⊗q ′)T ∞∑k=0

µk (A ⊗A ′)kp ⊗ p ′ .



GRaphe APproximated Energy (GRAPE)

Sum over walks of all lengths with penalization µk :

k(A ,A ′) = (q ⊗q ′)T ∞∑k=0

µk (A ⊗A ′)kp ⊗ p ′ .

For example µk = γk /k !:

k(A ,A ′) = q ⊗q ′eγA⊗A′p ⊗ p ′ .

Renormalization step:

K(A ,A ′) =

k(A ,A ′)√k(A ,A)k(A ′ ,A ′)

ζ ,for some ζ > 0, and derivative:

∂r ′k(A ,A ′) = q ⊗q ′(γA ⊗∂r ′A ′)eγA⊗A′p ⊗ p ′ .



4. Numerical results

GRAPE (Graph Approximated Potential)

Procedure for molecules:

• localize molecules into local environments

• define adjacency matrices inspired from SOAP

• run the localized kernel method with the exponential kernel

• adjust the numerical parameters on a validation set



Benchmark on molecular Data

Fitting the atomization energy of molecules constituted of C, N, O, S,and H (QM7 database).

-2000

-1500

-1000

-500

-2000 -1500 -1000 -500

GR

AP

E R

egre

ssio

n E

nerg

y [kcal/m

ol]

Reference Energy [kcal/mol]

N 100 300 500

CoulombMAE 25.6 19.8 17.9

RMSE 50.8 33.5 27.1

GRAPEMAE 11.2 10.1 9.6

RMSE 14.9 13.9 13.3

SOAPMAE 15.6 11.3 10.4

RMSE 21.0 15.6 14.5Mean average error and root mean squareerror for Coulomb, GRAPE and SOAP, where

N is the training set size.



Conclusion & Tracks

Conlusion

Results:

• density-geometry description

• graph kernel for energy regression

• satisfying results on a standard dataset

Future works• include more physics in the graph design

• uncomplete-overcomplete kernel, multiscale

• forces computation

• transferability capacities



learning potential energy lanscapes with localized graph kernels … · 2016. 11. 18. · ab initio...

Documents