learning potential energy lanscapes with localized graph kernels … · 2016. 11. 18. · ab initio...

www.enpc.fr

Learning Potential Energy

Lanscapes with Localized Graph

Kernels

IPAM Workshop

Grégoire Ferré – Terry Haut – Kipton Barros

CERMICS - ENPC & Los Alamos National Laboratory

Thursday, November 17th, 2016

Outline

1. Introduction

2. Density kernels and graphs

3. GRAPE

4. Numerical results

Grégoire Ferré, Terry Haut & Kipton Barros CERMICS - ENPC & Los Alamos National Laboratory

Learning Potential Energy Lanscapes with Localized Graph Kernels 2 / 20

1. Introduction

Molecular Dynamics

Langevin Dynamics dqt = ptdt ,

dpt = −∇V(qt )dt −γptdt +√

2γβ dWt ,

Schrödinger equation:

Hψ = Eψ

Molecular Dynamics

Langevin Dynamics dqt = ptdt ,

dpt = −∇V(qt )dt −γptdt +√

2γβ dWt ,

Schrödinger equation:

Hψ = Eψ

Goal of Machine Learning Potentials

Md requires the computation of forces for many configurations

Atomic environment

Objective

Estimate:

• the global energy E(x)

• the forces −∇E(x)

Md requires the computation of forces for many configurations

Atomic environment

Objective

Estimate:

• the global energy E(x)

• the forces −∇E(x)

Potential at an intermediate level

Ab Initio Calculations

Accurate, long to compute, depend on the number of electrons

Machine Learning Potentials and Forces

Ab Initio accuracy with lower computational cost

Analytical Potentials and Forces

Unable to reproduce all the properties of a complex material

Kernel Ridge Regression and localization

Approximated energy E

From a database (xi ,E(xi ))Ni=1 and a kernel K we have

E(x) =N∑i=1

αiK(xi ,x), with α =(K +NλI

)−1EN and Ki ,j = K(xi ,xj ).

Kernel Ridge Regression and localization

Approximated energy E

From a database (xi ,E(xi ))Ni=1 and a kernel K we have

E(x) =N∑i=1

αiK(xi ,x), with α =(K +NλI

)−1EN and Ki ,j = K(xi ,xj ).

Localization

Assume that the energy decomposes as E(xi ) =∑ni

j=1 ε(x ji ), then

E(x) =N∑i=1

nx∑j ′=1

ni∑j=1

K(x j′, x ji ), β = (LKLT +NλI)−1EN ,

where the entries of K are the localized correlations K(x ji , xj ′

i ′ ).

Difficulties

Physical properties:

• the number of atoms may vary,

• invariance with respect to ordering of atoms,

• rotation and translation invariance,

• stability, differentiability, control of the approximation,

• multiscale features.

Difficulties

Extensive literature on this problem:

• symmetry functions (Behler & Parrinello, 2007),

• Smooth Overlap of Atomic Position (Soap, Csányi, Bartók, Kondor,2010),

• Coulomb (Rupp, Tkatchenko, Müller, von Lilienfeld, 2012),

• Internal Vector coordinates (Li, Kermode, De Vita, 2015),

• scattering transform (Hirn, Poilvert, Mallat, 2015),

• Moment Tensor Polynomials (MTP, Shapeev, 2016).

Difficulties

Extensive literature on this problem:

• symmetry functions (Behler & Parrinello, 2007),

• Smooth Overlap of Atomic Position (Soap, Csányi, Bartók, Kondor,2010),

• Coulomb (Rupp, Tkatchenko, Müller, von Lilienfeld, 2012),

• Internal Vector coordinates (Li, Kermode, De Vita, 2015),

• scattering transform (Hirn, Poilvert, Mallat, 2015),

• Moment Tensor Polynomials (MTP, Shapeev, 2016).

2. Density kernels and graphs

Functional Representation of Atomic Position

−1 −0.5 0 0.5 1

(a)√

2πσe− ‖r‖

−1 −0.5 0 0.5 1

·10−2

xϕ(x)

(b) e− 1σ2−‖r‖2 1{‖r‖<σ }

−1 −0.5 0 0.5 1

(c) σ−11{‖r‖<σ }(r)

−1 −0.5 0 0.5 1

(d) Hat function

Figure: Functional representation of one atom

Function ϕσ : R3 7→R

Functional Representation of Atomic Position

−1 −0.5 0 0.5 1

(a)√

2πσe− ‖r‖

−1 −0.5 0 0.5 1

·10−2

xϕ(x)

(b) e− 1σ2−‖r‖2 1{‖r‖<σ }

−1 −0.5 0 0.5 1

(c) σ−11{‖r‖<σ }(r)

−1 −0.5 0 0.5 1

(d) Hat function

Figure: Functional representation of one atom

Function ϕσ : R3 7→R

«Smoothed» atomic position

Functional Representation of Atomic Configuration

Configuration x = (ri ,zi )ni=1 ∈ (R3 ×R+)n .

Associated density : ρ(r) =n∑

wziϕσ (r − ri ), ϕσ (r) = e− ‖r‖

Functional Representation of Atomic Configuration

Configuration x = (ri ,zi )ni=1 ∈ (R3 ×R+)n .

Associated density : ρ(r) =n∑

wziϕσ (r − ri ), ϕσ (r) = e− ‖r‖

Density kernel and graphs

Similarity and kernel between two densities ρ, ρ′ :

S(ρ,ρ′) =

3ρρ′ , K(ρ,ρ′) =

∫SO(3)

S(ρ,Rρ′)pdR , for some p ≥ 2.

S(ρ,ρ′) =

3ρρ′ , K(ρ,ρ′) =

∫SO(3)

Actually,

S(ρ,ρ′) =n∑

n ′∑j=1

wziwzj exp

− (ri − r ′j )2

= Tr(AAT ), where

Ai ,j =√wziwzjϕ2σ

(|ri − r ′j |

), is the

adjacency matrix of a bipartite graph.

S(ρ,ρ′) =

3ρρ′ , K(ρ,ρ′) =

∫SO(3)

Actually,

S(ρ,ρ′) =n∑

n ′∑j=1

wziwzj exp

− (ri − r ′j )2

= Tr(AAT ), where

Ai ,j =√wziwzjϕ2σ

(|ri − r ′j |

), is the

adjacency matrix of a bipartite graph. Figure: Associated bipartite graph

.Grégoire Ferré, Terry Haut & Kipton Barros CERMICS - ENPC & Los Alamos National Laboratory

3. GRAPE

Graph Representation

Define the adjacency matrix of a configuration as:

Ai ,j =ωziωzjϕσ (|ri − rj |)fcut(|ri |)fcut(|rj |)

Figure: Graph representation of amethane molecule, CH4.

Graph Representation

Define the adjacency matrix of a configuration as:

Ai ,j =ωziωzjϕσ (|ri − rj |)fcut(|ri |)fcut(|rj |)

Graph kernels since 2002 (Gärtner):

• spectral properties (Kondor et. al,2002),

• shortest paths (Borgwadt et al.,2005),

• graphlets (Shervashidze &Vishwanathan, 2009),

• random walks (Vishwanathan,Schraudolph, Kondor & Borgwardt,2010),

• functional embedding (Shrivastava &Li, 2014).

Figure: Graph representation of amethane molecule, CH4.

Random walk graph kernels

Adjacency matrices A , A ′ as generators of Markov processes.

Figure: Two systems constituted of 3 and 4particles represented as graphs with weights wikand w′jl . The direct product graph representedbelow has weights wij ,kl = wikw

′jl .

Random walk graph kernels

Adjacency matrices A , A ′ as generators of Markov processes.

Figure: Two systems constituted of 3 and 4particles represented as graphs with weights wikand w′jl . The direct product graph representedbelow has weights wij ,kl = wikw

′jl .

Kronecker product:

A ⊗A ′ =

A1,1A ′ . . . A1,nA ′...

...An ,1A ′ . . . An ,nA ′

,Idea:

• starting distributions p , p ′ ,

• stopping distributions q , q ′ ,

• compare the random walksthrough the power iterationsAk , A ′k .

Choose p , p ′ , q , q ′ uniform.

GRaphe APproximated Energy (GRAPE)

Sum over walks of all lengths with penalization µk :

k(A ,A ′) = (q ⊗q ′)T ∞∑k=0

µk (A ⊗A ′)kp ⊗ p ′ .

GRaphe APproximated Energy (GRAPE)

Sum over walks of all lengths with penalization µk :

k(A ,A ′) = (q ⊗q ′)T ∞∑k=0

µk (A ⊗A ′)kp ⊗ p ′ .

For example µk = γk /k !:

k(A ,A ′) = q ⊗q ′eγA⊗A′p ⊗ p ′ .

Renormalization step:

K(A ,A ′) =

k(A ,A ′)√k(A ,A)k(A ′ ,A ′)

ζ ,for some ζ > 0, and derivative:

∂r ′k(A ,A ′) = q ⊗q ′(γA ⊗∂r ′A ′)eγA⊗A′p ⊗ p ′ .

4. Numerical results

GRAPE (Graph Approximated Potential)

Procedure for molecules:

• localize molecules into local environments

• define adjacency matrices inspired from SOAP

• run the localized kernel method with the exponential kernel

• adjust the numerical parameters on a validation set

Benchmark on molecular Data

Fitting the atomization energy of molecules constituted of C, N, O, S,and H (QM7 database).

-2000 -1500 -1000 -500

y [kcal/m

Reference Energy [kcal/mol]

N 100 300 500

CoulombMAE 25.6 19.8 17.9

RMSE 50.8 33.5 27.1

GRAPEMAE 11.2 10.1 9.6

RMSE 14.9 13.9 13.3

SOAPMAE 15.6 11.3 10.4

RMSE 21.0 15.6 14.5Mean average error and root mean squareerror for Coulomb, GRAPE and SOAP, where

N is the training set size.

Conclusion & Tracks

Conlusion

Results:

• density-geometry description

• graph kernel for energy regression

• satisfying results on a standard dataset

Future works• include more physics in the graph design

• uncomplete-overcomplete kernel, multiscale

• forces computation

• transferability capacities

Conclusion & Tracks

Conlusion

Results:

• density-geometry description

• graph kernel for energy regression

• satisfying results on a standard dataset

Future works• include more physics in the graph design

• uncomplete-overcomplete kernel, multiscale

• forces computation

• transferability capacities

learning potential energy lanscapes with localized graph kernels … · 2016. 11. 18. · ab initio...

Documents

power point lanscapes of spain

spanish ab initio 2013 november paper 1 question booklet ·...

chinese ab initio vocabulary...

kernels - vermont foodbank

bioinformatics kernels relations

studies of religion 1 - religious lanscapes in australia

graph kernels - university of...

autoencoders& kernels

english ab initio language-specific...

extensible kernels

graph kernels · 2010-07-01 · graph kernels trast, in...

kernels - arxiv · among di erent kernels such security...

daftar pustaka ab initio, (2008). a practical introduction...

12 kernels from generative models - people › ~jordan ›...

lanscapes of taiwan - by kangour 06

group 2 / groupe 2 / grupo 2 spanish...

reproducing kernels in hilbert spaces -...

troubleshooting e1 kernels

kernels review

ab initio methods