Advancing first-principle symmetry-guidednuclear modeling for studies of nucleosynthesis

and fundamental symmetries in nature

NCSA Blue Waters Symposium for Petascale Science and Beyond, 2018

Students & Postdocs

Collaborators

Nuclear PhysicsNuclear Physics

neutron

proton

Accurate tests of fundamentals laws of nature

Nuclear interactions

Discovery potential in nuclear physics

Residual strong force → highly complex

two-, three- and four-body forces

Properties and reactions of nuclei at the edge of their existence

Universal internucleon interaction derived from QCD

Applications of Applications of Nuclear Structure & Reaction ModelingNuclear Structure & Reaction Modeling

Nuclear reactions for applied energy studies

Neutrino studies Dark matter experiments

Fusion energy

Standard Model & physics beyond

X-ray bursts

Astrophysics: thermonuclear processes in the cosmos

Neutrino & Cosmology research

Ab initioAb initio Approaches to Nuclear Structure and Reactions Approaches to Nuclear Structure and Reactions

Many-body dynamics Nuclear reactions

Realistic nuclear potentialmodels

Strong interaction

Energy

3H3H 4He

2H n

¼0

wave functions nuclear properties

reaction ratescross sections

Ab InitioAb Initio No-Core Shell Model No-Core Shell Model

Solve the non-relativistic quantum problem of A-interacting nucleonsGoal:

Calculate nuclear properties from resulting eigenvectors

Use resulting eigenvectors for ab initio nuclear reaction studies

2. Compute Hamiltonian matrix

3. Find lowest-lying eigenvalues and eigenvectors [Lanczos algorithm]

1+2+

0+

4+

1. Choose physically relevant model space and construct its basis

jÃi =NX

i = 1

ci jÁi i

Additional steps

Key ChallengeKey Challenge

Limits application of ab initio studies to lightest nuclei

Use partial symmetries of nuclear collective motion to adopt smaller physically relevant model spaces

Computational Scale Explosion

Why symmetry-adapted approach?

Why Blue Waters?

Large aggregate memory and amount of memory per node (64GB)

High peak memory bandwidth (102.4 GB/s)

[courtesy of Pieter Maris]

Many-nucleon basis natural for description of many-body dynamics of nuclei

N

Sp SnS

L

number of harmonic oscillator excitations

total proton, total neutron and total intrinsic spins

deformation

rotation

Many-nucleon basis natural for description of many-body dynamics of nuclei

Symmetry-Adapted No-Core Shell ModelSymmetry-Adapted No-Core Shell Model

MPI/OpenMP Implementation of Symmetry-Adapted No-Core Shell ModelMPI/OpenMP Implementation of Symmetry-Adapted No-Core Shell Model

Excellent scalability

1 process 15 processes 378 processes 37,950 processes

Load balanced computations

Computational effort: 80 % - computing matrix elements 10% - solving eigenvalue problem

Implementation

C++/Fortran code parallelized using hybrid MPI/OpenMP

Open source: https://sourceforge.net/p/lsu3shell/home/Home/

(0 0)

(1 1)

(0 3)

(3 0)

(2 2)

(1 4)

(4 1)

(3 3)

(0 6)

(6 0)

(2 5)

(5 2)

(4 4)

(7 1)

(6 3)

(9 0)

(8 2)

(10 1)

(12 0)

0.0%

0.3%

0.5%

(0 1)

(2 0)

0%

60%

(1 0)

(0 2)

(2 1)

(4 0)

0%

7%

14%

(0 0)

(1 1)

(0 3)

(3 0)

(2 2)

(4 1)

(6 0)

0%

5%

10%

(0 1)

(2 0)

(1 2)

(3 1)

(0 4)

(2 3)

(5 0)

(4 2)

(6 1)

(8 0)

0%

2%

4%

(1 0)

(0 2)

(2 1)

(1 3)

(4 0)

(3 2)

(0 5)

(2 4)

(5 1)

(4 3)

(7 0)

(6 2)

(8 1)

(10 0)

0.00%

0.75%

1.50%

(0 1)

(2 0)

(1 2)

(3 1)

(0 4)

(2 3)

(5 0)

(4 2)

(1 5)

(3 4)

(6 1)

(0 7)

(5 3)

(2 6)

(4 5)

(8 0)

(7 2)

(6 4)

(9 1)

(8 3)

(11 0)

(10 2)

(12 1)

(14 0)

0.00%

0.06%

0.11%

remaining Sp Sn SSp=1/2 Sn=3/2 S=2Sp=3/2 Sn=1/2 S=2Sp=3/2 Sn=3/2 S=3Sp=1/2 Sn=1/2 S=1

60.77%

18.82%

11.63%

5.37%

2.28%

0.85%

0.27%6L i : 1+gs

Discovery: Emergence of Simple Patterns in Complex NucleiDiscovery: Emergence of Simple Patterns in Complex Nuclei

Dytrych, Launey, Draayer, et al., PRL 111 (2013) 252501

Low spin

Large deformation

Key features of nuclear structure

Model space truncation

SA-NCSM on BlueWaters: reaching towards medium mass nucleiSA-NCSM on BlueWaters: reaching towards medium mass nuclei

Nuclear densityExcitation Spectrum

Binding energy

Complete space:

Symmetry-adapted space:

SA-NCSM on BlueWaters: reaching towards medium mass nucleiSA-NCSM on BlueWaters: reaching towards medium mass nuclei

Complete space dimension:

dimension:

complete selected

Novae and X-ray bursts

Astrophysical NucleosynthesisAstrophysical Nucleosynthesis

Astrophysical NucleosynthesisAstrophysical Nucleosynthesis

Calculation of reaction ratesCalculation of reaction rates

SA-NCSM

Probability to find cluster structure

Nuclear reaction:

Calculation of reaction ratesCalculation of reaction rates

Blue Waters

Probability to find cluster structure

astrophysicalsimulation

Nuclear reaction:

Response functionResponse function

Nucleus response to external probe (photon, neutrino, etc ..)

SA-NCSM

SA-NCSM

New approach: SA-NCSM + Lorentz Integral Transform Method

Response functions for neutrino studiesResponse functions for neutrino studies

Response functions – input for neutrino experiments

Nuclear input - 2nd largest source of uncertainties

SA-NCSM + LIT: preliminary results

: component of neutrino detectors

Code improvementsCode improvements

Dynamic memory allocation optimizations

Dynamic allocation – generally slow, and dependend on malloc implementation.

malloc replacement

We tested tcmalloc (Google), jemalloc (Facebook), tbbmalloc (Intel)

SA-NCSM – lot of concurrent small allocations

tcmalloc – best performance & memory footprint decrease

Memory pooling

Small buffer optimizations

allocations of a lot of small objects is inneficient request a big block of memory and do bookkeeping ourselves

Boost.Pool provides convenient classes for managing memory pools

use a small static buffer for a small number of elements, and only requests dynamic memory when we go over the specified threshold.

Code improvementsCode improvements

Nearly factor of 2 speedup

10-15% decrease of total memory footprint

20Ne J=0 20Ne J=2 16O J=00

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

1.8

2

legacy codeoptimized code

spee

dup

SummarySummary

Key challengesDescription of 99.9% mass of the Universe

Why it mattersUltimate source of energy in the Universe

Aggregate memory and high memory bandwidth

Why Blue Waters

Many papers in top journals and reaching beyond what competitives theories could accomplish

Accomplishments

Blue Waters team contributions Excellent support and guidance as needed

Broader impactsTraining next generation of STEM workforce

Shared DataCodes and results publicly available

Productshttps://sourceforge.net/p/lsu3shell/home/Home/