open source image based parallel linear algebra solver ...€¦ · open source image based parallel...
TRANSCRIPT
James Le Houx Denis Kramer
OPEN source IMage based PArallel Linear Algebra solver
IBSIM-4i 2020 James Le Houx
James Le Houx Denis Kramer
μCT
Iridis5
?
1. Recap
What?
One of the key design aims was to improve speed of calculation over existing open source methods. This was achieved by designing the software with MPI parallelisation from the start, solving the problem directly on the voxel dataset (removing meshing) and to make use of the linear scaling, open source linear algebra library, Hypre.
Why?
OpenImpala was originally designed to fit into the multi-scale modelling framework by quantifying transport processes of the microstructure, specifically for lithium-ion porous battery electrodes. Fig 1: Diagram of multi-scale modelling methods used at
different length scales for modelling lithium-ion batteries Fig 2: Graph showing time to solution to solve diffusion problem in a lithium iron phosphate electrode (106 voxels) for Taufactor (black) and OpenImpala (blue).
How?
OpenImpala uses segmented 3D image data directly as the computational domain using the Finite Differences method. The code uses a Fortran kernel for efficient array manipulation and a C++ wrapper for ease of use.
Fig 3: Diagram of an example workflow: lithium iron phosphate electrode, imaged using micro computed tomography and post-processed using Avizo to obtain segmented 2-phase image data, this data is then used as the computational domain in OpenImpala and the calculations are run on an HPC system, Iridis5, obtaining concentration profiles, porosity, tortuosity and diffusion gradients.
James Le Houx Denis Kramer
2. Software Updates
Software Integration
To facilitate the link between different levels of multi-scale modelling, OpenImpala now directly exports parameterisedelectrode files for use with popular battery continuum models: Dandeliion and PyBaMM.
Containerised Deployment
OpenImpala now uses containerised deployment, specifically Singularity to allow for full use even without root access, as is often the case on HPC systems. This ensures compatibility of dependent libraries as well as improved deployment across different HPC architectures.
Fig 1: Diagram demonstrating how OpenImpala runs within a container on the host system.
Fig 2: Software logos for directly compatible continuum models
Additional I/O Features
Addition of .dat files to input files, improving versatility, addition of parameterisationoutput files.
Fig 3: Diagram of an I/O features for OpenImpala
Speed Improvement
This has been achieved through code refactoring and by efficient division of the computational domain into computable boxes:
.tiff
.dat
.py
.txt
Linux Kernel
Singularity Container
Where x is the minimum integer that when cubed divides by the number of cores to give an integer.
Fig 4: Time taken to solve weak scaling diffusion problem, 2 x 106 voxels, comparing original and efficient box sizing
James Le Houx Denis Kramer
3. Use Case
CT Resolution Validation
Image-based modelling provides a possible path to computing relevant microstructural properties in lithium ion batteries. But how do we know the results are real? Our recent study explores the effect of image resolution on calculation of structural properties. Two datasets were analysed of the same microstructure, using two resolutions: a 499 cubic voxel set and a 998 cubic voxel set. Fig 2 shows that the calculations for the 499 voxel set that would have previously taken hours and days to solve, can now be solved in around 2 minutes with suitable computing architecture. Furthermore, large scale datasets typical of modern tomography, 109 voxels, can now be analysed to obtain real transport coefficients, using OpenImpala.
Fig 3: Variability of tortuosity calculated for two resolution scans, 400 nm and 801 nm, and using the Bruggeman relationship across six data samples.
Fig 2: Scaling performance graphs plotting number of computing cores on the x axis against (left) time taken to solve diffusion problem and (right) parallel efficiency for 499 and 998 cubic voxel datasets.
100
1000
10000
100000
0 200 400 600
Tim
e Ta
ken
/ s
Number of Computing Cores
1.24 x 10⁸ Voxels9.94 x 10⁸ Voxels
0%
20%
40%
60%
80%
100%
0 200 400 600
Para
llel E
ffic
ienc
y
Number of Computing Cores
1.24 x 10⁸ Voxels9.94 x 10⁸ Voxels
Fig 1: Concentration plots of 499 voxel cubic dataset for (a) x-, (b) y-, and (c) z-directions
Results
James Le Houx Denis Kramer
4. Special Session Discussion
Publications
Le Houx, James, and Denis Kramer. "Physics based modelling of porous lithium ion battery electrodes—A review." Energy Reports 6 (2020): 1-9.Le Houx, James, et al. "Effect of Tomography Resolution on Calculation of Microstructural Properties for Lithium Ion Porous Electrodes." ECS Transactions 97.7 (2020): 255.Ojha, Manoranjan, et al. "Lithium titanate/pyrenecarboxylic acid decorated carbon nanotubes hybrid-Alginate gel supercapacitor." Electrochimica Acta 309 (2019): 253-263.
Do you have an application you could use OpenImpala for?
Do you need to validate your CT/ FIB-SEM results?
Do you have a software you want to integrate with OpenImpala?
Do you need additional features (I/O)? Do you want to discuss my
publications (below)?
Rough(!) Plan13:20 – 13:35 Introduction
13:35‐13:50 Worked Example13:50‐14:10 Discussion