illuminating openmp + mpi performance€¦ · cpi-mpi.c:48 cpi-mpi.c:84 cpi-mpi.c:109 cpi-mpi.c:97...

Illuminating OpenMP + MPI Performance Beau Paisley, [email protected]

Upload: others

Post on 02-Oct-2020

79 views

Category:

Documents

0 download

Report

Download

Embed Size (px):

TRANSCRIPT

Illuminating OpenMP + MPI Performance

Beau Paisley, [email protected]

HPC means being able to work productively on remote machines

Page 3: Illuminating OpenMP + MPI Performance€¦ · cpi-mpi.c:48 cpi-mpi.c:84 cpi-mpi.c:109 cpi-mpi.c:97 1.0% cpi-mpi [program] main main [OpenMP region O] MPI Finalize MPI Reduce Showing

Calculating pi with 1 thread, 1 process

Page 4: Illuminating OpenMP + MPI Performance€¦ · cpi-mpi.c:48 cpi-mpi.c:84 cpi-mpi.c:109 cpi-mpi.c:97 1.0% cpi-mpi [program] main main [OpenMP region O] MPI Finalize MPI Reduce Showing

Profile with 1 process, 1 thread

Page 5: Illuminating OpenMP + MPI Performance€¦ · cpi-mpi.c:48 cpi-mpi.c:84 cpi-mpi.c:109 cpi-mpi.c:97 1.0% cpi-mpi [program] main main [OpenMP region O] MPI Finalize MPI Reduce Showing

Using MAP to analyze our 1 process, 1 thread run

Page 6: Illuminating OpenMP + MPI Performance€¦ · cpi-mpi.c:48 cpi-mpi.c:84 cpi-mpi.c:109 cpi-mpi.c:97 1.0% cpi-mpi [program] main main [OpenMP region O] MPI Finalize MPI Reduce Showing

Zooming in on the IO portion of our run

Page 7: Illuminating OpenMP + MPI Performance€¦ · cpi-mpi.c:48 cpi-mpi.c:84 cpi-mpi.c:109 cpi-mpi.c:97 1.0% cpi-mpi [program] main main [OpenMP region O] MPI Finalize MPI Reduce Showing

Always be thinking about vectorization

Page 8: Illuminating OpenMP + MPI Performance€¦ · cpi-mpi.c:48 cpi-mpi.c:84 cpi-mpi.c:109 cpi-mpi.c:97 1.0% cpi-mpi [program] main main [OpenMP region O] MPI Finalize MPI Reduce Showing

While still connected to the server we can switch to the debugger

Page 9: Illuminating OpenMP + MPI Performance€¦ · cpi-mpi.c:48 cpi-mpi.c:84 cpi-mpi.c:109 cpi-mpi.c:97 1.0% cpi-mpi [program] main main [OpenMP region O] MPI Finalize MPI Reduce Showing

Adding some OpenMP pragmas to multithread

Page 10: Illuminating OpenMP + MPI Performance€¦ · cpi-mpi.c:48 cpi-mpi.c:84 cpi-mpi.c:109 cpi-mpi.c:97 1.0% cpi-mpi [program] main main [OpenMP region O] MPI Finalize MPI Reduce Showing

Adding MPI support and submitting to and HPC batch system

Page 11: Illuminating OpenMP + MPI Performance€¦ · cpi-mpi.c:48 cpi-mpi.c:84 cpi-mpi.c:109 cpi-mpi.c:97 1.0% cpi-mpi [program] main main [OpenMP region O] MPI Finalize MPI Reduce Showing

Analyzing our OpenMP + MPI results

Page 12: Illuminating OpenMP + MPI Performance€¦ · cpi-mpi.c:48 cpi-mpi.c:84 cpi-mpi.c:109 cpi-mpi.c:97 1.0% cpi-mpi [program] main main [OpenMP region O] MPI Finalize MPI Reduce Showing

Why all this synchronization?

Page 13: Illuminating OpenMP + MPI Performance€¦ · cpi-mpi.c:48 cpi-mpi.c:84 cpi-mpi.c:109 cpi-mpi.c:97 1.0% cpi-mpi [program] main main [OpenMP region O] MPI Finalize MPI Reduce Showing

The workflow of pi from serial to HPC

caption

Page 14: Illuminating OpenMP + MPI Performance€¦ · cpi-mpi.c:48 cpi-mpi.c:84 cpi-mpi.c:109 cpi-mpi.c:97 1.0% cpi-mpi [program] main main [OpenMP region O] MPI Finalize MPI Reduce Showing

https://www.allinea.com/products/downloads/free-trial

Hybrid programming MPI+OpenMP · MPI+OpenMP hybrid paradigm is the trend for clusters with SMP architecture. Elegant in concept: use OpenMP within the node and MPI between nodes,

Hybrid OpenMP and MPI Programming and Tuning

MPI n OpenMP

Performance Analysis of Large-scale OpenMP and Hybrid MPI ... · Performance Analysis of Large-scale OpenMP and Hybrid MPI/OpenMP Applications with VampirNG Holger Brunst1 and Bernd

Paralelização de códigos, OpenMP, MPI e GPUs 2 – OpenMP, MPI, … › ast › pp_para_iiwcca_2.pdf · 2014-06-11 · OpenMP - Características Paralelismo apenas por memória

Hybrid MPI+OpenMP Parallel MD

MPI and OpenMP · •In general, MPI + OpenMP does not improve performance (and may be worse!) in the regime where the MPI application is scaling well. •Benefits come when MPI scalability

Hybrid Programming: MPI+OpenMP - HPC-Forge · 2013-07-22 · MPI+OpenMP hybrid paradigm is the trend for clusters with SMP architecture. Elegant in concept: use OpenMP within the

チュートリアル: MPI/OpenMP* ハイブリッド・アプリケーション … · 2018-03-23 · チュートリアル: MPI/OpenMP* ハイブリッド・アプリケーションの解析

Programmation hybride MPI-OpenMP - IDRIS · Programmation hybride MPI-OpenMP [email protected] [email protected] CNRS — IDRIS / LA Version 2.3 —

MPI/OpenMP Hybrid Parallelism for Multi-core Processors · Historical usage of MPI/OpenMP • MPI-OpenMP hybrid programming is not new • It was used to parallelise some codes on

Hybrid MPI and OpenMP Parallel Programmingmc.stanford.edu/cgi-bin/images/7/78/Hybrid_MPI_openMP.pdf · Hybrid MPI and OpenMP Parallel Programming MPI + OpenMP and other ... Processors

Combining MPI and OpenMP

Hybrid Programming with OpenMP and MPI

Tutorial Exercise NPB-MPI/BT - Prace Training Portal: Events · 2018-07-10 · NPB-BT Intermediate-level tutorial example Available in MPI, OpenMP, hybrid OpenMP/MPI variants also

Splotch: High Performance Visualization using MPI, OpenMP ... · Splotch: High Performance Visualization using MPI, OpenMP and CUDA Klaus Dolag (Munich University Observatory)

1 Parallel Programming With OpenMP. 2 Contents Overview of Parallel Programming & OpenMP Difference between OpenMP & MPI OpenMP Programming Model

Mpi Openmp Examples

Introduction to OpenMP. OpenMP Introduction Credits: allans/cs260/lectures/OpenMP.ppt douglas/Classes/cs521-s02/...openmp/MPI-OpenMP.ppt

Developing Hybrid OpenMP-MPI Parallelism for Fluidity ... · Developing Hybrid OpenMP-MPI Parallelism for Fluidity - Next Generation Geophysical Fluid Modelling Technology Xiaohu

Shared-Memory Parallelism and OpenMP · 2018-09-24 · Hybrid MPI + OpenMP! • The hybrid model!- OpenMP works in the memory space of each MPI process! • Shared memory within each

HPC course on MPI, PETSC, and OpenMP

Introduction to MPI and OpenMP - Electrical & Computer ...courses/ee8218/mpi_openmp.pdf · Introduction to MPI and OpenMP myson @ postech.ac.kr ... GCC and MPICH2 for MPI GCC-4.2

Hybrid MPI/OpenMP Power-Aware Computing

Parallel Programming with MPI and OpenMP

Introduction to MPI, OpenMP, Threads

Parallel Programming in C/C++ - OpenMP versus MPI MPI ...gibson/Teaching/CSC5021/L4-Parallel... · Note: OpenMP and MPI are oftencombined-in a hybridsystem-to provide parallelismatdifferentlevelsof

Hybrid MPI-OpenMP versus MPI Implementations: A Case Study · Hybrid MPI-OpenMP versus MPI Implementations: A Ca se Study Osvaldo Mangual +, Marvi Teixeira *, Reynaldo Lopez #, Felix

Insieme-an Optimization System for OpenMP, MPI and ...invasic.informatik.uni-erlangen.de/publications/Moritsch.pdf · Insieme-an Optimization System for OpenMP, MPI and OpenCLPrograms

Multigrid Method using OpenMP/MPI Hybrid Parallel ... · • Flat MPI vs. Hybrid (OpenMP+MPI) • Expectations for Hybrid Parallel Programming Model – Number of MPI processes (and

MPI and OpenMP - Archer · • Both OpenMP and MPI are themselves highly portable (but not perfect). • Combined MPI/OpenMP is less so • main issue is thread safety of MPI •

MPI and OpenMP

HPC1 OpenMP E. Bruce Pitman October, 2002. HPC1 Outline What is OpenMP Multi-threading How to use OpenMP Limitations OpenMP + MPI References

E5 Pthread OpenMP MPI Molinas Luque

Parallele Programmierung mit OpenMP und MPI - clug.de · Fahrplan Einfuhrung / Umfrage Parallelrechner OpenMP (Open)MPI Fazit Robin Geyer Parallele Programmierung mit OpenMP und MPI