Meeting Exascale-potential with research on highly pa­rallel­systems,­efficient­scalability,­energy­efficient­algorithms, and large-scale simulations.

Exascale Computing

4HLRS

HLRSHigh Performance Computing Center Stuttgart

The High Performance Compu-ting Center of Stuttgart (HLRS) of the University of Stuttgart is the first National Supercompu-ting Center in Germany and is offering services to both acade-mic users and industry. Apart from the operation of supercom-puters HLRS activities include teaching and training in distribu-ted systems, software enginee-ring and programming models, as well as development of new technologies. HLRS is an acti-ve player in the European rese-arch arena with special focus on Scientific Excellence and Indust-rial Leadership initiatives.

Our Network: HLRS is tightly connected to academia and indus-try through long term partners-hips with global market players such as Porsche and T-Systems, as well as worldwide companies, HPC centres and Universities. Particular attention is given to collaboration with Small and Me-dium Enterprises (SMEs).

Our Infrastructure: HLRS ope-rates a CRAY XC40 supercom-puter (peak performance > 7 Pe-taFlops), as well as a variety of smaller systems, reaching from clusters to cloud resources.

ProgrammingModels & Tools

CloudComputing

Optimization & Scalability

Energy Efficiency

Exascale Computing

Services

Big Data, Analytics & Management

Visualization

Featured Topics

5HLRS

Director HLRSProf. Dr. Michael Resch

Our Experience: HLRS has been at the forefront of regional, nati-onal and European research and innovation over the last 20 years. During this time, HLRS has parti-cipated successfully in more than 90 European research and inno-vation projects.

Our Expertise: HLRS is a lea-ding innovation center, applying software engineering methods to HPC and Cloud for the bene-fit of multiple application domains such as automotive, engineering, health, mobility, security, and energy. Thanks to the close inter-action with industry, the center’s capabilities and expertise sup-port the whole lifecycle of simu-lation covering research aspects, pre-competitive development and preparation for production. The HLRS innovation group, which actively examines and tests new technologies, can bring into pro-jects expertise on leading edge technologies hardware and scale up data analysis techniques.

6Exascale Computing

Exascale Computing

The shift from petascale compu-ting to exascale computing—a thousandfold increase in compu-ting power—constitutes the start of a new era within the commu-nity of High-Performance Compu-ting (HPC). The paradigm shift from petasca-le to exascale will not only provi-de faster HPC systems, but also influence the path of designing hardware components, software, applications, and platforms. The-se aspects of supercomputing will need to be adapted, optimized, or, in some cases, even reinven-ted. After all, it is the ultimate goal to efficiently solve computa-tional problems, which are so far too complex for recent systems.To this end, the High-Performan-ce Computing Center Stuttgart (HLRS) takes part in various re-search activities that are topics of interest on the path to exasca-le. Our research activities will

improve scalability of applications and enable them to run them on massively parallel systems. We tackle large problems with high numeric complexity and work to-ward energy-efficient algorithms, reducing highly parallelized sys-tems’ power consump tion. With this brochure, we invite you to discover not only how traditi-onal HPC-applications, such as computational fluid dynamics (CFD), can be improved on their path to exascale, but also how improvements need to be deliver-ed, such as supporting the evolu-tion of application-specific codes. Furthermore, there is a clear need to discover the full potenti-al to manage the emergence of increasingly higher data volumes in exascale leading to the emer-gence of High Performance Data Analytics (HPDA), which will be-come more and more important in the future.

7Exascale Computing

Project Overview

POP - Performance Optimisation and Productivity (A Centre of Excel-lence in Computing Applications)

Mont-Blanc 2/3

EXPERTISE - EXperiments and high PERformance computing for Turbine mechanical Integrity and Structural dynamics in Europe

EXASOLVERS - Extreme Scale Solvers for Coupled Problems

Page 8

Page 10

Page 12

Page 14

ExaFLOW - Enabling Exascale Fluid Dynamics Simulations

Page 16

Page 18CATALYST - Combining HPC and High Performance Data Analytics for Academia and Industry

8POP

POP

High performance computing is a fundamental tool for the progress of science and engineering and as such for the economic competiti-veness. The growing complexity of parallel computers is leading to a situation where code owners and users are not aware of the de-tailed issues affecting the perfor-mance of their applications. The result is often an inefficient use of computing resources. Code de-velopers often do not have suffi-cient insight in its detailed causes in order to address the problem properly. The objective of POP is to operate a Center of Excellence in perfor-mance optimisation and producti-vity and to share our expertise in the field with the computing com-munity. In particular, POP will offer the service of precisely assessing the performance of computing applications of any sort, from a few hundred to many thousands of processors. Also, POP will show users the specific issues affecting the performance of their code and the best way to allevia-

Performance Optimisation and Productivity (A Centre of Excellence in Computing Applications)

te them. POP will target and offer such services to code owners and users from all domains, including infrastructure operators, acade-mic and industrial users.The estimated population of such applications in Europe is 1500

9POP

and within the project lifetime POP has the ambition of serving over 150 such codes. The Added Value of POP’s services is the sa-vings generated in the operation and use of a code, which will re-sult in a significant Return on In-vestment (fixing a code costs less than running it below its optimal levels) by employing best-in-class services and release capacity for resolving other priority issues. POP will be a best-in-class centre. By bringing together the Europe-an world-class expertise in the area and combining excellent aca-demic resources with a practical, hand-on approach, it will improve

ContactDr. José Gracia

Christoph Niethammer

Phone: +49 (0) 711/ 685-87208

+49 (0) 711/ 685 87203

E-Mail: gracia@hlrs.de

niethammer@hlrs.de

Further Informationwww.pop-coe.eu

the access to computing appli-cations, thus allowing European researchers and industry to be more competitive.

Project Partners � Barcelona Supercomputing Center, Spain

� Numerical Algorithm Group,UK � RWTH Aachen � HLRS � Teratec, FR � Forschungszentrum Jülich

Project Information � Funding Organisation: EU-H2020

� Runtime: 10.2015 - 03.2018

10Mont-Blanc 2/3

model, and support for multi-no-de debugging. In addition, HLRS also contributed to evaluation of the programming model and prototype system by porting and benchmarking an application from the engineering domain.

Funding Organisation: EC FP7Runtime: 01.10.2013 – 31.1.2017

Mont-Blanc 3The Mont-Blanc project aims to design a new type of computer architecture capable of setting future HPC standards, built from energy efficient solutions used in embedded and mobile devices. The project has been running since 2011 and was extended in 2013 (Mont Blanc 2) and 2015 (Mont Blanc 3), respectively. In particular, Mont Blanc 3 will en-able further development of the OmpSs programming model to automatically exploit multiple clus-ter nodes, transparent applica-tion checkpointing for fault-tole-rance, support for ARMv8 64-bit

European approach towards Energy Efficient­High­Performance

Mont-Blanc 2/3

Mont-Blanc 2The limiting factor in the develop-ment of an Exascale High Per-formance Computer System is power consumption. The Mont-Blanc2 project focused on the task to develop a next generati-on HPC system using embedded technologies to reach this difficult task. After the development of the hardware architecture in the first phase of the Mont-Blanc pro-ject, Mont-Blanc2 focused more on the developed of the neces-sary system software stack and evolution of the system design. It examined a new programming model allowing to write efficient code for the new computer archi-tecture. It emphasized tools for the programmer like debugger and performance analysis tools, which increase the usability of such a system for the users.The main contribution of HLRS is the development of scalable de-bugging tools. In particular, HLRS extended the task-based graphi-cal debugger Temanejo with sup-port for the OmpSs programming

11Mont-Blanc 2/3

ContactDr. José Gracia

Phone: +49 (0) 711/ 685-87208

E-Mail: gracia@hlrs.de

processors, and the initial design of the Mont-Blanc Exascale archi-tecture. HLRS contribution to the project is twofold. Firstly, we will participate in the development of the programming model, in parti-cular combining MPI and OmpSs into a hybrid, task-aware MPI/OmpSs. This will allow to overlap MPI communication with com-putation with minimal effort for the application programmer. Se-condly, HLRS will contribute to the evaluation of the programming model and the architecture by porting a repres ventative scien-tific application.

Funding Organisation: EC H2020Runtime: 01.10.2015 – 31.09.2018

Further Informationwww.montblanc-project.eu

Project Partners

12EXPERTISE

models of turbine components to pave the way towards the virtual testing of the entire machine. Key aspects addressed thereby will be the understanding and accurate modeling of physics of frictional contact interfaces, new, highly efficient and accurate nonlinear dynamic analysis tools as well as the integration of all this into high performance computing (HPC) techniques, enabling for the first time the accurate dynamic analy-sis of a large scale turbomachi-nery model.

The research program of EXPER-TISE is based on the following Work Packages (WPs):

� WP1 – Advanced modeling of friction contacts

� WP2 – Identification of cont-act interfaces

� WP3 – Structural dynamics of turbine and its components

� WP4 – High Performance Computing for structural dyna-mics

EXperiments and high PERformance computing for­Turbine­mechanical­Integrity­and­Structural­ dynamics in Europe

EXPERTISE

EXPERTISE is a European Training Network (ETN) that will contribute to train the next generation of me-chanical and computer science engineers. Within the network 15 Early Stage Researchers (ESRs) will work on the big challenges on the way to a fully validated nonline-ar dynamic model of turbo-machi-nery components. Along their way they aresupervised by experts at world leading institutions from across Europe in this multidisci-plinary project. The ultimate re-search objective of EXPERTISE is to develop advanced tools for the dynamics analysis of large-scale

13EXPERTISE

ContactDr. José Gracia

Christoph Niethammer

Phone: +49 (0) 711/ 685-87208

+49 (0) 711/ 685 87203

E-Mail: gracia@hlrs.de

niethammer@hlrs.de

Further Informationwww.msca-expertise.eu

HLRS as expert in the field of high performance computing (HPC) will lead the HPC activities in EX-PERTISE. Also, HLRS will have a key role in the network by training all the researchers in modern HPC techniques and furthermo-re add its own research project, addressing the tremendous prob-lem of handling the huge amounts of data that are produced during these full model simulations and bring HPC systems to their limits.

BeneficiariesImperial College of Science Tech-nology and Medicine London | Uni-versität Stuttgart | University of Oxford | CRAY UK Limited| École Centrale de Lyon | Middle East Technical University | Vysoka Sko-la Banska – Technicka Univerzita Ostrava | Barcelona Supercompu-ting Center – Centro Nacional de Supercomputacion | Mavel AS | Technische Universität München|

Project Information � Runtime: 03.2017 – 02.2021 � Funding Organisation: Horizon 2020, Marie Sklodowska-Cu-rie Actions, Innovative Training Network (H2020-MSCA- ITN)

14EXASOLVERS

Optimization­and­inverse­prob-lems (Trier University)By means of inverse problems, it is possible to determine simu-lation parameters that can’t be measured due to e.g. subminia-ture structures, inaccessible en-vironments, etc. However, usage of the aforementioned methods for optimization and inverse prob-lems provides further potential to use exascale systems efficiently.

Uncertainty­quantification­ (RWTH Aachen)The group from Aachen uses low rank hierarchical tensors to quan-tify uncertainties of simulations, which allows to further increase the amount of parallelism that can be used efficiently.

Exascale systems will be charac-terized by bil lion-way parallelism. Computing on such extreme sca-les requires suitable methods. The ExaSolvers 2 project hence investigates such methods:

Parallel adaptive multigrid (G-CSC, University Frankfurt)The multigrid method is of opti-mal complexity and hence suited for extreme scale parallelism. The group from Frankfurt develops their own parallel multigrid frame-work ug4 which also adapts mesh resolution in order to increase the solution efficiency.

Time parallelization (ICS, USI Lugano)In transient simulations, not only the simulation domain but also the investigated time frame can be divided and handled on diffe-rent execution units in parallel in order to efficiently use the massi-ve parallelism of future systems.

EXASOLVERSExtreme­Scale­Solvers­for­Coupled­Problems­

15EXASOLVERS

Energy­efficiency­ (HLRS, University Stuttgart)Due to their massive parallelism, Exascale systems will require huge amounts of energy. We hence in-vestigate methods to increase the energy effi ciency of such systems on multiple levels, i.e. algo rithmic efficiency, efficiency-aware imple-mentation as well as adaption of hardware parameters (e.g. redu-cing the CPU’s core frequency, known as Dynamic Voltage and Frequency Scaling).

A collaboration with the Japane-se ADVENTURE project has been established in order to deploy the

ContactBjörn Dick

Dr. Ralf Schneider

Phone: +49 (0) 711/ 685-87189

+49 (0) 711/ 685-87236

E-Mail: dick@hlrs.de

schneider@hlrs.de

Further Informationwww.hlrs.de/about-us/research/

current-projects/exasolvers

performance engineering experti-se of the project partners from Japan on codes developed by the ExaSolvers 2 project. In return, ADVENTURE is going to integra-te our methods into their frame-work.In order to assess the developed methods, a simulation of trans-dermal drug delivery through the human skin with detailed resolu-tion of the lipid scale is used as benchmark application.

Project Information � Runtime: 05.2016 - 04.2019 � Funding Organisation: DFG

16ExaFLOW

The main goal of ExaFLOW is to address key algorithmic challen-ges in CFD (Computational Fluid Dynamics) to enable simulation at exascale, guided by a number of use cases of industrial relevance, and to provide open-source pilot implementations. Thus, driven by problems of practical engineering interest we focus on important si¬mulation aspects, including:

� error control and adaptive mesh refinement in complex computational domains

� resilience and fault tolerance in complex simulations

� solver efficiency via mixed dis-continuous and continuous Ga-lerkin methods and appropria-te optimised preconditioners

� heterogeneous modelling to allow for different solution al-gorithms in different domain zones

� evaluation of energy efficiency in solver design

� parallel input/output and in-si-tu compression for extreme data

Enabling­Exascale­Fluid­Dynamics­Simulations

ExaFLOW

We are surrounded by moving fluids (gases and liquids), be it breathing or the blood flow in our arteries; the flow around cars, ships, and airplanes; the changes in cloud formations or plankton transport in oceans; even forma-tions of stars and galaxies are modelled as phenomena in fluid dynamics. Fluid dynamics simula-tions provide a powerful tool for the analysis of fluid flows and are an essential element of many in-dustrial and academic problems. In fluid dynamics there is almost no limit to the size of the systems to be studied via numerical si-mulations. The complexities and nature of fluid flows, often com-bined with problems set in open domains, imply that the resour-ces needed to computationally model problems of industrial and academic relevance are almost unbounded. The main goal of this project is to address algorithmic challenges to enable the use of more accurate simulation models in exascale environments.

17ExaFLOW

ContactDr. Ralf Schneider

Phone: +49 (0) 711/ 685-87236

E-Mail: schneider@hlrs.de

Further Informationwww.exaflow-project.eu

In ExaFlow the High-Performan-ce Computing Center Stuttgart (HLRS), in cooperation with the in-stitute for Aero and Gas Dy-namics (IAG) of the University of Stuttgart, forms the second biggest partner in the ExaFlow Consortium. In terms of Data reduction, HLRS is especially re-

sponsible for the evaluation and development of data reduction al-gorithms based on dynamic-mode decompo-sition (DMD) and emer-ging new ideas related to the Ko-opman Operator. Additionally, the task of researching energy effi-ciency and awareness is located at HLRS. Within this scope, the power consumption of different implementations is measured, using both high-resolution compo-nent level and lower-resolution no-de-level measurement methods.

Project Information � Runtime: 10.2015 - 09.2018 � Funding Organisation: EU H2020

18CATALYST

As the majority of today’s data analytics algorithms are oriented towards text processing (e.g. bu-si-ness analytics) and graph analy-sis (e.g., social network studies), we are further in need to evaluate existing algorithms with respect to their applicability for the enginee-ring domain. Thus, CATALYST will examine future concepts for both hardware and software. The first case study conducted in collaboration with Cray Inc. addresses the performance va-riations of our Cray XC40 sys-tem. Performance variability on HPC platforms is a critical issue with serious implications for the users: irregular runtimes prevent users from correctly assessing performance and from efficiently planning allocated machine time. Consequently, monitoring today’s IT infra-structures has actually be-come a big data challenge on its own. The analysis workflow used to identify the causes of runtime variations consists of three steps including different configuration parameters:

Combining­HPC­and­High­Performance­Data­ Analytics for Academia and Industry

CATALYST

At the High Performance Com-puting Center Stuttgart (HLRS), customers tend to execute more and more data-intensive applica-tions. Since it no longer becomes feasible that data is processed and analysed manually by domain experts, HLRS and Cray Inc. have launched the CATALYST project to advance the field of data-intensi-ve computing by converging HPC and Big Data in order to allow a seamless workflow between com-pute-intensive simulations and data-intensive analytics. For that purpose, Cray Inc. designed the Urika-GX data analytics hardware, which supports Big Data techno-logies and furthermore, enhan-ces the analysis of semantic data. This system has been installed as an extension of Hazel Hen, the current HPC-flagship system of HLRS. The main objective of CATALYST is to evaluate the hardware as well as the software stack of the Ur-ika-GX and its usefulness with a particular focus on applications from the engineering domain.

19CATALYST

ContactMichael Gienger

Phone: +49 (0) 711 / 685-63824

E-Mail: gienger@hlrs.de

With the help of this workflow, 470 so called „Victim“ applications have been identified that suffered from the particular behaviour of 3 „Aggressors“. Consequently, HLRS took this information and approached the responsible sta-keholders in order to optimise their applications in general. So not only the performance of these applications has been improved, but also the entire system perfor-mance in production.

Further Informationwww.hlrs.de/en/about-us/

research/current-projects/

data-analytics-for-hpc

Outlook � Big Data application evaluation � Close cooperation with part-ners from both, industry and academia

� Seamless integration of the Big Data system into our exis-ting HPC infrastructure

� Develop and evaluate practical case studies to advertise the solution

Project Information � Runtime: 10.2016 – 09.2019 � Funding Organisation: Ministry of Science, Research and the Arts Baden-Württemberg

� Partners: HLRS, Cray Inc. & Daimler AG (associated)

High Performance Computing Center Stuttgart (HLRS) University of Stuttgart Nobelstrasse 19 | 70569 Stuttgart | Germany Phone: +49 (0)711 / 685 87 269 Fax: +49 (0)711 / 685 87 209 Mail: pr@hlrs.de www.hlrs.de

Editor: Lena Bühler, Eric Gedenk, Dr. Bastian Koller Design: Janine Jentsch, Ellen Ramminger Picture Credits: Cover and Interior shot: Bohris Lehner for HLRS Back cover shot: Simon Sommer for HLRS © HLRS 2018