Search

Significant improvements are presented for the molecular dynamics code ls1 mardyn — a linked cell-based code for simulating a large number of small, rigid molecules with application areas in chemical engineering. The changes consist of a redesign of the SIMD vectorization via wrappers, MPI improvements and a software redesign to allow memory-efficient execution with the production trunk to increase portability and extensibility. Two novel, memory-efficient OpenMP schemes for the linked cell-based force calculation are presented, which are able to retain Newton’s third law optimization. Comparisons to well-optimized Verlet list-based codes, such as LAMMPS and GROMACS, demonstrate the viability of the linked cell-based approach. The present version of ls1 mardyn is used to run simulations on entire supercomputers, maximizing the number of sampled atoms. Compared to the preceding version of ls1 mardyn on the entire set of 9216 nodes of SuperMUC, Phase 1, 27% more atoms are simulated. Weak scaling performance is increased by up to 40% and strong scaling performance by up to more than 220%. On Hazel Hen, strong scaling efficiency of up to 81% and 189 billion molecule updates per second is attained, when scaling from 8 to 7168 nodes. Moreover, a total of 20 trillion atoms is simulated at up to 88% weak scaling efficiency running at up to 1.33 PFLOPS. This represents a fivefold increase in terms of the number of atoms simulated to date.

ls1 mardyn is a molecular dynamics (MD) simulation framework that enables investigations of multicomponent and multiphase processes relevant to engineering applications, such as droplet coalescence or bubble formation. These scenarios require the simulation of ensembles containing a large number of molecules. We present recent advances in ls1 mardyn both from the software design and high-performance computing perspective. From the former we describe the recently introduced plugin framework, from the latter we will look at some recent load balancing improvements to ls1 mardyn. We further present preliminary results of the integration of AutoPas, a C++ node-level library employing auto-tuning to achieve optimal node-level performance for particle simulations, into ls1 mardyn.

preCICE is a coupling library for partitioned multi-physics simulations. Partitioned means that preCICE couples existing programs capable of simulating a subpart of the complete physics involved in a simulation. This allows for the high flexibility that is needed to keep a decent time-to-solution for complex multi-physics scenarios. preCICE runs efficiently on a wide spectrum of systems, from consumer laptops up to complete compute clusters and has proven scalability on 10000s of MPI Ranks. The impact of preCICE has been growing significantly during the past five years. Today, we know of more than 60 research groups in academia, non-academic research centers, and industry that actively use preCICE.preCICE is funded as part of the German Research Foundation's call for research software sustainability. The aim of the funded project is to lift the state of the library from a scientific prototype to a sustainable and usable library. This goal has a wide range of facets and poses significant challenges: On the one hand, improving the usability of the library for non-experts requires a set of well documented tutorials as a solid starting point, supplemented by comprehensive user documentation. On the other hand, improving the usability for developers requires the migration to well-established development practices and the extension of the testing framework on various levels of complexity up to regression tests for complete coupled simulations. A poster presented at the SIAM CSE21 conference, minisymposterium PP7: Software Productivity and Sustainability for CSE.

preCICE is a coupling library for partitioned multi-physics simulations. Partitioned means that preCICE couples existing programs capable of simulating a subpart of the complete physics involved in a simulation. This allows for the high flexibility that is needed to keep a decent time-to-solution for complex multi-physics scenarios. preCICE runs efficiently on a wide spectrum of systems, from laptops up to 10000s of MPI Ranks. This poster shows how preCICE has developed over the years and highlights three major challenges concerning sustainable software development we faced over the years including strategies used to tackle them. Firstly, how to decide on a range of versions for dependencies to support and what type of package distribution to use. Secondly, how to test a coupling library with logically partitioned tests. Finally, how to profile a coupled simulation in contrast to a single program.

It is difficult to provide live simulation systems for decision support. Time is limited and uncertainty quantification requires many simulation runs. We combine a surrogate model with the stochastic collocation method to overcome time and storage restrictions and show a proof of concept for a de-boarding scenario of a train.

For symmetric (hermitian) (dense or banded) matrices the computation of eigenvalues and eigenvectors Ax = λBx is an important task, e.g. in electronic structure calculations. If a larger number of eigenvectors are needed, often direct solvers are applied. On parallel architectures the ELPA implementation has proven to be very efficient, also compared to other parallel solvers like EigenExa or MAGMA. The main improvement that allows better parallel efficiency in ELPA is the two-step transformation of dense to band to tridiagonal form. This was the achievement of the ELPA project. The continuation of this project has been targeting at additional improvements like allowing monitoring and autotuning of the ELPA code, optimizing the code for different architectures, developing curtailed algorithms for banded A and B, and applying the improved code to solve typical examples in electronic structure calculations. In this paper we will present the outcome of this project.

Plasma fusion is one of the promising candidates for an emission-free energy source and is heavily investigated with high-resolution numerical simulations. Unfortunately, these simulations suffer from the curse of dimensionality due to the five-plus-one-dimensional nature of the equations. Hence, we propose a sparse grid approach based on the sparse grid combination technique which splits the simulation grid into multiple smaller grids of varying resolution. This enables us to increase the maximum resolution as well as the parallel efficiency of the current solvers. At the same time we introduce fault tolerance within the algorithmic design and increase the resilience of the application code. We base our implementation on a manager-worker approach which computes multiple solver runs in parallel by distributing tasks to different process groups. Our results demonstrate good convergence for linear fusion runs and show high parallel efficiency up to 180k cores. In addition, our framework achieves accurate results with low overhead in faulty environments. Moreover, for nonlinear fusion runs, we show the effectiveness of the combination technique and discuss existing shortcomings that are still under investigation.

We apply the Koopman operator framework to pedestrian dynamics. In an example scenario, we generate crowd density time series data with a microscopic pedestrian simulator. We then approximate the Koopman operator in matrix form through Extended Dynamic Mode Decomposition, using Geometric Harmonics on the data as a dictionary. The Koopman matrix is integrated into a surrogate model, which allows to approximate crowd density time series data to be generated, independently from the original microscopic simulator. The evaluation of the constructed surrogate model is orders of magnitude faster, and enables us to use methods that require many model evaluations.

SPPEXA, the Priority Program 1648 “Software for Exa-scale Computing” of the German Research Foundation (DFG), was established in 2012. SPPEXA was DFG’s first strategic Priority Program—strategic in the sense that it had been the initiative of DFG’s board to suggest a larger and trans-disciplinary funding scheme to support the development of software at all levels that would be able to benefit from future exa-scale systems. A proposal had been formulated by a team of scientists representing domains across the STEM fields, evaluated in the standard format for Priority Programs, and financed via special funds. Operations started in January 2013, and after two 3-year funding phases and a cost-neutral extension, SPPEXA’s activities will come to an end by end of April, 2020. A final international symposium took place on October 21–23, 2019, in Dresden, and this volume of Springer’s Lecture Notes in Computational Science and Engineering—the second SPPEXA-related one after the corresponding report of Phase 1 (see Appendix 3 in [1])—contains reports of 16 out of 17 SPPEXA projects (the project ExaSolvers will deliver its report as a special issue of Springer’s journal Computing and Visualization in Science) and is, thus, a comprehensive overview of research within SPPEXA.

Universitaten sind intrinsisch konservativ. Meist im edlen, bewahrenden Sinne des Wortes, manchmal aber auch mit etwas zu viel Beharrungsvermogen. Die Wissenschaft als Programm sucht zwar stets nach dem Neuen, dem Unbekannten, dem Unerforschten. Die institutionalisierte Wissenschaft funktioniert aber auch und vor allem in den Kategorien des Bekannten, bereits Erforschten, Bewahrten und Traditionellen. Und so machen sich Universitaten, wenn sich irgendwo etwas Neues auftut, zwar durchaus sofort daran, dieses zu erkunden und zu verstehen; anschliesend, es zuganglich zu machen; und schlieslich, das Wissen daruber weiterzugeben. Die Selbstreflexion – also z. B. die Frage danach, ob der Forschungsgegenstand und die erzielten Erkenntnisse auch Implikationen auf die eigene Institution, das eigene Wesen und Agieren haben – halt sich dagegen meist vornehm im Hintergrund. Wenn es um Astrophysik, um die Gartenbaukunst der Renaissance oder die Statik von Zeltkonstruktionen geht, gibt es diese Implikationen ja wahrscheinlich gar nicht so unmittelbar. Bei dem, was heute gemeinhin unter dem Schlagwort „Digitalisierung“ firmiert, verhalt es sich jedoch vollig anders. Auch wenn ich ungern von „Universitat irgendwas-Punkt-null“ reden mochte – das, worum es hierbei geht, hat das Zeug zu einer disruptiven Transformation, die wenige Steine dort liegen lasst, wo sie zuvor lagen. Dies betrifft alle Kernbereiche einer Universitat – Forschung, Lehre, Transfer, Kooperationen, Verwaltung, Governance, und was sonst noch so von Bedeutung ist fur eine moderne Universitat. Darum soll es im folgenden Beitrag gehen.

Der vorliegende Ubersichtsartikel berichtet uber Fortschritte in der molekularen Modellierung und Simulation mittels massiv-paralleler Hoch- und Hochstleistungsrechner (HPC). Im SkaSim-Projekt arbeiteten dazu Partner aus der HPC-Community mit Anwendern aus Wissenschaft und Industrie zusammen. Ziel dabei war es mittels HPC-Methoden die Vorhersage von thermodynamischen Stoffdaten in Bezug auf Effizienz, Qualitat und Zuverlassigkeit weiter zu optimieren. In diesem Zusammenhang wurden verschiedene Themen bearbeitet: Atomistische Simulation der homogenen Gasblasenbildung, Oberflachenspannung klassischer Fluide und ionischer Flussigkeiten, multikriterielle Optimierung molekularer Modelle, Weiterentwicklung der Simulationscodes ls1 mardyn und ms2, atomistische Simulation von Gastrennprozessen, molekulare Membran-Strukturgeneratoren, Transportwiderstande und gemischtypenspezifische Bewertung pradiktiver Stoffdatenmodelle.

To foster predictive simulations, a variety of methods have recently been developed to efficiently tackle uncertainty quantification (UQ) in complex, computational intensive problems. Many of these methods are non-intrusive and, thus, result in a (large) number of embarrassingly parallel black-box evaluations of the underlying simulation codes. While the focus of development is typically on the number of black-box evaluations, which represents the bulk of the computational workload, an additional level of potential performance gains exists. In many scenarios, uncertain input leads not only to uncertain outputs, but also to a varying and thus stochastic runtime of the simulation codes. For scheduling the individual black-box runs, this information is typically not taken into account, resulting in non-negligible idling times on parallel systems. In this contribution, we compare a variety of different scheduling strategies for non-intrusive UQ scenarios using the non-intrusive polynomial chaos approach. In particular, we propose to construct a surrogate model for the runtime of the application using the identical UQ methodology as for the original problem. Using this model to predict the runtimes for subsequent black-box runs allows for (heuristical) optimization of the scheduling. The method has been tested for the forward quantification of uncertainty on academic models and on a pedestrian simulation in the context of evacuation scenarios. This approach allows speed-up factors of about two for the total runtime and can be generalised to a large variety of applications that incorporate parameter-dependent runtime.

Over the last decades, quantum cascade lasers (QCLs) have become established sources of mid-infrared and terahertz light. For their anticipated applications, e.g., in spectroscopy, their dynamical behavior is particularly interesting. Numerical simulations constitute an essential tool for investigating the QCL dynamics but exhibit considerable computational workload. In order to accelerate the simulations and thereby aid the design process of QCLs, we present efficient parallel implementations of an established numerical method using OpenMP. Performance measurements on a 28-core CPU confirm their efficiency.

Molecular dynamics (MD) simulations enable the investigation of multicomponent and multiphase processes relevant to engineering applications, such as droplet coalescence or bubble formation. These scenarios require the simulation of ensembles containing a large number of molecules. We present recent advances within the MD framework ls1 mardyn which is being developed with particular regard to this class of problems. We discuss several OpenMP schemes that deliver optimal performance at node-level. We have further introduced nonblocking communication and communication hiding for global collective operations. Together with revised data structures and vectorization, these improvements unleash PetaFLOP performance and enable multi-trillion atom simulations on the HLRS supercomputer Hazel Hen. We further present preliminary results achieved for droplet coalescence scenarios at a smaller scale.

The C++ library AutoPas aims at delivering optimal node-level performance for particle simulations. This paper describes the internally implemented algorithms, and how the library uses auto-tuning to dynamically select their optimal combination at run-time. Results are presented, which show that all available algorithms and configuration options have their specific advantages. To demonstrate the library's capabilities in relevant application settings, it has been integrated into the software package ls1 mardyn. An example of a realistic molecular dynamics simulation from thermodynamics is shown in which AutoPas detects a change in the best possible algorithm configuration. It adapts the simulation algorithm accordingly, sustaining optimal performance without additional user input.

In the emerging field of multi-physics simulations, we often face the challenge to establish new connections between physical fields, to add additional aspects to existing models, or to exchange a solver for one of the involved physical fields. If in such cases a fast prototyping of a coupled simulation environment is required, a partitioned setup using existing codes for each physical field is the optimal choice. As accurate models require also accurate numerics, multi-physics simulations typically use very high grid resolutions and, accordingly, are run on massively parallel computers. Here, we face the challenge to combine flexibility with parallel scalability and hardware efficiency. In this paper, we present the coupling tool preCICE which offers the complete coupling functionality required for a fast development of a multi-physics environment using existing, possibly black-box solvers. We hereby restrict ourselves to bidirectional surface coupling which is too expensive to be done via file communication, but in contrast to volume coupling still a candidate for distributed memory parallelism between the involved solvers. The paper gives an overview of the numerical functionalities implemented in preCICE as well as the user interfaces, i.e., the application programming interface and configuration options. Our numerical examples and the list of different open-source and commercial codes that have already been used with preCICE in coupled simulations show the high flexibility, the correctness, and the high performance and parallel scalability of coupled simulations with preCICE as the coupling unit.

The macro–micro-coupling tool (MaMiCo) was developed to ease the development of and modularize molecular-continuum simulations, retaining sequential and parallel performance. We demonstrate the functionality and performance of MaMiCo by coupling the spatially adaptive Lattice Boltzmann framework waLBerla with four molecular dynamics (MD) codes: the light-weight Lennard-Jones-based implementation SimpleMD, the node-level optimized software ls1 mardyn, and the community codes ESPResSo and LAMMPS. We detail interface implementations to connect each solver with MaMiCo. The coupling for each waLBerla-MD setup is validated in three-dimensional channel flow simulations which are solved by means of a state-based coupling method. We provide sequential and strong scaling measurements for the four molecular-continuum simulations. The overhead of MaMiCo is found to come at 10%–20% of the total (MD) runtime. The measurements further show that scalability of the hybrid simulations is reached on up to 500 Intel SandyBridge, and more than 1000 AMD Bulldozer compute cores. Program summary Program title: MaMiCo Catalogue identifier: AEYW_v1_0 Program summary URL: http://cpc.cs.qub.ac.uk/summaries/AEYW_v1_0.html Program obtainable from: CPC Program Library, Queen’s University, Belfast, N. Ireland Licensing provisions: BSD License No. of lines in distributed program, including test data, etc.: 67905 No. of bytes in distributed program, including test data, etc.: 1757334 Distribution format: tar.gz Programming language: C, C++II. Computer: Standard PCs, compute clusters. Operating system: Unix/Linux. RAM: Test cases consume ca. 30–50 MB Classification: 7.7. External routines: Scons ( http:www.scons.org ), ESPResSo, LAMMPS, ls1 mardyn, waLBerla Nature of problem: Coupled molecular-continuum simulation for multi-resolution fluid dynamics: parts of the domain are resolved by molecular dynamics whereas large parts are covered by a CFD solver, e.g. a lattice Boltzmann automaton Solution method: We couple existing MD and CFD solvers via MaMiCo (macro–micro coupling tool). Data exchange and coupling algorithmics are abstracted and incorporated in MaMiCo. Once an algorithm is set up in MaMiCo, it can be used and extended, even if other solvers are used (as soon as the respective interfaces are implemented). Restrictions: Currently, only single-centered Lennard-Jones systems are supported. Running time: Runtime depends on the underlying coupled problem and may range from minutes to days. The provided test cases for all different solver couplings (incl. one complete coupling cycle of avg. domain size) take ca. 10 h on a regular Desktop.

One approach for tackling the challenge of efficient implementations for parallel PDE simulations on dynamically changing grids is the usage of space-filling curves (SFCs). While SFC algorithms possess advantageous properties such as low memory requirements and close-to-optimal partitioning approaches with linear complexity, they require efficient communication strategies for keeping and utilizing the connectivity information, in particular for dynamically changing grids. Our approach is to use a sparse communication graph to store the connectivity information and to transfer data blockwise. This permits efficient generation of multiple partitions per memory context (denoted by clustering), which---in combination with a run-length encoding (RLE)---directly leads to elegant solutions for shared, distributed, and hybrid parallelization and allows cluster-based optimizations. While previous work focused on specific aspects, we present in this paper an overall compact summary of the stack-RLE clustering appro...

To understand how the motion of a droplet on a surface can be controlled by wettability gradients is interesting in a variety of technical applications. Phase field (PF) models can be used to study such scenarios but their application requires suitable models of the properties of the interacting phases: vapor, liquid, and solid. In this work, the PF simulations are linked to molecular models by using an equation of state as well as a correlation for the viscosity, that are both consistent with results determined by molecular dynamics (MD) simulations. The motion of a nanoscale droplet on a surface with a wettability gradient is studied both by MD simulations and PF simulations and the results are compared. In both methods, the wettability gradient is solely determined by the surface tensions of the liquid–vapor, solid–liquid, and solid–vapor interfaces. Simulations are conducted for two different profiles of the wettability and at two different temperatures. The qualitative and the quantitative behavior such as the shape of the droplet and the velocity of the motion are in good agreement. This validates the PF model for the determination of nanoscale phenomena, and enables an efficient investigation of nanoscale as well as larger scenarios.

ls1 mardyn is a molecular dynamics (MD) simulation framework that en- ables investigations of multicomponent and multiphase processes relevant to engi- neering applications, such as droplet coalescence or bubble formation. These sce- narios require the simulation of ensembles containing a large number of molecules. We present recent advances in ls1 mardyn both from the software design and high- performance computing perspective. From the former we describe the recently intro- duced plugin framework, from the latter we will look at some recent load balancing improvements to ls1 mardyn. We further present preliminary results of the integration of AutoPas, a C++ node- level library employing auto-tuning to achieve optimal node-level performance for particle simulations, into ls1 mardyn.

Quantifying uncertainty in predictive simulations for real-world problems is of paramount importance - and far from trivial, mainly due to the large number of stochastic parameters and significant computational requirements. Adaptive sparse grid approximations are an established approach to overcome these challenges. However, standard adaptivity is based on global information, thus properties such as lower intrinsic stochastic dimensionality or anisotropic coupling of the input directions, which are common in practical applications, are not fully exploited. We propose a novel structure-exploiting dimension-adaptive sparse grid approximation methodology using Sobol' decompositions in each subspace to introduce a sensitivity scoring system to drive the adaptive process. By employing local sensitivity information, we explore and exploit the anisotropic coupling of the stochastic inputs as well as the lower intrinsic stochastic dimensionality. The proposed approach is generic, i.e., it can be formulated in terms of arbitrary approximation operators and point sets. In particular, we consider sparse grid interpolation and pseudo-spectral projection constructed on (L)-Leja sequences. The power and usefulness of the proposed method is demonstrated by applying it to the analysis of gyrokinetic microinstabilities in fusion plasmas, one of the key scientific problems in plasma physics and fusion research. In this context, it is shown that a 12D parameter space can be scanned very efficiently, gaining more than an order of magnitude in computational cost over the standard adaptive approach. Moreover, it allows for the uncertainty propagation and sensitivity analysis in higher-dimensional plasma microturbulence problems, which would be almost impossible to tackle with standard screening approaches., 34 pages, 14 figures

IB163. First, we verify that our implementations of the methods (using the OpenMP standard for parallelization) exhibit good performance scalability. Then, we compare the absolute performance values of the methods and discuss their advantages and drawbacks.

The Maxwell–Liouville–von Neumann (MLN) equations are a valuable tool in nonlinear optics in general and to model quantum cascade lasers in particular. Several numerical methods to solve these equations with different accuracy and computational complexity have been proposed in related literature. We present an open-source framework for solving the MLN equations and parallel implementations of three numerical methods using OpenMP. The performance measurements demonstrate the efficiency of the parallelization.

We present a multilevel stochastic collocation (MLSC) with a dimensionality reduction approach to quantify the uncertainty in computationally intensive applications. Standard MLSC typically employs grids with predetermined resolutions. Even more, stochastic dimensionality reduction has not been considered in previous MLSC formulations. In this paper, we design an MLSC approach in terms of adaptive sparse grids for stochastic discretization and compare two sparse grid variants, one with spatial and the other with dimension adaptivity. In addition, while performing the uncertainty propagation, we analyze, based on sensitivity information, whether the stochastic dimensionality can be reduced. We test our approach in two problems. The first one is a linear oscillator with five or six stochastic inputs. The dimensionality is reduced from five to two and from six to three. Furthermore, the dimension-adaptive interpolants proved superior in terms of accuracy and required computational cost. The second test case is a fluid-structure interaction problem with five stochastic inputs, in which we quantify the uncertainty at two instances in the time domain. The dimensionality is reduced from five to two and from five to four.

In this paper we show how to benefit from the numerical properties of a well-established extrapolation method—the combination technique—to make it tolerant to silent data corruption (SDC). The term SDC refers to errors in data not detected by the system. We use the hierarchical structure of the combination technique to detect if parts of the floating point data are corrupted. The method we present is based on robust regression and other well-known outlier detection techniques. It is a lossy approach, meaning we sacrifice some accuracy but we benefit from the small computational overhead. We test our algorithms on a d-dimensional advection-diffusion equation and inject SDC of different orders of magnitude. We show that our method has a very good detection rate: large errors are always detected, and the small errors that go undetected do not noticeably damage the solution. We also carry out scalability tests for a 5D scenario. We finally discuss how to deal with false positives and how to extend these ideas to more general quantities of interest.

We first briefly report on the status and recent hievements of the ELPA-AEO (Eigen-value Solvers for Petaflop Applications -- Algorithmic Extensions and Optimizations) and ESSEX II (Equipping Sparse Solvers for Exascale) projects. In both collaboratory efforts, scientists from the application areas, mathematicians, and computer scientists work together to develop and make available efficient highly parallel methods for the solution of eigenvalue problems. Then we focus on a topic addressed in both projects, the use of mixed precision computations to enhance efficiency. We give a more detailed description of our approaches for benefiting from either lower or higher precision in three selected contexts and of the results thus obtained.