Search

This paper investigates the exploitation of heterogeneous DVFS (dynamic voltage frequency scaling) control for improving the energy efficiency of data-parallel applications on ccNUMA shared-memory systems. We propose to adjust the clock frequency individually for the appropriately selected groups of cores, taking into account the diversified costs of parallel computation. This paper aims to evaluate the proposed approach using two different data-parallel applications: solving the 3D diffusion problem, and MPDATA fluid dynamics application. As a result, we observe the energy-savings gains of up to 20 percentage points over the traditional homogeneous frequency scaling approach on the server with two 18-core Intel Xeon Gold 6240. Additionally, we confirm the effectiveness of our strategy using two 64-core AMD EPYC 7773X. This paper also introduces two pruning algorithms that help select the optimal heterogeneous DVFS setups taking into account the energy or performance profile of studied applications. Finally, the cost and efficiency of developed algorithms are verified and compared experimentally against the brute-force search. • Heterogeneous DVFS method for energy efficiency of regular data-parallel applications. • Individually adjusting clock frequency for cores based on workload distribution. • Pruning algorithms for selecting optimal heterogeneous DVFS setups. [ABSTRACT FROM AUTHOR]

This paper meets the challenge of harnessing the heterogeneous communication architecture of ccNUMA multiprocessors for heterogeneous stencil computations, an important example of which is the Multidimensional Positive Definite Advection Transport Algorithm (MPDATA). We propose a method for optimization of parallel implementation of heterogeneous stencil computations that is a combination of the islands-of-core strategy and ( 3 + 1 )D decomposition. The method allows a flexible management of the trade-off between computation and communication costs in accordance with features of modern ccNUMA architectures. Its efficiency is demonstrated for the implementation of MPDATA on the SGI UV 2000 and UV 3000 servers, as well as for 2- and 4-socket ccNUMA platforms based on various Intel CPU architectures, including Skylake, Broadwell, and Haswell. [ABSTRACT FROM AUTHOR]

In this work, we take up the challenge of performance portable programming of heterogeneous stencil computations across a wide range of modern shared-memory systems. An important example of such computations is the Multidimensional Positive Definite Advection Transport Algorithm (MPDATA), the second major part of the dynamic core of the EULAG geophysical model. For this aim, we develop a set of parametric optimization techniques and four-step procedure for customization of the MPDATA code. Among these techniques are: islands-of-cores strategy, (3+1)D decomposition, exploiting data parallelism and simultaneous multithreading, data flow synchronization, and vectorization. The proposed adaptation methodology helps us to develop the automatic transformation of the MPDATA code to achieve high sustained scalable performance for all tested ccNUMA platforms with Intel processors of last generations. This means that for a given platform, the sustained performance of the new code is kept at a similar level, independently of the problem size. The highest performance utilization rate of about 41–46% of the theoretical peak, measured for all benchmarks, is provided for any of the two-socket servers based on Skylake-SP (SKL-SP), Broadwell, and Haswell CPU architectures. At the same time, the four-socket server with SKL-SP processors achieves the highest sustained performance of around 1.0–1.1 Tflop/s that corresponds to about 33% of the peak. [ABSTRACT FROM AUTHOR]

Modern heterogeneous computing platforms have become powerful HPC solutions, which could be applied to a wide range of real-life applications. In particular, the hybrid platforms equipped with Intel Xeon Phi coprocessors offer the advantages of massively parallel computing, while supporting practically the same parallel programming model as conventional homogeneous solutions. However, there is still an open issue as to how scientific applications can efficiently utilize hybrid platforms with Intel MIC coprocessors. In this article, we propose an approach for porting a real-life scientific application to such hybrid platforms, assuming no significant modifications of the application code. It allows us to take advantage of all the computing components, including two CPUs and two coprocessors, for the parallel execution of computational workloads. In this study, we focus on the parallel implementation of a numerical model of the dendritic solidification process in isothermal conditions. We develop a sequence of steps that are necessary for the porting and optimization of the solidification application to hybrid platforms with Intel coprocessors. The main challenges include not only overlapping data movements with computations, but also ensuring adequate utilization of cores/threads and vector units of processors, as well as coprocessors. To reach this aim, we propose an efficient and flexible method for the workload distribution between heterogeneous computing components. For implementing the potential benefits of the proposed approach, we choose a heterogeneous programming model based on a combination of the offload mode for Intel MIC and OpenMP programming standard. The developed approach allows us to execute the whole application up to 9.33× faster than the original parallel version that uses two CPUs. Furthermore, the CPU–MIC hybrid platforms enable achieving the speedup of about 1.9× that of the CPU platform with 24 cores based on the Ivy Bridge architecture, and about 1.5× that of the Haswell-based CPU platform with 36 cores. [ABSTRACT FROM AUTHOR]

In this paper, an innovative strategy for the data-flow synchronization in shared-memory systems is proposed. This strategy assumes to synchronize only interdependent threads instead of using the barrier approach that—in contrast to our approach—synchronize all threads. We demonstrate the adaptation of the data-flow synchronization strategy to two complex scientific applications based on stencil codes. An algorithm for the data-flow synchronization is developed and successfully used for both applications. The proposed approach is evaluated for various Intel microarchitectures released in the last 5 years, including the newest processors: Skylake and Knights Landing. The important part of this assessment is the performance comparison of the proposed data-flow synchronization with the OpenMP barrier. The experimental results show that the performance of the studied applications can be accelerated up to 1.3 times using the proposed data-flow synchronizations strategy. [ABSTRACT FROM AUTHOR]

Load balancing is a widely accepted technique for performance optimization of scientific applications on parallel architectures. Indeed, balanced applications do not waste processor cycles on waiting at points of synchronization and data exchange, maximizing this way the utilization of processors. In this paper, we challenge the universality of the load-balancing approach to optimization of the performance of parallel applications. First, we formulate conditions that should be satisfied by the performance profile of an application in order for the application to achieve its best performance via load balancing. Then we use a real-life scientific application, EULAG MPDATA kernel, to demonstrate that its performance profile on a modern parallel architecture, Intel Xeon Phi, significantly deviates from these conditions. Based on this observation, we propose a method of performance optimization of scientific applications through load imbalancing. In the case of data parallel application, the method uses functional performance models of the application to find partitioning that minimizes its computation time but not necessarily balances the load of processors. We apply this method to optimization of MPDATA on Intel Xeon Phi. Experimental results demonstrate that the performance of this carefully optimized load-balanced application can be further improved by 15percent using the proposed load-imbalancing technique. [ABSTRACT FROM PUBLISHER]

EULAG (Eulerian/semi-Lagrangian fluid solver) is an established computational model developed for simulating thermo-fluid flows across a wide range of scales and physical scenarios. The dynamic core of EULAG includes the multidimensional positive definite advection transport algorithm (MPDATA) and elliptic solver. In this work we investigate aspects of an optimal parallel version of the 2D MPDATA algorithm on modern hybrid architectures with GPU accelerators, where computations are distributed across both GPU and CPU components. Using the hybrid OpenMP-OpenCL model of parallel programming opens the way to harness the power of CPU-GPU platforms in a portable way. In order to better utilize features of such computing platforms, comprehensive adaptations of MPDATA computations to hybrid architectures are proposed. These adaptations are based on efficient strategies for memory and computing resource management, which allow us to ease memory and communication bounds, and better exploit the theoretical floating point efficiency of CPU-GPU platforms. The main contributions of the paper are: * method for the decomposition of the 2D MPDATA algorithm as a tool to adapt MPDATA computations to hybrid architectures with GPU accelerators by minimizing communication and synchronization between CPU and GPU components at the cost of additional computations; * method for the adaptation of 2D MPDATA computations to multicore CPU platforms, based on space and temporal blocking techniques; * method for the adaptation of the 2D MPDATA algorithm to GPU architectures, based on a hierarchical decomposition strategy across data and computation domains, with support provided by the developed GPU task scheduler allowing for the flexible management of available resources; * approach to the parametric optimization of 2D MPDATA computations on GPUs using the autotuning technique, which allows us to provide a portable implementation methodology across a variety of GPUs. Hybrid platforms tested in this study contain different numbers of CPUs and GPUs - from solutions consisting of a single CPU and a single GPU to the most elaborate configuration containing two CPUs and two GPUs. Processors of different vendors are employed in these systems - both Intel and AMD CPUs, as well as GPUs from NVIDIA and AMD. For all the grid sizes and for all the tested platforms, the hybrid version with computations spread across CPU and GPU components allows us to achieve the highest performance. In particular, for the largest MPDATA grids used in our experiments, the speedups of the hybrid versions over GPU and CPU versions vary from 1.30 to 1.69, and from 1.95 to 2.25, respectively. [ABSTRACT FROM AUTHOR]

Heterogeneous (or hybrid) computing platforms with Intel Xeon Phi accelerators offer potential advantages of energy efficient, massively parallel computing, while supporting parallel programming models familiar for users of multicore CPUs. However, realizing this potential for real-world applications still remains a challenging issue. The main goal of this paper is the suitability assessment of offload-based programming environments for porting a real-life scientific application to hybrid platforms with Intel KNC and KNL accelerators, assuming no significant modifications of the application code. The main criterion of this assessment is the application performance. The evaluated environments include: 1) Intel Offload coupled with OpenMP, 2) OpenMP 4.0 and 3) OpenMP 4.5 Accelerator Models, and 4) hStreams Library with OpenMP. A real-life application dedicated to the numerical modeling of alloy solidification is used as a testbed in the assessment. An experimental evaluation of the four versions of the application code for a platform with KNC coprocessors shows that excluding OpenMP 4.0, the rest of them are able to adapt to expansion of available resources, however, with different efficiency. While the shortest execution time is achieved for Intel Offload, the high-level abstractions of hStreams contribute considerably to making porting and tuning the application easier, with low performance overheads in comparison to the low-level Intel Offload extension. Benchmarking the application performance and scalability on a platform with multiple KNL processors, using the Offload over Fabric technology with Intel Offload and OpenMP 4.5, concludes the assessment. [ABSTRACT FROM AUTHOR]

• For a real-life scientific application, we assess the impact of five state-of-the-art C/C++ compilers available for novel AMD EPYC processors. • Besides performance, the numerical accuracy of computation is verified using statistical metrics. • For objectivity, we evaluate various versions of the code with the increasing complexity of memory access patterns and control structures. • Our assessment confirms the advantage of the Intel compiler over other compilers. • We show that the Intel compiler's performance advantage over the AMD compiler results from better utilization of AVX2 vector instructions. • The Intel compiler usage permits us to fully reveal the performance potential of the AMD EPYC against the Intel Xeon architecture. The phase-field (PF) method is a powerful tool for solving interfacial problems in materials science. This paper's primary goal is to assess the impact of various state-of-the-art C/C++ compilers available for novel AMD EPYC Rome and Milan multi-core processors on the performance of real-world scientific codes corresponding to the solidification modeling application using the PF method and generalized finite difference scheme for solving governing PDEs. Among the studied compilers are AOCC, Clang, GCC, Intel compiler, and PGI. Besides performance, the numerical accuracy of the simulation is verified since various optimizations used by the compilers can cause differences in the simulation results. For the objectivity of the assessment, we study different application versions with the increasing complexity of memory access patterns and control structures. All tested codes are executed on a dual-socket server equipped with 64-core AMD EPYC Rome 7742 CPUs. This assessment confirms the advantage of the Intel compiler over other compilers. In particular, while for the static intensity only GCC lags noticeably behind other compilers, in the case of the dynamic intensity, the clear winner is the Intel compiler, which allows increasing the performance up to about 1.3 times over the AMD compiler. Moreover, we reveal the sensitivity of the application version and compiler to selecting a different number of NUMA domains and enabling/disabling the NUMA balancing option in EPYC processors. By comparing the performance gain achieved by various compilers due to vectorization of the application codes, we show that the Intel compiler's performance advantage results primarily from considerably better utilization of capabilities of AVX2 vector instructions. Finally, we show the importance of choosing a compiler when comparing the performance of AMD EPYC and Intel Xeon processors. [ABSTRACT FROM AUTHOR]

Abstract: The main delivery of this paper is a model-driven approach to adaptation of the double-precision matrix multiplication to architectures of blade systems based on two types of Cell processors. A hierarchical algorithm used for adaptation consists of four levels. The first level provides sharing computation among all the 16 SPE cores of the IBM BladeCenter QS21 or QS22. The second level corresponds to a macro-kernel, and is responsible for the data management in the main memory, as well as communication between the main memory and local stores of SPE cores. Each macro-kernel operation is implemented within the local store of an SPE core. The third level corresponds to a kernel of the algorithm; each kernel operation is implemented on a single SPE within its local store as a sequence of micro-kernel operations. The fourth level is a micro-kernel implemented within the register file of an SPE core. The proposed approach is based on two performance models. The purpose of the first model is optimization of communication across all 16 SPE cores of the IBM BladeCenter, including the main memory and local stores of SPEs. It is constructed as a function of size of matrix blocks. This model allows for selecting “the best” size of the macro-kernel. The second performance model is aiming at optimization of computations within a single SPE core, taking into account constraints on traffic between the local store and register file of SPE. The model accounts for such factors as size of local store, number of registers, properties of double-precision operations, balance between pipelines, etc. This model allows for selecting “the best” size of kernel and micro-kernel operations. The model-driven adaptation is followed by a series of systematic optimization steps. They include loop unrolling, double buffering on register and memory levels, as well as using NUMA library. The proposed adaptation and optimization steps are fully implemented in C language, without optimizing code manually. For the IBM QS21 system, which uses two Cell processors of the first generation, this implementation allows for achieving 27.24Gflop/s, which is 93.1% of the peak performance. This result is obtained for matrices of size 4096 by 4096. For the IBM QS22 system, based on PowerXCell 8i processors, the performance of double-precision arithmetic is extremely higher, so 184.4Gflop/s is achieved, as 90.0% of the peak performance. This result is reported for the matrix multiplication of size 15,872 by 15,872. The overall performance could be slightly improved by substituting the macro-kernel developed in this work with the highly optimized Cell BLAS dgemm_64x64 kernel. [Copyright &y& Elsevier]

The K-means algorithm is one of the most popular algorithms in Data Science, and it is aimed to discover similarities among the elements belonging to large datasets, partitioning them in K distinct groups called clusters. The main weakness of this technique is that, in real problems, it is often impossible to define the value of K as input data. Furthermore, the large amount of data used for useful simulations makes impracticable the execution of the algorithm on traditional architectures. In this paper, we address the previous two issues. On the one hand, we propose a method to dynamically define the value of K by optimizing a suitable quality index with special care to the computational cost. On the other hand, to improve the performance and the effectiveness of the algorithm, we propose a strategy for parallel implementation on modern multicore CPUs. • For several problems the number of clusters in the K-means algorithm is unavailable. • Adaptive strategies enhance the performance of the K-means algorithm. • Parallel implementation is mandatory for large clustering problems. [ABSTRACT FROM AUTHOR]