Search

Summary: This article introduces a new library that leverages and extends the MPI (message passing interface) standard, capable of aggregating servers and clusters located in multiple domains, as well as cloud resources. The execution platform is created and presented to the MPI application transparently, without the need of recompiling any code. The solution also provides functions for provisioning, adding, and removing nodes at runtime, bringing elasticity to the application. Through benchmarks, its performance was compared with the native execution of the application using the native MPI library. A prototype of an elastic application was also developed, with positive results within the expected range. [ABSTRACT FROM AUTHOR]

Summary: The Oak Ridge Leadership Computing Facility (OLCF) has been preparing the nation's first exascale system, Frontier, for production and end users. Frontier is based on HPE Cray's new EX architecture and Slingshot interconnect and features 74 cabinets of optimized 3rd Gen AMD EPYC CPUs for HPC and AI and AMD Instinct 250X accelerators. As a part of this preparation, "real‐world" user codes have been selected to help assess the functionality, performance, and usability of the system. This article describes early experiences using the system in collaboration with the Hamburg Observatory for two selected codes, which have since been adopted in the OLCF test harness. Experiences discussed include efforts to resolve performance variability and per‐cycle slowdowns. Results are shown for a performance portable astrophysical magnetohydronamics code, AthenaPK, and a mini‐application stressing the core functionality of a performance portable block‐structured adaptive mesh refinement framework, Parthenon‐Hydro. These results show good scaling characteristics to the full system. At the largest scale, the Parthenon‐Hydro miniapp reaches a total of 1.7×1013 zone‐cycles/s on 9216 nodes (73,728 logical GPUs) at ≈92% weak scaling parallel efficiency (starting from a single node using a second‐order, finite‐volume method). [ABSTRACT FROM AUTHOR]

Summary: We present a simple recurrence for the surface area of the (n,k)$$ \left(n,k\right) $$‐star graph, 0

Summary: Voronoi diagram is a versatile geometric structure used in solving many computational geometry problems in the field of image analysis, computer vision, engineering, medicine, molecular biology and various other disciplines. As a result it is a popular research domain. The literature boasts of many elegant and efficient methods for the construction of the Voronoi diagram in the real space. But these algorithms often fail to perform well when applied to the discrete domain. We present in this paper a novel algorithm for the construction of discrete Voronoi diagram using digital geometric concepts that takes care of the geometric perspective of the Voronoi diagram in the discrete domain. Although our technique is more suited for the CPU but by careful use of some cut‐off heuristics we are able to design a parallel programming approach for the construction of the Voronoi diagram making use of the GPU computational power. The speedup of our algorithm is compatible with the popular GPU technique for Voronoi diagram construction, namely, Jump flood algorithm. But since our algorithm uses the geometric perspective for the construction of Voronoi diagram it gives more accurate output and hence requires no correction step as is the case for the Jump flood algorithm. Furthermore, our approach focuses on extensibility, allowing for easy amenability for construction of higher dimensional Voronoi diagram. This ensures that the Voronoi diagram generation process can be adapted and expanded upon to meet specific requirements or incorporate future enhancements. We present an implementation of the proposed algorithm on a GPU using TAICHI language and describe its performance benefits on multiple input sites. [ABSTRACT FROM AUTHOR]

Heterogeneous platforms composed of multiple different types of computing devices (such as CPUs, GPUs, and Intel MICs) have been widely used recently. However, most of parallel applications developed in such a heterogeneous platform usually only utilize a certain kind of computing device due to the lack of easy‐to‐use heterogeneous cooperative parallel programming models. To reduce the difficulty of heterogeneous cooperative parallel programming, a directive‐based heterogeneous cooperative parallel programming framework called HeteroPP is proposed. HeteroPP provides an easier way for programmers to fully exploit multiple different types of computing devices to concurrently and cooperatively perform data‐parallel applications on heterogeneous platforms. An extension to OpenMP directives and clauses is proposed to make it possible for programmers to easily offload a data‐parallel compute kernel to multiple different types of computing devices. A source‐to‐source compiler is designed to help programmers to automatically generate multiple device‐specific compute kernels that can be concurrently and cooperatively performed on heterogeneous platforms. Many experiments are conducted with 12 typical data‐parallel applications implemented with HeteroPP on a heterogeneous CPU‐GPU‐MIC platform. The results show that HeteroPP not only greatly simplifies the heterogeneous cooperative parallel programming, but also can fully utilize the CPUs, GPU, and MIC to efficiently perform these applications. [ABSTRACT FROM AUTHOR]

We present updates to the Cray Graph Engine, a high performance in‐memory semantic graph database, which enable performant execution across multiple architectures as well as deployment in a container to support cloud and as‐a‐service graph analytics. This paper discusses the changes required to port and optimize CGE to target multiple architectures, including Cray Shasta systems, large shared‐memory machines such as SuperDome Flex (SDF), and cluster environments such as Apollo systems. The porting effort focused primarily on removing dependences on XPMEM and Cray PGAS and replacing these with a simplified PGAS library based upon POSIX shared memory and one‐sided MPI, while preserving the existing Coarray‐C++ CGE code base. We also discuss the containerization of CGE using Singularity and the techniques required to enable container performance matching native execution. We present early benchmarking results for running CGE on the SDF, Infiniband clusters and Slingshot interconnect‐based Shasta systems. [ABSTRACT FROM AUTHOR]

Summary: We are concerned with the mapping on high performance hybrid architectures of a parallel software implementing a two level overlapping domain decomposition, that is, along space and time directions, of the four dimensional variational data assimilation model. The reference architecture belongs to the SCoPE (Sistema Cooperativo Per Elaborazioni scientifiche multidisciplinari) data center, located at University of Naples Federico II. We consider the initial boundary problem of the shallow water equation and analyse both strong and weak scaling. Keeping the efficiency always greater than 60%$$ 60\% $$ and about 90%$$ 90\% $$ in most cases, we experimentally find that the isoefficiency function grows a little more than linearly with respect to the number of processes. Results, obtained by using the parallel computing toolbox of MATLABR2013a, are in agreement with the algorithm's performance prevision based on the scale up factor, confirming the appropriate mapping of the algorithm on the hybrid architecture. [ABSTRACT FROM AUTHOR]

Summary: The development of computational fluid dynamics (CFD) highly depends on high‐performance computers. Computer hardware has evolved rapidly, yet scalable CFD parallel software remains scarce. In this article, we design a highly scalable CFD parallel paradigm for both homogeneous and heterogeneous supercomputers. The paradigm achieves the separation of communication and computation and automatically adapts to various solvers and hardware environments, thus reducing programming difficulties and increasing automatic parallelization. Meanwhile, the number of communications is greatly reduced and the scalability of the program is improved through implementing centralized communication and two‐level partitioning techniques. Complex flow problems for real aircraft were then computed on different hardware platforms with a grid size of ten billion. The homogeneous computer hardware includes Intel Xeon Gold 6258R and Phytium 2000+ processors, and the heterogeneous computer platforms include NVIDIA Tesla V100 and SW26010 processors. High parallel efficiency was obtained on all computer platforms, verifying that the paradigm has good automatic parallelization, scalability, and stability. The paradigm in this article has an important reference value for CFD massively parallel computing and can promote the development and application of CFD technology. [ABSTRACT FROM AUTHOR]

Summary: The efficiency of metaheuristic algorithms depends significantly on the number of fitness value evaluations performed on candidate solutions. In addition to various intelligent techniques used to obtain better results, parallelization of calculations can substantially improve the solutions in cases where the problem is NP‐hard and requires many evaluations. This study proposes a new parallel tabu search method for solving the Maximum Vertex Weight Clique Problem (MVWCP) on the Non‐Uniform Memory Access (NUMA) architectures using the OpenMP parallel programming paradigm. Achieving scalability in the NUMA architectures presents significant challenges due to the high complexity of their memory systems, which can lead to performance loss. However, our proposed Tabu‐NUMA algorithm provides up to 18×$$ 18\times $$ speed‐up with 64 cores for ten basic problem instances in DIMACS‐W and BHOSLIB‐W benchmarks. And it improves the performance of the serial Multi Neighborhood Tabu Search (MN/TS) algorithm for 38 problem instances in DIMACS‐W and BHOSLIB‐W benchmarks. We further evaluate our algorithm on larger datasets with thousands of edges and vertices from Network Data Repository benchmark problem instances, and we report significant improvements in terms of speed up. Our results confirm that the Tabu‐NUMA algorithm is among the best recent algorithms for solving MVWCP on the NUMA architectures. [ABSTRACT FROM AUTHOR]

Summary: This article evaluates the current support for heterogeneous OpenMP 5.2 applications regarding the simultaneous activation of host and device computing units (e.g., CPUs, GPUs, or FPGAs). The article identifies limitations in the current OpenMP specification and describes the design and implementation of novel OpenMP extensions and runtime support for heterogeneous parallel programming. The Compute Unit (CUs) abstraction is introduced in the OpenMP programming model. The Compute Unit abstraction is defined in terms of an aggregation of computing elements (e.g., CPUs, GPUs, FPGAs). On top of CUs, the article describes dynamic work sharing constructs and schedulers that address the inherent differences in compute power of host and device CUs. New constructs and the corresponding runtime support are described for the new abstractions. The article evaluates the case of a hybrid multilevel parallelization of the NPB‐MZ benchmark suite. The implementation exploits both coarse‐grain and fine‐grain parallelism, mapped to CUs of different nature (GPUs and CPUs). All CUs are activated using the new extensions and runtime support. We compare hybrid and nonhybrid executions under two state‐of‐the‐art work‐distribution schemes (Static and Dynamic Task schedulers). On a computing node composed of one AMD EPYC 7742 @ 2.250GHz (64 cores and 2 threads/core, totalling 128 threads per node) and 2×$$ \times $$ GPU AMD Radeon Instinct MI50 with 32GB, hybrid executions present speedups from 1.08×$$ \times $$ up to 3.18×$$ \times $$ with respect to a nonhybrid GPU implementation, depending on the number of activated CUs. [ABSTRACT FROM AUTHOR]

SpMV (Sparse matrix‐vector multiplication) is an important computing core in traditional high‐performance computing and also one of the emerging data‐intensive applications. For diagonal sparse matrices, it is frequently necessary to fill in a large number of zeros to maintain the diagonal structure as for using DIA (Diagonal) storage format. The fact that filling with zeros may consume additional computing and memory resources, will certainly lead to degradation of the parallel computing performance of SpMV, further causing computing and storage redundancy. To solve the deficiencies of the DIA format, a Two‐stage parallel SpMV method is presented in this paper, which can reasonably distribute the data of diagonal matrix and irregular matrix to different CUDA kernels. As different corresponding compression methods are particularly designed for different matrix forms, a partition‐based hybrid format of DIA and CSR (HPDC) is therefore adopted in the two‐stage method to ensure load balancing among computing resources and continuity of data access on the diagonal. Simultaneously, a standard deviation among blocks is used as a criterion to obtain the optimal number of blocks and distribution of data. The experimental data were implemented in the Florida data set. Compared to DIA, cuSPARSE‐CSR, HDC, and BRCSD, the execution time of the Two‐stage method is shortened by 4×$$ \times $$, 3.4×$$ \times $$, 1.9×$$ \times $$, and 1.15×$$ \times $$, respectively. [ABSTRACT FROM AUTHOR]

Summary: The wide adoption of SYCL as an open‐standard API for accelerating C++ software in domains such as HPC, automotive, artificial intelligence, machine learning, and other areas necessitates efficient compiler and runtime support for a growing number of different platforms. Existing SYCL implementations provide support for various devices like CPUs, GPUs, DSPs, FPGAs and so forth, typically via OpenCL or CUDA backends. While accelerators have increased the performance of user applications significantly, employing CPU devices for further performance improvement is beneficial due to the significant presence of CPUs in existing data‐centers. SYCL applications on CPUs, currently go through an OpenCL backend. Though an OpenCL backend is valuable in supporting accelerators, it may introduce additional overhead for CPUs since the host and device are the same. Overheads like a run‐time compilation of the kernel, transferring of input/output memory to/from the OpenCL device, invoking the OpenCL kernel and so forth, may not be necessary when running on the CPU. While some of these overheads (such as data transfer) can be avoided by modifying the application, it can introduce disparity in the SYCL application's ability to achieve performance portability on other devices. In this article, we propose an alternate approach to running SYCL applications on CPUs. We bypass OpenCL and use a CPU‐directed compilation flow, along with the integration of whole function vectorization to generate optimized host and device code together in the same translation unit. We compare the performance of our approach—the CPU‐directed compilation flow, with an OpenCL backend for existing SYCL‐based applications, with no code modification for BabelStream benchmark, Matmul from the ComputeCpp SDK, N‐body simulation benchmarks and SYCL‐BLAS (Aliaga et al. Proceedings of the 5th International Workshop on OpenCL; 2017.), on CPUs from different vendors and architectures. We report a performance improvement of up to 72%$$ 72\% $$ on BabelStream benchmarks, up to 63%$$ 63\% $$ on Matmul, up to 21%$$ 21\% $$ on the N‐body simulation benchmark and up to 16% on SYCL‐BLAS. [ABSTRACT FROM AUTHOR]

Summary: In the article, we have proposed a framework that allows programming a parallel application for a multi‐node system, with one or more graphical processing units (GPUs) per node, using an OpenMP+extended CUDA API. OpenMP is used for launching threads responsible for management of particular GPUs and extended CUDA calls allow to transfer data and launch kernels on local and remote GPUs. The framework hides inter‐node MPI communication from the programmer. For optimization, the implementation takes advantage of the MPI_THREAD_MULTIPLE mode allowing: multiple threads handling distinct GPUs as well as overlapping communication and computations transparently using multiple CUDA streams. The solution allows data parallelization across available GPUs in order to minimize execution time and supports a power‐aware mode in which GPUs are automatically selected for computations using a greedy approach in order not to exceed an imposed power limit. We have implemented and benchmarked three parallel applications including: finding the largest divisors; verification of the Collatz conjecture; finding patterns in vectors. These were tested on three various systems: a GPU cluster with 16 nodes, each with NVIDIA GTX 1060 GPU; a powerful 2‐node system—one node with 8×$$ \times $$ NVIDIA Quadro RTX 6000 GPUs, the second with 4×$$ \times $$ NVIDIA Quadro RTX 5000 GPUs; a heterogeneous environment with one node with 2×$$ \times $$ NVIDIA RTX 2080 and 2 nodes with NVIDIA GTX 1060 GPUs. We demonstrated effectiveness of the framework through execution times versus power caps within ranges of 100–1400 W, 250–3000 W, and 125–600 W for these systems respectively as well as gains from using two versus one CUDA streams per GPU. Finally, we have shown that for the testbed applications the solution allows to obtain high speed‐ups between 89.3% and 97.4% of the theoretically assessed ideal ones, for 16 nodes and 2 CUDA streams, demonstrating very good parallel efficiency. [ABSTRACT FROM AUTHOR]

Summary: While heterogeneous computing has emerged as a dominant trend in current and future High‐Performance Computing (HPC) systems, it is also widely recognized that this shift has led to increased software complexity due to a proliferation of programming systems for different heterogeneous processors. One such example is the Heterogeneous‐Compute Interface for Portability from AMD (HIP), which is composed of a C Runtime API and C++ Kernel Language. Many HPC applications will likely use HIP on future exascale systems (e.g., Frontier and El Capitan), but HIP currently only targets AMD and NVIDIA processors. This limitation creates challenges for users who would also like to run their applications on exascale systems based on other architectures (e.g., Aurora, which is based on Intel hardware) that are currently not targeted by HIP. In this paper, we introduce the design and implementation of HIPLZ , a compiler and runtime system that uses the Intel Level Zero API to support HIP on Intel GPU architectures. We discuss the design of HIPLZ , derived from HIPCL (an implementation of HIP on top of OpenCL), and portability issues that occur from using the Level Zero runtime as a backend. We evaluate our implementation by running several performance benchmarks and mini‐apps written in HIP on Intel architectures using HIPLZ. Our results show that this approach provides competitive performance relative to Intel's OpenCL implementations on Intel Gen9 and UHD Graphics 770 GPUs, while providing good coverage of features needed by HPC applications. Overall, this approach is a promising demonstration of enabling performance portability for exascale systems. [ABSTRACT FROM AUTHOR]

Summary: Sparse triangular solve (SpTRSV) is an extensively studied computational kernel. An important obstacle in parallel SpTRSV implementations is that in some parts of a sparse matrix the computation is serial. By transforming the dependency graph, it is possible to increase the parallelism of the parts that lack it. In this work, we present a novel graph transformation strategy to increase the parallelism degree of a sparse matrix and compare it to our previous strategy. It is seen that our transformation strategy can provide a speedup as high as 1.42x$$ 1.42x $$. [ABSTRACT FROM AUTHOR]

Lightweight manycore processors deliver high performance and energy efficiency by bundling hundreds of low‐power cores, a distributed memory architecture with small local memories and Networks‐on‐Chip in a single die. However, the lack of rich and portable programming models for these processors makes software development a challenging task. Currently, two approaches are employed to address programmability in lightweight manycores: Operating Systems (OSes) and baremetal runtime libraries. The former provides portability but exposes complex Operating System (OS)‐level programming interfaces to developers. The latter focuses on providing rich and high performance interfaces, which are vendor‐specific and yield to non‐portable software. In this work, we address these programmability and portability challenges by combining a rich OS with a well‐known standard for parallel programming. We propose a portable and lightweight Message Passing Interface (MPI) library (LWMPI) designed from scratch to cope with restrictions and intricacies of lightweight manycores. We integrated LWMPI into Nanvix, an open‐source distributed OS that runs on silicon lightweight manycores. The results obtained with a synthetic benchmark and a subset of the CAP Bench applications running on Kalray MPPA‐256 unveil that LWMPI not only delivers a lightweight and richer programming interface but also presents good performance and scalability results. [ABSTRACT FROM AUTHOR]

Large‐scale computing and data processing with cloud resources is gaining popularity. However, the usage of the cloud differs from traditional high‐performance computing (HPC) systems and both algorithms and codes have to be adjusted. This work is often time‐consuming and performance is not guaranteed. To address this problem we have developed the PCJ library (parallel computing in Java), a novel tool for scalable HPC and big data processing in Java. In this article, we present a performance evaluation of parallel applications implemented in Java using the PCJ library. The performance evaluation is based on the examples of highly scalable applications of different characteristics focusing on CPU, communication or I/O. They run on the traditional HPC system and Amazon web services Cloud as well as Linaro Developer Cloud. For the clouds, we have used Intel x86 and ARM processors for running Java codes without changing any line of the program code and without the need for time‐consuming recompilation. Presented applications have been parallelized using the partitioned global address space programming model and its realization in the PCJ library. Our results prove that the PCJ library, due to its performance and ability to create simple portable code, has great promise to be successful for the parallelization of various applications and run them on the cloud with a performance close to HPC systems. [ABSTRACT FROM AUTHOR]

Certain workloads such as in‐memory databases are inherently hard to scale‐out and rely on cache‐coherent scale‐up non‐uniform memory access (NUMA) systems to keep up with the ever‐increasing demand for compute resources. However, many parallel programming frameworks such as OpenMP do not make efficient use of large scale‐up NUMA systems as they do not consider data locality sufficiently. In this work, we present PGASUS, a C++ framework for NUMA‐aware application development that provides integrated facilities for NUMA‐aware task parallelism and data placement. The framework is based on an extensive review of parallel programming languages and frameworks to incorporate the best practices of the field. In a comprehensive evaluation, we demonstrate that PGASUS provides average performance improvements of 1.56× and peak performance improvements of up to 4.67×across a wide range of workloads. [ABSTRACT FROM AUTHOR]

Summary: The Floyd‐Warshall algorithm is a well‐known algorithm to compute the distance of all pairs of nodes of a graph. The Blocked Floyd‐Warshall algorithm, a variant of the Floyd‐Warshall has been proposed to accelerate the Floyd‐Warshall algorithm by means of a graphics processing unit (GPU) architecture. The previously published GPU implementations for the Blocked Floyd‐Warshall algorithm perform many separated kernel calls for costly barrier synchronization. The main contribution of this article is to present efficient implementations of the Blocked Floyd‐Warshall algorithm, which performs no barrier synchronization and invokes only one kernel call. Experimental results using NVIDIA Tesla V100 show that our implementation runs 1.05‐1.31 times faster than the previously published one. Our implementation with SIMD functions also runs 1.00‐1.28 times faster than it. Second, we propose efficient GPU implementations to execute the Blocked Floyd‐Warshall algorithm for many graphs at the same time. From the experimental results, our single kernel implementation runs 1.03‐1.60 times faster than multiple kernel one. In terms of implementations with SIMD functions, our single kernel implementation runs 1.01‐1.89 times faster than it. We also propose the low‐latency implementations for many graphs. Finally, we implemented the parallel Floyd‐Warshall algorithm on the multicore processors. [ABSTRACT FROM AUTHOR]

As an improved method of lattice Boltzmann method (LBM), regularized lattice Boltzmann method (RLBM) has been applied to simulate fluid flow. Nevertheless, the performance of RLBM needs to be considered when simulating actual problems. The rise of multicore platforms, especially the popularity of graphics processor units (GPUs), has provided possible implementation solutions for parallel computing. In this article, an RLBM parallel model on the CPU/GPU heterogeneous platforms is proposed. To solve the problem of possible GPU memory shortage, the CPU controls the startup of the kernel function in the RLBM algorithm and participates in the calculation. Due to the characteristics of the algorithm, the entire flow field is divided into CPU computing areas and GPU computing areas according to the given flow field division rules. OpenMP and CUDA are, respectively, applied to CPU and GPU for parallel computing. The startup of the kernel function takes a short time, and the CPU and GPU can be approximately regarded as performing calculations simultaneously. Since the data exchange between the CPU and GPU has a significant impact on performance, a buffer is set at the boundary of the CPU and GPU to reduce the frequency of data exchange. The buffer size determines the number of program iterations before exchanging data. The performance of the algorithm is measured by MFLUPS, and the algorithm is applied to the 3D lid‐driven cavity flow. The obtained results show that the MFLUPS of the algorithm is 25 times that of CPU MFLUPS, and it adds an increment equivalent to CPU MFLUPS compared with GPU MFLUPS. And the algorithm is extended to a multi‐GPU version, which is also applied to the 3D lid‐driven cavity flow. The results obtained show that the MFLUPS of the multi‐GPU version is 1.8 times that of the single GPU. [ABSTRACT FROM AUTHOR]

The efficiency of the simulations with the discrete element method (DEM) is significantly improved using a novel computational strategy. The new method is developed with a focus on platforms equipped with multi‐core central processing units (CPU) and general‐purpose graphics processing units (GPU). The DEM calculations are performed in parallel on the CPU and on the GPU using pre‐calculated Verlet lists with a posteriori analysis of their consistency. The operations related to the search for possible contacts are performed on the CPU, whereas the processing of interactions, and integration of motion, are executed on the GPU. Performance analysis done for various types of tasks has shown that the new method allows to significantly decrease the average computational time and to utilize available computational resources more efficiently compared to the sequential CPU–GPU execution mode. Furthermore, due to more efficient calculations, the overall energy requirement for the proposed strategy does not exceed the demand for conventional sequential CPU–GPU computations. [ABSTRACT FROM AUTHOR]

Summary: In the direction of arrival (DoA) estimation, typically sensor arrays are used where the number of required sensors can be large depending on the application. With the help of compressed sensing (CS), hardware complexity of the sensor array system can be reduced since reliable estimations are possible by using the compressed measurements where the compression is done by measurement matrices. After the compression, DoAs are reconstructed by using sparsity promoting algorithms such as alternating direction method of multipliers (ADMM). For the given procedure, both the measurement matrix design and the reconstruction algorithm may include computationally intensive operations, which are addressed in this study. The presented simulation results imply the feasibility of the system in real‐time processing with energy efficient implementations. We propose employing parallel programming to satisfy the real‐time processing requirements. While the measurement matrix design has been accelerated 16× with CPU based parallel version with respect to the fastest serial implementation, ADMM based DoA estimation has been improved 1.1× with GPU based parallel version compared to the fastest CPU parallel implementation. In addition, we achieved, to the best of our knowledge, the first energy‐efficient real‐time DoA estimation on embedded Jetson GPGPUs in 15 W power consumption without affecting the DoA accuracy performance. [ABSTRACT FROM AUTHOR]

Summary: A recurrent challenge in real‐world applications is autonomous management of the executions at run‐time. In this vein, stream processing is a class of applications that compute data flowing in the form of streams (e.g., video feeds, images, and data analytics), where parallel computing can help accelerate the executions. On the one hand, stream processing applications are becoming more complex, dynamic, and long‐running. On the other hand, it is unfeasible for humans to monitor and manually change the executions continuously. Hence, self‐adaptation can reduce costs and human efforts by providing a higher‐level abstraction with an autonomic/seamless management of executions. In this work, we aim at providing a literature review regarding self‐adaptation applied to the parallel stream processing domain. We present a comprehensive revision using a systematic literature review method. Moreover, we propose a taxonomy to categorize and classify the existing self‐adaptive approaches. Finally, applying the taxonomy made it possible to characterize the state‐of‐the‐art, identify trends, and discuss open research challenges and future opportunities. [ABSTRACT FROM AUTHOR]

Summary: The applications of computer vision (CV) are continuously increasing along with the enormous demand for real‐time data processing. This visual data processing is done with various compute‐intensive image/video processing algorithms that may belong to traditional approaches or deep learning approaches. This article aims to provide a survey of state‐of‐the‐art hardware platforms and software frameworks for parallel implementation of traditional CV applications. The article discusses various options for hardware platforms for centralized‐computing architecture and edge‐computing architecture, and various software frameworks that can be used to leverage the hardware. This discussion is based on a systematic survey of studies/works that show the use of various hardware platforms and software frameworks in order to achieve real‐time processing for CV algorithms. Based on the survey, some possible future directions are also discussed. [ABSTRACT FROM AUTHOR]

Summary: With the development of genome sequencing technology, the cost of genome sequencing is continuously reducing, while the efficiency is increasing. Therefore, the amount of genomic data has been increasing exponentially, making the transmission and storage of genomic data an enormous challenge. Although many excellent genome compression algorithms have been proposed, an efficient compression algorithm for large collections of FASTA genomes, especially can be used in the distributed system of cloud computing, is still lacking. This article proposes two optimization schemes based on HRCM compression method. One is MtHRCM adopting multi‐thread parallel technology. The other is HadoopHRCM adopting distributed computing parallel technology. Experiments show that the schemes recognizably improve the compression speed of HRCM. Moreover, BSC algorithm instead of PPMD algorithm is used in the new schemes, the compression ratio is improved by 20% compared with HRCM. In addition, our proposed methods also perform well in robustness and scalability. The Java source codes of MtHRCM and HadoopHRCM can be freely downloaded from https://github.com/haicy/MtHRCM and https://github.com/haicy/HadoopHRCM. [ABSTRACT FROM AUTHOR]

Summary: Heterogeneous architectures proved successful in achieving unprecedented performance and energy‐efficiency. However, taking advantage of these diverse processing elements is still hard. Programmers need to code through the different approaches suitable for each target architecture and need to decide the distribution of activities on the different resources. The majority of current frameworks focuses on either performance or productivity. The former mainly provides low‐level target‐specific programming interfaces, and the latter offers high‐level tools that often fail in achieving high‐performance. In both cases, the design is usually data‐parallel, as task‐parallelism is not supported. In this work, we propose a task‐based solution within the data‐parallel heterogeneous single‐source PHAST library. Tasks can be coded in a target‐agnostic fashion, can be compiled and parallelized on multi‐core CPUs and NVIDIA GPUs automatically and support the choice of the execution platform at runtime. We evaluate the capabilities of the proposed task‐directed acyclic graph support in case of an extensive set of randomly generated task‐based applications with different sizes and characteristics. We compare it against a SYCL implementation in terms of performance and complexity metrics, highlighting that PHAST achieves about 1.56× and 2.60× speedup over SYCL for multi‐core CPU and GPU, respectively, while improving also code complexity metrics. [ABSTRACT FROM AUTHOR]

Therefore, innovative models, algorithms, programming environments and tools are required to efficiently tackle the challenges and fully exploit the resources of modern parallel and heterogeneous platforms. Traditional parallel algorithms, programming environments and tools designed for legacy homogeneous multiprocessors achieve at best a small fraction of the efficiency and performance expected from highly heterogeneous parallel computing systems. [Extracted from the article]

Summary: Computing methodologies with rapid proliferation and need for greater concurrency have evolved over the years from parallel and distributed computing to latest technologies like cloud computing. A paradigm shift in computing methodologies has been witnessed in the form of exponential speed up, optimization of problem solutions, and solvability of few classically unsolvable problems with emergence of quantum computing. The study in this article presents observations on executing basic quantum operations that play a vital role in quantum computing. The results and analysis demonstrate how parameters of time, deviation, and shots have a significant effect on outcomes of quantum circuits executed which can be a reference point for the community in general engaged in designing of quantum circuits, protocols, algorithms, and quantum hardware to leverage quantum characteristics and properties. The study by means of experimental use cases provides a rigorous review of qubit behavior, standard, and entangled qubit measurement with practical results over a simulator and real quantum hardware for variable qubit configurations using a selected platform of IBM Q Experience. With the paradigm shift from classical to quantum computing, this article provides basic but very vital observations which can be brought to use by anyone intending to build quantum applications. [ABSTRACT FROM AUTHOR]

Summary: Efforts to support high performance computing (HPC) applications' requirements in the context of cloud computing have motivated us to design HPC Shelf, a cloud computing services platform to build and deploy large‐scale parallel computing systems. We introduce Alite, the contextual contract system of HPC Shelf, to select component implementations according to requirements of the host application, target parallel computing platform characteristics (e.g., clusters and MPPs), quality of service (QoS) properties, and cost restrictions. It is evaluated through a small‐scale case study employing two complementary component‐based frameworks. The first one aims to represent components that implement linear algebra computations based on the BLAS interface. In turn, the second one aims to represent parallel computing platforms on the IaaS cloud offered by Amazon EC2 Service. [ABSTRACT FROM AUTHOR]

Summary: As the number of cores equipped in a single computing node is rapidly increasing, utilizing many cores for contemporary applications in an efficient manner is a challenging issue. We need to consider both parallelization and locality to fully exploit many cores for multifarious operations of emerging applications. In particular, big data applications perform computation and I/O intensive operations alternately. For instance, Apache Hadoop MapReduce assumes local persistent storage for each computing node. Thus, unlike traditional parallel programming models, the MapReduce framework performs not only networking but also storage I/O. In this study, we aim to improve the locality of network and storage I/O operations on many‐core systems by partitioning cores for I/O system calls and event handlers. In order to implement fine‐grained many‐core partitioning, we decouple the system call context from the user‐level process by suggesting message‐based system calls. The suggested design provides user‐level transparency and does not require any kernel‐level modifications. In addition, we propose a scheme that dynamically decides the core affinity of system calls and event handlers by considering locality, run‐time loads, and hardware architectures. The experimental results show that the proposed many‐core partitioning can improve the locality of network and storage I/O operations in an integrated manner for MapReduce applications. [ABSTRACT FROM AUTHOR]

Memory prefetcher algorithms are widely used in processors to mitigate the performance gap between the processors and the memory subsystem. The complexities behind the architectures and prefetcher algorithms, however, not only hinder the development of accurate architecture simulators, but also hinder understanding the prefetcher's contribution to performance, on both a real hardware and in a simulated environment. In this paper, we contribute to shed light on the memory prefetcher's role in the performance of parallel High‐Performance Computing applications, considering the prefetcher algorithms offered by both the real hardware and the simulators. We performed a careful experimental investigation, executing the NAS parallel benchmark (NPB) on a real Skylake machine, and as well in a simulated environment with the ZSim and Sniper simulators, taking into account the prefetcher algorithms offered by both Skylake and the simulators. Our experimental results show that: (i) prefetching from the L3 to L2 cache presents better performance gains, (ii) the memory contention in the parallel execution constrains the prefetcher's effect, (iii) Skylake's parallel memory contention is poorly simulated by ZSim and Sniper, and (iv) Skylake's noninclusive L3 cache hinders the accurate simulation of NPB with the Sniper's prefetchers. [ABSTRACT FROM AUTHOR]

Algorithms for the extraction of formal concepts are widely studied in several areas of knowledge, such as finance, health, and statistics. However, these algorithms require high‐performance processing due to their combinatorial characteristics. In this work, an Open computing language (OpenCL)‐based Brute Force algorithm is proposed and evaluated for formal concept extraction on heterogeneous architectures (CPU+GPU and CPU+FPGA). The CPU+GPU architecture presents higher performance and scalability than other architectures when our Brute Force algorithm processes high dimensional contexts with many objects and attributes. Our parallel approach shows performance results up to 18× better than a smarter sequential algorithm called Data‐Peeler. Moreover, our Brute Force algorithm running on CPU+GPU architecture has greater energy efficiency, reaching at least 1.79× more operations per energy consumption than other algorithms on different architectures explored in this work. [ABSTRACT FROM AUTHOR]

There is a growing demand for correct parallel programs, mainly due to nowadays availability of parallel architectures. Structural testing allows identifying defects by analyzing the internal structures of a program. However, communication and synchronization in parallel programs bring new challenges to the testing activity, such as nondeterminism. Message‐passing parallel programs require structural testing criteria to support test models and tools capable of covering synchronization events with dynamic behaviors. Testing such primitives inside loops in message‐passing programs is still challenging with nontrivial solutions for criteria, models, and tools. This article proposes new structural testing criteria to guide the selection of test cases, improving the quality of message‐passing programs by revealing nondeterminism‐related defects present in loops. We present a new test model to support our criteria for structural testing of MPI‐applications and implement the proposed criteria in the tool ValiMPI. The analysis of nondeterminism‐related defects, paths inside loops, loop iterations, and nested loops allow us to establish a structured testing model. We validate the testing criteria through experimental studies using ValiMPI. Our results show that our criteria reveal unknown defects from communication and synchronization events in different loop iterations, increasing the quality of message‐passing parallel programs. [ABSTRACT FROM AUTHOR]

Summary: The Exascale Computing Project (ECP) focuses on the development of future exascale‐capable applications. Most ECP applications use the message passing interface (MPI) as their parallel programming model with mini‐apps serving as proxies. This paper explores the explicit usage of MPI in such ECP proxy applications. We empirically analyze 14 proxy applications from the ECP Proxy Apps Suite. We use the MPI profiling interface (PMPI) to collect MPI usage patterns in ECP proxy apps. Our analysis shows that a small subset of features from MPI is commonly used in the proxies of exascale‐capable applications, even when they reference third‐party libraries. This study is intended to provide a better understanding of the use of MPI in current exascale applications. The findings can help focus software investments made for exascale systems in the MPI middleware including optimization, fault‐tolerance, tuning, and hardware‐offload. [ABSTRACT FROM AUTHOR]

Summary: With the increase in size and complexity of high‐performance computing systems, the probability of failures, and the cost of recovery grow. Parallel applications running on these systems should be able to continue running in spite of node failures at arbitrary times. Collective operations are essential for many parallel MPI applications, and are often the first to detect such failures. This work presents tree‐based fault‐tolerant collective operations, which combine fault detection and recovery as an integral part each operation. We do this by extending existing tree‐based algorithms, to allow for a collective operation to succeed despite failing nodes before or during its run. This differs from other approaches, where recovery takes place after a failure of such operations have failed. The article includes a comparison between the performance of the proposed algorithm and other approaches, as well as a simulator‐based analysis of performance at scale. [ABSTRACT FROM AUTHOR]

Summary: This paper presents semi‐automatic software refactorings to introduce and tune structured parallelism in sequential Erlang code, as well as to generate code for running computations on GPUs and possibly other accelerators. Our refactorings are based on the lapedo framework for programming heterogeneous multi‐core systems in Erlang. lapedo is based on the PaRTE refactoring tool and also contains (1) a set of hybrid skeletons that target both CPU and GPU processors, (2) novel refactorings for introducing and tuning parallelism, and (3) a tool to generate the GPU offloading and scheduling code in Erlang, which is used as a component of hybrid skeletons. We demonstrate, on four realistic use‐case applications, that we are able to refactor sequential code and produce heterogeneous parallel versions that can achieve significant and scalable speedups of up to 220 over the original sequential Erlang program on a 24‐core machine with a GPU. [ABSTRACT FROM AUTHOR]

Summary: The fulfillment of non‐functional requirements like timing or energy consumption is of utmost importance in many embedded systems and respective applications. Especially, with the introduction of multi‐core architectures, the ability to predict non‐functional execution qualities becomes more and more difficult, as multiple concurrent application programs may interfere in execution when typically sharing all the resources. In this paper, we advocate a novel parallel computing paradigm called invasive computing that allows to isolate application programs on multi‐core targets. For a presented case study of a cyber‐physical real‐time control system, we show that invasive computing enables composability that in fact allows to characterize and analyze each application program statically and independent from each other. More specifically, it is shown that a distributed object detection algorithm for controlling an inverted pendulum and implemented on a heterogeneous invasive multi‐processor SoC (MPSoC) is able to provide real‐time guarantees as well as reliability requirements on demand. [ABSTRACT FROM AUTHOR]

Summary: As core counts in processors increases, it becomes harder to schedule and distribute work in a timely and scalable manner. This article enhances the scalability of parallel loop schedulers by specializing schedulers for fine‐grain loops. We propose a low‐overhead work distribution mechanism for a static scheduler that uses no atomic operations. We integrate our static scheduler with the Intel OpenMP and Cilkplus parallel task schedulers to build hybrid schedulers. Compiler support enables efficient reductions for Cilk, without changing the programming interface of Cilk reducers. Detailed, quantitative measurements demonstrate that our techniques achieve scalable performance on a 48‐core machine and the scheduling overhead is 43% lower than Intel OpenMP and 12.1× lower than Cilk. We demonstrate consistent performance improvements on a range of HPC and data analytics codes. Performance gains are more important as loops become finer‐grain and thread counts increase. We observe consistently 16%–30% speedup on 48 threads, with a peak of 2.8× speedup. [ABSTRACT FROM AUTHOR]

With the increase of the search for computational models where the expression of parallelism occurs naturally, some paradigms arise as options for the current generation of computers. In this context, dynamic dataflow and Gamma—General Abstract Model for Multiset mAnipulation—emerge as interesting computational model choices. In dynamic dataflow model, operations are performed as soon as their associated operands are available, without rely on a Program Counter to dictate the execution order of instructions. The Gamma paradigm is based on a parallel multiset rewriting scheme. It provides a nondeterministic execution model inspired by an abstract chemical machinemetaphor, where operations are formulated as reactions that occur freely among matching elements belonging to the multiset. In this work, equivalence relations between the dynamic dataflow and Gamma paradigms are exposed and explored, while methods to convert from dataflow to Gamma paradigm and vice versa are provided. It is shown that vertices and edges of a dynamic dataflow graph can correspond, respectively, to reactions and multiset elements in the Gamma paradigm. This work provides the scientific community with the possibility of taking profit of both parallel programming models, contributing with a versatility component to researchers and developers. [ABSTRACT FROM AUTHOR]

Summary: This paper presents how parallel advanced graph partitioning algorithms can be designed and improved with the use of global application states monitoring of distributed programs. The proposed algorithms have been implemented inside a novel distributed program design framework Program Execution Governed by Asynchronous Supervision of States in Distributed Applications (PEGASUS DA). This framework provides system support for the design of execution control in distributed applications based on automated global state monitoring. Two strategies for the control design of advanced parallel/distributed graph partitioning algorithms are presented and discussed. In the first one, the parallel algorithm control runs on top of the popular graph partitioning METIS tool. The second control strategy is based on genetic programming so that partitioning primitives and the overall algorithmic control can be freely designed by the user. Advanced partitioning methods have been conveniently embedded inside the global state monitoring driven PEGASUS DA framework which controls partitioning distributed actions at the level of processes and threads. The use of the framework allowed easy design and testing of different graph optimization strategies. The presented graph partitioning methods are illustrated by experiments with benchmark graphs. The experiments have comparatively assessed the obtained graph partitioning quality (visible improvement has been observed) and have identified benefits of the proposed approach for programmers. [ABSTRACT FROM AUTHOR]

Summary: Stream processing is a parallel paradigm used in many application domains. With the advance of graphics processing units (GPUs), their usage in stream processing applications has increased as well. The efficient utilization of GPU accelerators in streaming scenarios requires to batch input elements in microbatches, whose computation is offloaded on the GPU leveraging data parallelism within the same batch of data. Since data elements are continuously received based on the input speed, the bigger the microbatch size the higher the latency to completely buffer it and to start the processing on the device. Unfortunately, stream processing applications often have strict latency requirements that need to find the best size of the microbatches and to adapt it dynamically based on the workload conditions as well as according to the characteristics of the underlying device and network. In this work, we aim at implementing latency‐aware adaptive microbatching techniques and algorithms for streaming compression applications targeting GPUs. The evaluation is conducted using the Lempel‐Ziv‐Storer‐Szymanski compression application considering different input workloads. As a general result of our work, we noticed that algorithms with elastic adaptation factors respond better for stable workloads, while algorithms with narrower targets respond better for highly unbalanced workloads. [ABSTRACT FROM AUTHOR]

Summary: Web directories like Wikipedia and Open Directory Mozilla facilitate efficient information retrieval (IR) of web documents from a huge web corpus. Maintenance of these web directories is understandably a difficult task that requires manual curation by human editors or semi‐automated mechanisms. Research on parallel algorithms for the automated curation of these web directories will be beneficial to the IR domain. Hence, in this article, we propose a parallel algorithm for automatically creating web directories from a corpus of web‐documents. We have used centrality‐based techniques to split the corpus into fine‐grained clusters and subsequently an agglomeration based on locality sensitive hashing to identify coarse‐grained clusters in the web‐directory. Experimental results show that the algorithm generates meaningful hierarchies of the input corpus as measured by cluster‐validity indices, like F‐measure, rand index, and cluster purity. The algorithm achieves a significant speedup and scales well both with the number of processors and the size of the input corpus. [ABSTRACT FROM AUTHOR]

Summary: With the increasing size of the RDF dataset, the application of parallel computing frameworks is applying more and more widely in big data. In order to realize the parallel query of data, an edge partitioning algorithm is proposed based on RDF sentence graph. Firstly, the algorithm transforms the RDF graph into an RDF sentence graph using coarsening and partitioning. Secondly, the minimum degree vertex partitioning algorithm is put forward to partition the RDF sentence graph. Finally, according to the equivalence thought between edge segmentation of RDF sentence and vertex segmentation of RDF graph, the intersection between RDF subgraphs is the vertex cut set of RDF graph to achieve parallel query of RDF data. The experimental results show that the algorithm's partitioning time and efficiency are better than the traditional algorithms. [ABSTRACT FROM AUTHOR]

The proposed work introduces a fully automatic modified fuzzy c‐means (MFCM) algorithm for segmenting brain tissue into gray matter (GM), white matter (WM), and cerebrospinal fluid (CSF) which identifies the pathological conditions of magnetic resonance human head scans. The present work implements histogram smoothing using Gaussian distribution for finding the number of clusters (K) and cluster centers (C) to initialize modified FCM algorithm (MFCM). The modification includes the local impact of each pixel based on the median of local neighborhoods. This needs more computational power to reduce the processing time and requires a parallel programming environment like the Graphics Processing Unit. The parallel MFCM is performed with the help of compute unified device architecture language and reduced the processing time up to 80 speedup folds than the serial implementation in Matlab and 20 speedup folds than C programming implementation. The method is examined with the Internet Brain Segmentation Repository (IBSR20) T1W dataset. The quantitative and qualitative results of the proposed method are compared with state‐of‐the‐art‐methods using the Dice coefficient (DC). Proposed method yields high DC 0.84 ± 0.03 for GM, 0.83 ± 0.04 for WM, and 0.41 ± 0.12 for CSF segmentation. In post‐processing, 3D volumes of segmented regions have been constructed and compared with the gold standard quantitatively and qualitatively. [ABSTRACT FROM AUTHOR]

Summary: Radix sorting is an essential basic data processing operation in many computer fields. It has important practical significance to accelerate its performance through Graphic Processing Unit (GPU). The heterogeneous parallel computing technology attracts much attention and is widely applied for its effective computation efficiency and parallel real‐time data processing capability. Taking advantage of the parallelism of GPU in numerical computation processing, a parallelization design method of the Binary_Least Significant Digit (LSD) first Radix Sorting (B_LSD_RS) algorithm based on Open Computing Language (OpenCL) is proposed. The radix sorting algorithm is divided into multiple kernel tasks, and the kernels are sequentially controlled by the event information transfer. The parallel algorithm is implemented and verified on the GPU + CPU heterogeneous platform. The experimental results show that compared with the performance of the B_LSD_RS sequential algorithm based on AMD Ryzen5 1600X CPU, B_LSD_RS parallel algorithm based on Open Multi‐Processing (OpenMP) and B_LSD_RS parallel algorithm based on Compute Unified Device Architecture (CUDA), the B_LSD_RS parallel algorithm based on OpenCL obtained 28.86 times, 11.01 times and 2.14 times speedup in the NVIDIA GTX 1070 computing platform respectively, not only achieves high performance but also achieves performance portability among different GPU computing platforms. [ABSTRACT FROM AUTHOR]

Summary: The field of telecommunication has undergone a very rapid technological evolution, which has forced researchers to find techniques that allow better exploitation of hardware and software. Among the proposed technologies, cognitive radio, a concept that was designed after several technologies such as software radio. Cognitive radio has been widely used for opportunistic access of the shared spectrum and has defined the cognitive nodes by their ability to intelligently adapt the environment to achieve specific objectives through advanced techniques. In this context, clustering techniques were adopted in cognitive radio networks (CRNs) due to their great advantages especially for routing. In this article, we propose a parallel mode of the k‐NN algorithm. The aim is to make a fast assignment of radio nodes in CRNs organized in the form of clusters. The obtained results are very satisfactory because we have been able to reduce to about 50% the execution time of the basic algorithm (sequential). [ABSTRACT FROM AUTHOR]

The selected papers cover a good spectrum of research topics in the area of heterogeneous computing showing the challenges present on these modern platforms. Traditional parallel algorithms, programming environments, and tools, designed for legacy homogeneous multiprocessors, achieve a small fraction of the efficiency and the potential performance that can be obtained in current and future heterogeneous computing platforms. [Extracted from the article]

Summary: A parallel implementation of an agent‐based, artificial life model for graphical processing units (GPUs) is presented. The model represents predator‐prey populations as a collection of agents (animats) who act independently and interact with their environment and each other. Individual animat interactions include avoidance, mating, and hunting which contribute to emergent behavior in the system as a whole such as herd formation and defensive spirals. Computational simulations of the model exploring the behavior of the model are limited in size due to the large populations that may result from certain environments and rule combinations. Parallel methods for computing the model on GPUs and a CUDA implementation are presented and are shown to provide a significant performance improvement over existing sequential implementations without simplifying the model behavior. The reduced simulation run‐time can allow both larger systems to be computed and a wider range of rule combinations that may produce larger populations to be investigated in a feasible amount of time. [ABSTRACT FROM AUTHOR]

Summary: In order to take advantage of the processing power of current computing platforms, programmers typically need to develop software versions for different target devices. This task is time‐consuming and requires significant programming and computer architecture expertise. A possible and more convenient alternative is to start with a single high‐level description of a program with minimum implementation details, and generate custom implementations according to the target platform. In this paper, we use MATLAB as a high‐level programming language and propose a compiler that targets CPU/GPU computing platforms by generating customized implementations in C and OpenCL. We propose a number of compiler techniques to automatically generate efficient C and OpenCL code from MATLAB programs. One of such compiler techniques relies on heuristics to decide when and how to use Shared Virtual Memory (SVM). The experimental results show that our approach is able to generate code that provides significant speedups (eg, geometric mean speedup of 11× for a set of simple benchmarks) using a discrete GPU over equivalent sequential C code executing on a CPU. With more complex benchmarks, for which only some code regions can be parallelized, and are thus offloaded, the generated code achieved speedups of up to 2.2×. We also show the impact of using SVM, specifically fine‐grained buffers, and the results show that the compiler is able to achieve significant speedups, both over the versions without SVM and with naïve aggressive SVM use, across three CPU/GPU platforms. [ABSTRACT FROM AUTHOR]

Summary: Scheduling is a widely used method in parallel computing, which assigns tasks to compute resources of a parallel environments. In this article, we consider independent parallel tasks to be scheduled onto a heterogeneous execution platform consisting of a set of multicores of different architecture. Each parallel task has an internal potential parallelism which allows a parallel execution on any multicore processors. However, the execution time may differ due a different computation speed of different multicores. In this article, we propose a new search‐based scheduling algorithm Heterogeneous Parallel task scheduling based on A* (called HP*) to solve the problem of scheduling independent parallel tasks onto heterogeneous multicore platforms. Specifically, we propose a heuristic cost function needed for an informed search. Also, three pruning techniques are proposed, which are shown to significantly reduce the search space of HP*. Performance measurements on a heterogeneous platform are performed and the results of HP* are compared to scheduling results of other popular scheduling methods. The performance results with benchmark tasks from the SPLASH‐3 benchmark suite demonstrate the good scheduling results and the improvements achieved by HP*. [ABSTRACT FROM AUTHOR]