Search

The efficient utilization of mixed-precision numerical linear algebra algorithms can offer attractive acceleration to scientific computing applications. Especially with the hardware integration of low-precision special-function units designed for machine learning applications, the traditional numerical algorithms community urgently needs to reconsider the floating point formats used in the distinct operations to efficiently leverage the available compute power. In this work, we provide a comprehensive survey of mixed-precision numerical linear algebra routines, including the underlying concepts, theoretical background, and experimental results for both dense and sparse linear algebra problems. [ABSTRACT FROM AUTHOR]

The first is the ongoing emergence of multicore and hybrid microprocessor designs, ushering in a new era of computing in which system designers must accept energy usage as a first-order constraint, and application designers must be able to exploit parallelism and data locality to an unprecedented degree. The research areas of cluster, cloud, and data analytics computing, which today provide fundamental infrastructure for all areas of advanced computational science, are being radically transformed by the convergence of at least two unprecedented trends. [Extracted from the article]

Recently some scientific computing users have discovered that they can replace 64-bit with 32-bit operations for carefully selected portions of the computation, and still retain acceptable accuracy in the final results. In addition, developers of some emerging applications such as machine learning have discovered that they can achieve acceptable results with only 16-bit precision in certain portions of the code. At the other end of the precision spectrum, some users have explored using 128-bit arithmetic in some particularly demanding applications, while others have done computations using much higher precision—hundreds or even thousands of digits. Such work has underscored the need to develop new mathematical and software frameworks to support a dynamically variable level of precision, and, more generally, to rethink what "reproducibility" means in a variable precision environment. This article summarizes some of the work being done in this arena, and lists research problems that need to be solved. [ABSTRACT FROM AUTHOR]

Scientific discovery increasingly relies on computation through simulations, analytics, and machine and deep learning. Of these, simulations on high-performance computing (HPC) platforms have been the cornerstone of scientific computing for more than two decades. However, the development of simulation software has, in general, occurred through accretion, with a few exceptions. With an increase in scientific understanding, models have become more complex, rendering an accretion mode untenable to the point where software productivity and sustainability have become active concerns in scientific computing. In this survey paper, we examine a modest set of HPC scientific simulation applications that are already using cutting-edge HPC platforms. Several have been in existence for a decade or more. Our objective in this survey is twofold: first, to understand the landscape of scientific computing on HPC platforms in order to distill the currently scattered knowledge about software practices that have helped both developer and software productivity, and second, to understand the kind of tools and methodologies that need attention for continued productivity. [ABSTRACT FROM AUTHOR]

In January 2019, the US Department of Energy, Office of Science program in Advanced Scientific Computing Research, convened a workshop to identify priority research directions (PRDs) for in situ data management (ISDM). A fundamental finding of this workshop is that the methodologies used to manage data among a variety of tasks in situ can be used to facilitate scientific discovery from many different data sources—simulation, experiment, and sensors, for example—and that being able to do so at numerous computing scales will benefit real-time decision-making, design optimization, and data-driven scientific discovery. This article describes six PRDs identified by the workshop, which highlight the components and capabilities needed for ISDM to be successful for a wide variety of applications—making ISDM capabilities more pervasive, controllable, composable, and transparent, with a focus on greater coordination with the software stack and a diversity of fundamentally new data algorithms. [ABSTRACT FROM AUTHOR]

We present a domain-decomposition-based preconditioner for the solution of partial differential equations (PDEs) that is resilient to both soft and hard faults. The algorithm reformulates the PDE as a sampling problem, followed by a solution update through data manipulation that is resilient to both soft and hard faults. This reformulation allows us to recast the problem as a set of independent tasks, and exploit data locality to reduce global communication. We discuss two different parallel implementations: (a) a single program multiple data (SPMD) version based on a one-to-one mapping between subdomain and MPI processes responsible for both state and computation; and (b) an asynchronous server–client implementation where all state information is held by the servers and clients are designed solely as computational units. We present a scalability comparison of both implementations under nominal conditions, showing efficiency within ~80% for up to 12,000 cores. We present a resilience analysis under different fault scenarios based on the server–client implementation. This framework provides resiliency to hard faults such that if a client crashes, it stops asking for work, and the servers simply distribute the work among all of the other clients alive. Erroneous subdomain solves (e.g. due to soft faults) appear as corrupted data, which is either rejected if that causes a task to fail, or is seamlessly filtered out during the regression stage through a suitable noise model. Three different types of faults are modeled: hard faults modeling nodes (or clients) crashing; soft faults occurring during the communication of the tasks between server and clients; and soft faults occurring during task execution. We demonstrate the resiliency of the approach for a 2D elliptic PDE, and explore the effect of the faults at various failure rates. [ABSTRACT FROM AUTHOR]

Over the past four years, the Big Data and Exascale Computing (BDEC) project organized a series of five international workshops that aimed to explore the ways in which the new forms of data-centric discovery introduced by the ongoing revolution in high-end data analysis (HDA) might be integrated with the established, simulation-centric paradigm of the high-performance computing (HPC) community. Based on those meetings, we argue that the rapid proliferation of digital data generators, the unprecedented growth in the volume and diversity of the data they generate, and the intense evolution of the methods for analyzing and using that data are radically reshaping the landscape of scientific computing. The most critical problems involve the logistics of wide-area, multistage workflows that will move back and forth across the computing continuum, between the multitude of distributed sensors, instruments and other devices at the networks edge, and the centralized resources of commercial clouds and HPC centers. We suggest that the prospects for the future integration of technological infrastructures and research ecosystems need to be considered at three different levels. First, we discuss the convergence of research applications and workflows that establish a research paradigm that combines both HPC and HDA, where ongoing progress is already motivating efforts at the other two levels. Second, we offer an account of some of the problems involved with creating a converged infrastructure for peripheral environments, that is, a shared infrastructure that can be deployed throughout the network in a scalable manner to meet the highly diverse requirements for processing, communication, and buffering/storage of massive data workflows of many different scientific domains. Third, we focus on some opportunities for software ecosystem convergence in big, logically centralized facilities that execute large-scale simulations and models and/or perform large-scale data analytics. We close by offering some conclusions and recommendations for future investment and policy review. [ABSTRACT FROM AUTHOR]

Floating-point computation is ubiquitous in high-performance scientific computing, but rounding error can compromise the results of extended calculations, especially at large scales. In this paper, we present new techniques that use binary instrumentation and modification to do fine-grained floating-point precision analysis, simulating any level of precision less than or equal to the precision of the original program. These techniques have an average of 40–70% lower overhead and provide more fine-grained insights into a program’s sensitivity than previous mixed-precision analyses. We also present a novel histogram-based visualization of a program’s floating-point precision sensitivity, as well as an incremental search technique that allows developers to incrementally trade off analysis time for detail, including the ability to restart analyses from where they left off. We present results from several case studies and experiments that show the efficacy of these techniques. Using our tool and its novel visualization, application developers can more quickly determine for specific data sets whether their application could be run using fewer double precision variables, saving both time and memory space. [ABSTRACT FROM AUTHOR]

This paper presents dispel4py, a new Python framework for describing abstract stream-based workflows for distributed data-intensive applications. These combine the familiarity of Python programming with the scalability of workflows. Data streaming is used to gain performance, rapid prototyping and applicability to live observations. dispel4py enables scientists to focus on their scientific goals, avoiding distracting details and retaining flexibility over the computing infrastructure they use. The implementation, therefore, has to map dispel4py abstract workflows optimally onto target platforms chosen dynamically. We present four dispel4py mappings: Apache Storm, message-passing interface (MPI), multi-threading and sequential, showing two major benefits: a) smooth transitions from local development on a laptop to scalable execution for production work, and b) scalable enactment on significantly different distributed computing infrastructures. Three application domains are reported and measurements on multiple infrastructures show the optimisations achieved; they have provided demanding real applications and helped us develop effective training. The dispel4py.org is an open-source project to which we invite participation. The effective mapping of dispel4py onto multiple target infrastructures demonstrates exploitation of data-intensive and high-performance computing (HPC) architectures and consistent scalability. [ABSTRACT FROM AUTHOR]

The use of the Python programming language for scientific computing has been gaining momentum in the last years. The fact that it is compact and readable and its complete set of scientific libraries are two important characteristics that favour its adoption. Nevertheless, Python still lacks a solution for easily parallelizing generic scripts on distributed infrastructures, since the current alternatives mostly require the use of APIs for message passing or are restricted to embarrassingly parallel computations. In that sense, this paper presents PyCOMPSs, a framework that facilitates the development of parallel computational workflows in Python. In this approach, the user programs her script in a sequential fashion and decorates the functions to be run as asynchronous parallel tasks. A runtime system is in charge of exploiting the inherent concurrency of the script, detecting the data dependencies between tasks and spawning them to the available resources. Furthermore, we show how this programming model can be built on top of a Big Data storage architecture, where the data stored in the backend is abstracted and accessed from the application in the form of persistent objects. [ABSTRACT FROM AUTHOR]

Soft errors in scientific computing applications are becoming inevitable with the ever-increasing system scale and execution time, and new technologies that feature increased transistor density and lower voltage. Soft errors can be mainly classified into two categories: bit-flipping error (e.g. 1 becomes −1) in random access memory; and computation error (e.g. 1+1 = 3) in floating point units. Traditionally, bit-flipping error is handled by the Error Correcting Code (ECC) technique, and computation error is dealt with the Triple Modular Redundancy (TMR) method. Note that, ECC cannot handle computation error, while TMR cannot deal with bit-flipping error and is not efficient on handling computation error. To uniformly and efficiently handle both computation and bit-flipping errors in matrix operations, the Algorithm-Based Fault Tolerance (ABFT) method is developed. This paper focuses on the detection of soft errors in the LU Decomposition with Partial Pivoting (LUPP) algorithm, which is widely used in scientific computing applications. First, this paper notes that existing ABFT methods are not adequate to detect soft errors in LUPP in terms of time or space. Then we propose a new ABFT algorithm which can detect soft errors in LUPP both flexible in time and comprehensive in space. Flexible in time means that soft errors can be detected flexibly during the execution instead of only at the end of LUPP, while comprehensive in space indicates that all of the elements in data matrices (L and U) will be covered for detecting soft errors. To show the feasibility and efficiency of the proposed algorithm, this paper has incorporated it into the implementation of LUPP in the widely used benchmark High Performance Linpack (HPL). Experiment results verify the feasibility of this algorithm: for soft errors injected at various timings and to different elements in LUPP, this algorithm has detected most of the injected errors, which have covered all of the errors that cannot pass the residual check of HPL. Both theoretical overhead analysis and experiments demonstrate that this ABFT algorithm is also very efficient at detecting soft errors in LUPP. [ABSTRACT FROM AUTHOR]

High-performance scientific applications are usually built from software modules written in multiple programming languages. This raises the issue of language interoperability which involves making calls between languages, converting basic types, and bridging disparate programming models. Babel provides a feature-rich, extensible, high-performance solution to the language interoperability problem currently supporting C, C++, FORTRAN 77, Fortran 90/95, Fortran 2003/2008, Python, and Java. Babel supports object-oriented programming features and interface semantics with runtime enforcement. In addition to in-process language interoperability, Babel includes remote method invocation to support hybrid parallel and distributed computing paradigms. [ABSTRACT FROM AUTHOR]

This paper presents a scalable high-performance software library to be used for graph analysis and data mining. Large combinatorial graphs appear in many applications of high-performance computing, including computational biology, informatics, analytics, web search, dynamical systems, and sparse matrix methods. Graph computations are difficult to parallelize using traditional approaches due to their irregular nature and low operational intensity. Many graph computations, however, contain sufficient coarse-grained parallelism for thousands of processors, which can be uncovered by using the right primitives. We describe the parallel Combinatorial BLAS, which consists of a small but powerful set of linear algebra primitives specifically targeting graph and data mining applications. We provide an extensible library interface and some guiding principles for future development. The library is evaluated using two important graph algorithms, in terms of both performance and ease-of-use. The scalability and raw performance of the example applications, using the Combinatorial BLAS, are unprecedented on distributed memory clusters. [ABSTRACT FROM PUBLISHER]

The article focuses on the need to have a global mechanism for planning and coordinating technologies necessary for working on the high performance computing (HPC) systems, across the open-source community. International Exascale Software Project (IESP) is stated to work across the community to identify critical lacunae in the overall software environment and spot opportunities for beneficial integration into the HPC. Also discussed in the article are the benefits of the project.

The article presents information on the implementation of a parallel symmetric solver involving real symmetric positive definite or complex symmetric non-Hermitian dense systems. The behavior that one can expect from the theoretical model has been confirmed by experiments up to 32 processors. The obtained solution complies with the experimental operational constraints and one can solve problems of size corresponding to the target applications. The parallel distributed symmetric solver is efficient to solve geodesy least-squares problems by a normal equations method and also to solve complex linear systems encountered in electromagnetism. From a numerical point of view, some additional work can be developed. Furthermore, other approaches to design parallel linear solvers might consist in using high-level parallel routines in PLAPACK or PBLAS/ ScaLAPACK. The article further proposes to extend the factorization technique that one developed in real arithmetic and thereby to save 50% of the storage compared with standard software.

Focuses on Virtual and Dynamic Hierarchical Architecture, an e-science grid architecture. Features of the architecture; Idea behind e-science; Characteristics of e-science.

This special issue gathers selected papers of the Workshop on Clusters, Clouds and Data for Scientific Computing that was held at La Maison des Contes, Dareize-France, on October 3-6, 2016. [ABSTRACT FROM AUTHOR]