Search

In this paper, we revisit the Krylov multisplitting algorithm presented in Huang and O'Leary (Linear Algebra Appl 194:9-29, ) which uses a sequential method to minimize the Krylov iterations computed by a multisplitting algorithm. Our new algorithm is based on a parallel multisplitting algorithm with few blocks of large size using a parallel GMRES method inside each block and on a parallel Krylov minimization to improve the convergence. Some large-scale experiments with a 3D Poisson problem are presented with up to 8,192 cores. They show the obtained improvements compared to a classical GMRES both in terms of number of iterations and in terms of execution times. [ABSTRACT FROM AUTHOR]

In a wide variety of applications from different scientific and engineering fields, the solution of complex and/or nonsymmetric linear systems of equations is required. To solve this kind of linear systems the BiConjugate Gradient method (BCG) is especially relevant. Nevertheless, BCG has a enormous computational cost. GPU computing is useful for accelerating this kind of algorithms but it is necessary to develop suitable implementations to optimally exploit the GPU architecture. In this paper, we show how BCG can be effectively accelerated when all operations are computed on a GPU. So, BCG has been implemented with two alternative routines of the Sparse Matrix Vector product (SpMV): the CUSPARSE library and the ELLR-T routine. Although our interest is focused on complex matrices, our implementation has been evaluated on a GPU for two sets of test matrices: complex and real, in single and double precision data. Experimental results show that BCG based on ELLR-T routine achieves the best performance, particularly for the set of complex test matrices. Consequently, this method can be useful as a tool to efficiently solve large linear system of equations (complex and/or nonsymmetric) involved in a broad range of applications. [ABSTRACT FROM AUTHOR]

In loops, some arrays are referenced with induction variables. To parallelize such kind of loops, those induction variables should be substituted. Thus, those array references that were substituted are formulated as nonlinear expressions. The goal of data alignment is to intelligently map the computations and data onto a set of virtual processors which are organized as a Cartesian grid (or a template in HPF terms), and to provide data locality for parallelizing compilers so that data access communication costs can be minimized. Most data alignment methods are mainly devised to align the referenced arrays using linear subscripts or quadratic subscripts with n loop index variables, and the methods are well developed. Seldom work, however, is researched on the nonlinear expressions of index variables. This paper proposes a new communication-free data alignment technique to align the referenced arrays using exponential subscripts with n loop index variables or other complex nonlinear expressions. The experimental results using SPEC95FP Benchmarks point out that the techniques proposed in the paper can improve the execution time of the subroutines in these benchmarks. [ABSTRACT FROM AUTHOR]

This paper compares various contemporary multicore-based microprocessor architectures from different vendors with different memory interconnects regarding performance, speedup, and parallel efficiency. Sparse matrix decomposition is used as a benchmark application. The example matrix used in the experiments comes from an electrical engineering application, where numerical simulation of physical processes plays an important role in the design of industrial products. Within this context, thread-to-core pinning and cache optimization are two important aspects which are investigated in more detail. [ABSTRACT FROM AUTHOR]

Traditionally, loop nests are fused only when the data dependences in the loop nests are not violated. This paper presents a new loop fusion algorithm that is capable of fusing loop nests in the presence of fusion-preventing anti-dependences. All the violated anti-dependences are removed by automatic array copying. As a case study, this aggressive loop fusion strategy is applied to a Jacobi solver. The performance of iterative methods is typically limited by the speed of the memory system. Fusing the two loop nests in the Jacobi solver into one reduces data cache misses, and consequently, improves the performance results of both sequential and parallel versions of the Jacobi program, as validated by our experimental results on an HP AlphaServer SC45 supercomputer. [ABSTRACT FROM AUTHOR]

Approaches for dealing with scheduling and load-balancing in PC-based cluster systems are famous and well known. In such environments, Self-Scheduling Schemes are suitable for parallel loops with independent iterations. However, while schemes such as FSS, GSS, and TSS fit most computer systems, they cannot provide good load-balancing. Chao-Tung Yang and Shun-Chi Chang proposed a parallel loop scheduling scheme for heterogeneous PC cluster systems in Yang and Chang [13]. Though the proposed scheme allows users to choose parameters before execution initialization, weaknesses in it motivated us to develop further improvements. For instance, using fixed and monotonous parameters can easily lead to invalid scheduling due to use of previously input information. Thus, in this paper we propose a new scheme that fits most widely available computer systems and allows the scheduling parameter to be adjusted dynamically in order to provide higher overall performance. [ABSTRACT FROM AUTHOR]

Loop tiling and unrolling are two important program transformations to exploit locality and expose instruction level parallelism, respectively. However, these transformations are not independent and each can adversely affect the goal of the other. Furthermore, the best combination will vary dramatically from one processor to the next. In this paper, we therefore address the problem of how to select tile sizes and unroll factors simultaneously. We approach this problem in an architecturally adaptive manner by means of iterative compilation, where we generate many versions of a program and decide upon the best by actually executing them and measuring their execution time. We evaluate several iterative strategies based on genetic algorithms, random sampling and simulated annealing. We compare the levels of optimization obtained by iterative compilation to several well-known static techniques and show that we outperform each of them on a range of benchmarks across a variety of architectures. Finally, we show how to quantitatively trade-off the number of profiles needed and the level of optimization that can be reached. In this way, we can reach high levels of optimization within 50 iterations. [ABSTRACT FROM AUTHOR]

This paper proposes a flow analysis approach to provide accurate flow facts such as number of loop iterations and infeasible paths for WCET analysis. To achieve this, a novel approach to model program flows of given loops as compositional condition of controls that can model the program flow of the sequence of basic blocks on an execution path through a single loop iteration is proposed. The proposed approach builds distinct symbolic expressions each representing a path condition as conjunctions of branch conditions along the path. Then, the path conditions are formed via substituting all constituent variables of the branch conditions with symbolic expressions computing the value of these variables. The final symbolic expression will contain variables denoting theirs value at the loop entry point. Considering the change in the value of the loop variables of the path condition, the feasible values of these variables are computed and consequently the number of the loop iterations along the iteration path is determined. Applying a SMT solver to the symbolic expressions representing each path condition, all the infeasible paths along the loop body are detected. The results of applying our flow analysis approach to a number of programs addressed in the Mälardalen benchmark suite reveal the capability and efficiency of our proposed approach. [ABSTRACT FROM AUTHOR]

Programmers usually implement iterative methods that solve partial differential equations by expressing them using a sequence of basic kernels from libraries optimized for the graphics processing unit (GPU). The global runtime of the resulting combination is often penalized by the smallest and most inefficient vector operations. To improve the GPU exploitation, we identify and analyze the potential kernels to be fused according to the data dependence, data type and size, and GPU resources. This paper provides an extensive analysis of the impact of fusing vector operations [level 1 of Basic Linear Algebra Subprograms (BLAS)] on the performance of the GPU. The experimental evaluation shows that this optimization provides noticeable improvement especially for kernels with lower memory requirements and on more modern GPUs. It is worth noting that the fused BLAS operations can be very useful to help programmers efficiently code iterative methods to solve large linear systems of equations for the GPU. Iterative methods such as biconjugate gradient method (BCG) are one of the examples that can benefit from this optimization strategy. Indeed, kernel fusion of vector routines makes the most efficient GPU implementation of BCG run between $$1.09\times $$ and $$1.27\times $$ faster on three GPUs of different characteristics. [ABSTRACT FROM AUTHOR]

This paper develops an algorithm for iterative learning control on the basis of the quasi-Newton method for nonlinear systems. The new quasi-Newton iterative learning control scheme using the rank-one update to derive the recurrent formula has numerous benefits, which include the approximate treatment for the inverse of the system's Jacobian matrix. The rank-one update-based ILC also has the advantage of extension for convergence domain and hence guaranteeing the choice of initial value. The algorithm is expressed as a very general norm optimization problem in a Banach space and, in principle, can be used for both continuous and discrete time systems. Furthermore, a detailed convergence analysis is given, and it guarantees theoretically that the proposed algorithm converges at a superlinear rate. Initial conditions which the algorithm requires are also established. The simulations illustrate the theoretical results. [ABSTRACT FROM AUTHOR]

Speculative multithreading (SpMT) is a thread-level automatic parallelization technique, which partitions sequential programs into multithreads to be executed in parallel. This paper presents different thread partitioning strategies for nonloops and loops. For nonloops, we propose a cost estimation based on combined run-time effects of various speculation factors to predict the resulting performance of candidate threads to guide the thread partitioning. For loops, we parallelize all the profitable loops that can potentially offer additional performance benefits by multilevel spawning in loop bodies, loop iterations, and inner loops. Then we select a proper thread boundary located in the front of loop branch instruction to reduce invalid spawning threads that waste core resources. Experimental results show that the proposed approach can obtain a significant increase in speedup and Olden benchmarks reach a performance improvement of 6.62 % on average. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new parallel sparse iterative method (PPSIA) for computing the stationary distribution of large-scale Markov chains. The PPSIA method is based on Markov chain state isolation and aggregation techniques. The parallel method conserves as much as possible the benefits of aggregation, and Gauss-Seidel effects contained in the sequential algorithm (SIA) using a pipelined technique. Both SIA and PPSIA exploit sparse matrix representation in order to solve large-scale Markov chains. Some Markov chains have been tested to compare the performance of SIA, PPSIA algorithms with other techniques such as the power method, and the generalized minimal residual GMRES method. In all the tested models, PPSIA outperforms the other methods and shows a super-linear speed-up. [ABSTRACT FROM AUTHOR]

Heterogeneous computing (HC) is the coordinated use of different types of machines, and networks to process a diverse workload in a manner that will maximize the combined performance and/or cost effectiveness of the system. Heuristics for allocating resources in an HC system are based on some optimization criterion. A common optimization criterion is to minimize the completion time of the machine that finishes last (makespan). In this study, we consider an iterative approach that repeatedly runs a mapping heuristic to minimize the makespan of the considered machines and tasks. For each successive iteration, the makespan machine of the previous iteration and the tasks assigned to it are removed from the set of considered machines and tasks. This study focuses on understanding the different mathematical characteristics of resource allocation heuristics that cause them to behave differently when combined with this iterative approach. This paper has three main contributions. The first contribution is the study of an iterative technique used in conjunction with resource allocation heuristics. The second contribution is the definition and mathematical characterization of 'iteration invariant' heuristics. The third contribution is to determine the characteristics of a heuristic that will cause the mapping to change across iterations. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new scalable parallel block aggregated iterative method (PBA) for computing the stationary distribution of a Markov chain. The PBA technique is based on aggregation of groups (block) of Markov chain states. Scalability of the PBA algorithm depends on varying the number of blocks and their size, assigned to each processor. PBA solves the aggregated blocks very efficiently using a modified LU factorization technique. Some Markov chains have been tested to compare the performance of PBA algorithm with other block techniques such as parallel block Jacobi and block Gauss-Seidel. In all the tested models PBA outperforms the other parallel block methods. [ABSTRACT FROM AUTHOR]

Solving large sparse linear systems is essential in numerous scientific domains. Several algorithms, based on direct or iterative methods, have been developed for parallel architectures. On distributed grids consisting of processors located in distant geographical sites, their performance may be unsatisfactory because they suffer from too many synchronizations and communications. The GREMLINS code has been developed for solving large sparse linear systems on distributed grids. It implements the multisplitting method that consists in splitting the original linear system into several subsystems that can be solved independently. In this paper, the performance of the GREMLINS code obtained with several libraries for solving the linear subsystems is analyzed. Its performance is also compared with that of the widely used PETSc library that enables one to develop portable parallel applications. Numerical experiments have been carried out both on local clusters and on distributed grids. [ABSTRACT FROM AUTHOR]

This paper proposes a novel approach for the parallel execution of tiled Iteration Spaces onto a cluster of SMP PC nodes. Each SMP node has multiple CPUs and a single memory mapped PCI-SCI Network Interface Card. We apply a hyperplane-based grouping transformation to the tiled space, so as to group together independent neighboring tiles and assign them to the same SMP node. In this way, intranode (intragroup) communication is annihilated. Groups are atomically executed inside each node. Nodes exchange data between successive group computations. We schedule groups much more efficiently by exploiting the inherent overlapping between communication and computation phases among successive atomic group executions. The applied non-blocking schedule resembles a pipelined datapath, where group computation phases are overlapped with communication ones, instead of being interleaved with them. Our experimental results illustrate that the proposed method outperforms previous approaches involving blocking communication or conventional grouping schemes. [ABSTRACT FROM AUTHOR]

Pricing of derivatives is one of the central problems in computational finance. Since the theory of derivative pricing is highly mathematical, numerical techniques such as lattice approach, finite-difference and finite-element among others have been employed. Recently fast Fourier transform (FFT) has been employed for derivative pricing in sequential computers. In this paper, we report development of a multithreaded FFT pricing algorithm and performance evaluation on a multithreaded platform. The focus of this study is on the effectiveness of using a parallel computer for financial problems and performance evaluation of a multithreaded algorithm for finance applications such as derivative pricing. In general, a parallel algorithm for FFT, with blocked data distribution of N elements on P processors, involves communication for log P iterations and terminates after log N iterations. The first (log N - log P) iterations therefore, require no communication and a sequential algorithm can be used in each processor. We call this a local algorithm. At the end of the (log N - log P) iterations, the processors switch to a multithreaded algorithm where sending and receiving of threads is through message passing. We call this a remote algorithm. The algorithm has been implemented on the EARTH (Efficient Architecture for Running THreads) multithreaded platform. Our results indicate that the FFT multithreaded algorithm for option pricing is very efficient giving a relative speedup of 50% on 64 processors. This study reveals an important commercial application for High Performance Computing. [ABSTRACT FROM AUTHOR]

This work studies the problem of balancing the workload of iterative algorithms on heterogeneous multiprocessors. An approach, called ADITHE, is proposed and evaluated. Its main features are: (1) using a homogeneous distribution of the workload on the heterogeneous system, the speed of every node is estimated during the first iterations of the algorithm; (2) according to the speed of every node, a new workload distribution is carried out; (3) the remaining iterations of the algorithm are executed. The result of this workload redistribution is that the execution times for every iteration at every node are similar and, consequently, the penalties due to synchronization between nodes at every iteration are mostly eliminated. This approach is appropriate for iterative algorithms with similar workload at every iteration, and with a relevant number of iterations. The high portability of ADITHE is guaranteed because the estimation of speed of nodes is included in the execution of the parallel algorithm. There is a wide variety of iterative algorithms related to science and engineering which can take advantage of ADITHE. An example of this kind of algorithms (morphological processing of hyperspectral images) is considered in this work to evaluate its performance when ADITHE is applied. The analysis of the results shows that ADITHE significantly improves the performance of parallel iterative algorithms on heterogeneous platforms. [ABSTRACT FROM AUTHOR]

Heterogeneous computers require a well-distributed workload to operate efficiently. When possible, this load balancing procedure should redistribute the workload with minimal knowledge of the system architecture, to reduce overhead. We propose a generic dynamic load balancing technique for iterative problems, independent from the resource to optimize. Proof of this generalization is given through formalization of the designed technique. A heuristic algorithm is defined based upon this formalization, with a structure that facilitates different objective functions. As a result, swapping the objective function can be done with relatively low effort. This heuristic is implemented to minimize energy consumption in an application. We use this application to solve three different dynamic programming problems with multiple GPUs. The implementation is described and then compared against two different workloads, the homogeneous distribution and another dynamic load balancing technique. Our experimentation shows good results in minimizing the overall energy consumption with low overhead. [ABSTRACT FROM AUTHOR]

Dynamic programming (DP) approaches, in particular value iteration, is often seen as a method to derive optimal policies in inventory management. The challenge in this approach is to deal with an increasing state space when handling realistic problems. As a large part of world food production is thrown out due to its perishable character, a motivation exists to have a good look at order policies in retail. Recently, investigation has been introduced to consider substitution of one product by another, when one is out of stock. Taking this tendency into account in a policy requires an increasing state space. Therefore, we investigate the potential of using GPU platforms in order to derive optimal policies when the number of products taken into account simultaneously is increasing. First results show the potential of the GPU approach to accelerate computation in value iteration for DP. [ABSTRACT FROM AUTHOR]

The Apriori algorithm is one of the most well-known and widely accepted methods for the association rule mining. In Apriori, it uses a prefix tree to represent k-itemsets, generates k-itemset candidates based on the frequent (k-1)-itemsets, and determines the frequent k-itemsets by traversing the prefix tree iteratively based on the transaction records. When k is small, the execution of Apriori is very efficient. However, the execution of Apriori could be very slow when k becomes large because of the deeper recursion depth to determine the frequent k-itemsets. From the perspective of graph computing, the transaction records can be converted to a graph G(V,E), where V is the set of vertices of G that represents the transaction records and E is the set of edges of G that represents the relations among transaction records. Each k-itemset in the transaction records will have a corresponding connected component in G. The number of vertices in the corresponding connected component is the support of the k-itemset. Since the time to find the corresponding connected component of a k-itemset in G is constant for any k, the graph computing method will be very efficient if the number of k-itemsets is relatively small. Based on Apriori and graph computing techniques, a hybrid method, called Apriori and Graph Computing (ANG), is proposed to compute the frequent itemsets. Initially, ANG uses Apriori to compute the frequent k-itemsets and then switches to the graph computing method when k becomes large (where the number of k-itemset candidates is relatively small). The experimental results show that ANG outperforms both Apriori and the graph computing method for all test cases. [ABSTRACT FROM AUTHOR]

MFiX, a general-purpose Fortran-based suite, simulates the complex flow in fluidized bed applications via BiCGStab and GMRES methods along with plane relaxation preconditioners. Trilinos, an object-oriented framework, contains various first- and second-generation Krylov subspace solvers and preconditioners. We developed a framework to integrate MFiX with Trilinos as MFiX does not possess advanced linear methods. The framework allows MFiX to access advanced linear solvers and preconditioners in Trilinos. The integrated solver is called MFiX-Trilinos, here after. In the present work, we study the performance of variants of GMRES and CGS methods in MFiX-Trilinos and BiCGStab and GMRES solvers in MFiX for a 3D gas-solid fluidized bed problem. Two right preconditioners employed along with various solvers in MFiX-Trilinos are Jacobi and smoothed aggregation. The flow from MFiX-Trilinos is validated against the same from MFiX for BiCGStab and GMRES methods. And, the effect of the preconditioning on the iterative solvers in MFiX-Trilinos is also analyzed. In addition, the effect of left and right smoothed aggregation preconditioning on the solvers is studied. The performance of the first- and second-generation solver stacks in MFiX-Trilinos is studied as well for two different problem sizes. [ABSTRACT FROM AUTHOR]

This work is focused on the application of the new AXC format in iterative algorithms on the Intel Xeon Phi coprocessor to solve linear systems by accelerating the sparse matrix-vector (SpMV) product. These algorithms are the Conjugate Gradient (CG) and the Biconjugate Gradient Stabilised (BiCGS) methods used to solve symmetric and non-symmetric linear systems. Two highly efficient formats were selected to compare the AXC format: the Compressed Sparse Row (CSR) format and the Sliced ELLPACK-C-α (SELL-C-α) format. The evaluation process consists of two phases. The first phase is focused on the performance comparison of the SpMV kernels alone. The second phase compares the performance of the solvers with the different kernels integrated within them. As this work is oriented to explore the full capabilities of the Intel Xeon Phi architecture, the Intel’s intrinsic instructions set were used to code the kernels for the AXC and SELL-C-α formats and compared with the Intel’s MKL optimal implementation of the CSR format. Numerical results demonstrate that the AXC format achieves the higher average performance 15.9 GFLOPS against the 5.5 GFLOPS for the SELL-C-α format and 8.4 GFLOPs for the MKL CSR format, and it also have the lowest variability when performing the SpMV product alone. Results also show that the AXC format achieves up to 6.5×, 1.5× and 1.9× greater performance over the MKL CSR format, its closest competitor, for the SpMV product, the CG and the BiCGS algorithms, respectively. [ABSTRACT FROM AUTHOR]

A new parallel Self Mesh-Adaptive N-body method based on approximate inverses is proposed. The scheme is a three-dimensional Cartesian-based method that solves the Poisson equation directly in physical space, using modified multipole expansion formulas for the boundary conditions. Moreover, adaptive-mesh techniques are utilized to form a class of separate smaller n-body problems that can be solved in parallel and increase the total resolution of the system. The solution method is based on multigrid method in conjunction with the symmetric factored approximate sparse inverse matrix as smoother. The design of the parallel Self Mesh-Adaptive method along with discussion on implementation issues for shared memory computer systems is presented. The new parallel method is evaluated through a series of benchmark simulations using N-body models of isolated galaxies or galaxies interacting with dwarf companions. Furthermore, numerical results on the performance and the speedups of the scheme are presented. [ABSTRACT FROM AUTHOR]

This article presents a new communication library developed to ease the implementation of both asynchronous and synchronous iterative methods. A mathematical and algorithmic framework about fixed-point methods is described to introduce this class of parallel iterative algorithms, although this library can be used for a larger class of parallel algorithms. After an overview of the main features, we describe detailed implementation aspects arising from the asynchronous context. While the library is mainly based on top of Message Passing Interface library, it has been designed to be easily extended to other types of communication middleware. Finally, some numerical experiments validate this new library, used for implementing both a classical parallel scheme and a sub-structuring approach of the Jacobi iterative method. [ABSTRACT FROM AUTHOR]

This article presents an algorithm that performs a decentralized detection of the global convergence of parallel asynchronous iterative applications. This algorithm is fault tolerant. It runs a decentralized saving procedure which enables this algorithm, after a node’s crash, to replace the dead node by a new one which will continue the computing task from the last check point. Combined with the advantages of the asynchronous iteration model, this method allows us to compute very large scale problems using highly volatile parallel architectures like Peer-to-Peer and distributed clusters architectures. We also present the implementation of this algorithm in the JaceP2P platform which is dedicated to designing and executing parallel asynchronous iterative applications in volatile environments. Numerous experiments show the robustness and the efficiency of our algorithm. [ABSTRACT FROM AUTHOR]