Search

This paper presents three tensor randomized extended Kaczmarz methods for solving a tensor inconsistent linear system of equations under the t-product between tensors. The randomized extended Kaczmarz, the randomized extended block Kaczmarz, and the randomized extended greedy block Kaczmarz methods in tensor form are proposed to solve inconsistent tensor linear system of equations, respectively. The convergence of each method is proved. Several numerical examples are given to show the efficiency and effectiveness of our methods. [ABSTRACT FROM AUTHOR]

The parameterized block splitting (PBS) is a convergent and efficient iterative method to solve the large complex symmetric linear systems. In this paper, by using PBS iterative technique, the Newton equation is approximately solved, then we establish the modified Newton-PBS iterative method to solve the complex nonlinear systems whose Jacobian matrices are large, sparse, and complex symmetric. Subsequently, the local convergence analysis are explored under appropriate conditions. Ultimately, we apply the new method and several known methods to experimental numerical examples, and experimental results verify the superiority and efficiency of our new method. Especially, in terms of CPU time and iteration steps, our method is obviously better. [ABSTRACT FROM AUTHOR]

A modular string averaging procedure (MSA, for short) for a finite number of operators was first introduced by Reich and Zalas in 2016. The MSA concept provides a flexible algorithmic framework for solving various feasibility problems such as common fixed point and convex feasibility problems. In 2001 Bauschke and Combettes introduced the notion of coherence and applied it to proving weak and strong convergence of many iterative methods. In 2019 Barshad, Reich and Zalas proposed a stronger variant of coherence which provides a more convenient sufficient convergence condition for such methods. In this paper we combine the ideas of both modular string averaging and coherence. Focusing on extending the above MSA procedure to an infinite sequence of operators with admissible controls, we establish strong coherence of its output operators. Various applications of these concepts are presented with respect to weak and strong convergence. They also provide important generalizations of known results, where the weak convergence of sequences of operators generated by the MSA procedure with intermittent controls was considered. [ABSTRACT FROM AUTHOR]

In this paper, we make a theoretical analysis of the convergence rates of Kaczmarz and extended Kaczmarz projection algorithms for some of the most practically used control sequences. We first prove an at least linear convergence rate for the Kaczmarz-Tanabe and its extended version methods (the one in which a complete set of projections using row/column indices is performed in each iteration). Then, we apply the main ideas of this analysis in establishing an at least sublinear, respectively, linear convergence rate for the Kaczmarz algorithm with almost cyclic and the remotest set control strategies, and their extended versions, respectively. These results complete the existing ones related to the random selection procedures. [ABSTRACT FROM AUTHOR]

In this paper, we trace back the genesis of Aitken's Δ2 process and Shanks' sequence transformation. These methods, which are extrapolation methods, are used for accelerating the convergence of sequences of scalars, vectors, matrices, and tensors. They had, and still have, many important applications in numerical analysis and in applied mathematics. They are related to continued fractions and Padé approximants. We go back to the roots of these methods and analyze the original contributions. New and detailed explanations on the building and properties of Shanks' transformation and its kernel are provided. We then review their historical algebraic and algorithmic developments. We also analyze how they were involved in the solution of systems of linear and nonlinear equations, in particular in the methods of Steffensen, Pulay, and Anderson. Testimonies by various actors of the domain are given. The paper can also serve as an introduction to this domain of numerical analysis. [ABSTRACT FROM AUTHOR]

Consider the problem of solving systems of linear algebraic equations A x = b with a real symmetric positive definite matrix A using the conjugate gradient (CG) method. To stop the algorithm at the appropriate moment, it is important to monitor the quality of the approximate solution. One of the most relevant quantities for measuring the quality of the approximate solution is the A -norm of the error. This quantity cannot be easily computed; however, it can be estimated. In this paper we discuss and analyze the behavior of the Gauss-Radau upper bound on the A -norm of the error, based on viewing CG as a procedure for approximating a certain Riemann-Stieltjes integral. This upper bound depends on a prescribed underestimate μ to the smallest eigenvalue of A . We concentrate on explaining a phenomenon observed during computations showing that, in later CG iterations, the upper bound loses its accuracy, and is almost independent of μ . We construct a model problem that is used to demonstrate and study the behavior of the upper bound in dependence of μ , and developed formulas that are helpful in understanding this behavior. We show that the above-mentioned phenomenon is closely related to the convergence of the smallest Ritz value to the smallest eigenvalue of A . It occurs when the smallest Ritz value is a better approximation to the smallest eigenvalue than the prescribed underestimate μ . We also suggest an adaptive strategy for improving the accuracy of the upper bounds in the previous iterations. [ABSTRACT FROM AUTHOR]

When exploring the literature, it can be observed that the operator obtained when applying Newton-like root finding algorithms to the quadratic polynomials z2 − c has the same form regardless of which algorithm has been used. In this paper, we justify why this expression is obtained. This is done by studying the symmetries of the operators obtained after applying Newton-like algorithms to a family of degree d polynomials p(z) = zd − c. Moreover, we provide an iterative procedure to obtain the expression of new Newton-like algorithms. We also carry out a dynamical study of the given generic operator and provide general conclusions of this type of methods. [ABSTRACT FROM AUTHOR]

Recently, a number of efficient iteration methods for the solution of a special class of block 3 × 3 saddle point systems have been proposed by some authors. In order to easily evaluate the strong stability of these numerical algorithms and provide a practical and reliable termination criterion, in this paper, we perform the structured backward error analysis for this type of block 3 × 3 saddle point system and present an explicit and computable formula of the normwise structured backward error. Some numerical experiments are performed to demonstrate that our results can be used to easily test the stability of running algorithms, and the new stopping criterion based on the derived structured backward error is more suitable and efficient than the commonly used residue-based one. [ABSTRACT FROM AUTHOR]

In this paper, first we propose a general modulus-based matrix splitting iteration method for solving horizontal linear complementarity problems. In order to improve the computing efficiency, we further propose a preconditioned general modulus-based matrix splitting iteration method. We establish the convergence theorems when the coefficient matrices are (symmetric) positive definite matrices and H+-matrices, respectively. Numerical results show that the proposed preconditioned general modulus-based matrix splitting iteration method is superior than some existing methods and the general modulus-based matrix splitting iteration method. [ABSTRACT FROM AUTHOR]

In practical computations, the (preconditioned) conjugate gradient (P)CG method is the iterative method of choice for solving systems of linear algebraic equations Ax = b with a real symmetric positive definite matrix A. During the iterations, it is important to monitor the quality of the approximate solution xk so that the process could be stopped whenever xk is accurate enough. One of the most relevant quantities for monitoring the quality of xk is the squared A-norm of the error vector x − xk. This quantity cannot be easily evaluated; however, it can be estimated. Many of the existing estimation techniques are inspired by the view of CG as a procedure for approximating a certain Riemann–Stieltjes integral. The most natural technique is based on the Gauss quadrature approximation and provides a lower bound on the quantity of interest. The bound can be cheaply evaluated using terms that have to be computed anyway in the forthcoming CG iterations. If the squared A-norm of the error vector decreases rapidly, then the lower bound represents a tight estimate. In this paper, we suggest a heuristic strategy aiming to answer the question of how many forthcoming CG iterations are needed to get an estimate with the prescribed accuracy. Numerical experiments demonstrate that the suggested strategy is efficient and robust. [ABSTRACT FROM AUTHOR]

The initial boundary value problem of the tempered fractional diffusion equations is a kind of important equations arising in many application fields. In this paper, the Crank-Nicolson scheme is applied in the discretization of the tempered fractional diffusion equations. We then get the discretized system of linear equations with the coefficient matrix having the structure of the sum of an identity matrix and the product of a diagonal and a symmetric positive-definite Toeplitz matrix. A scaled diagonal and Toeplitz-approximate splitting (SDTAS) preconditioner is developed, and the GMRES method combined with this preconditioner is applied to solve the linear system. The spectral distribution of the preconditioned matrix is analyzed and some theoretical results are given. Numerical results demonstrate that the proposed preconditioner is efficient in accelerating the convergence rate of the GMRES method. [ABSTRACT FROM AUTHOR]

The multi-step matrix splitting iteration (MPIO) for computing PageRank is an efficient iterative method by combining the multi-step power method with the inner-outer iterative method. In this paper, with the aim of accelerating the computation of PageRank problems, a new method is proposed by preconditioning the MPIO method with an adaptive generalized Arnoldi (GArnoldi) method. The new method is called as an adaptive GArnoldi-MPIO method, whose construction and convergence analysis are discussed in detail. Numerical experiments on several PageRank problems are reported to illustrate the effectiveness of our proposed method. [ABSTRACT FROM AUTHOR]

The block version of GMRES (BGMRES) is most advantageous over the single right hand side (RHS) counterpart when the cost of communication is high while the cost of floating point operations is not. This is the particular case on modern graphics processing units (GPUs), while it is generally not the case on traditional central processing units (CPUs). In this paper, experiments on both GPUs and CPUs are shown that compare the performance of BGMRES against GMRES as the number of RHS increases, with a particular forcus on GPU performance. The experiments indicate that there are many cases in which BGMRES is slower than GMRES on CPUs, but faster on GPUs. Furthermore, when varying the number of RHS on the GPU, there is an optimal number of RHS where BGMRES is clearly most advantageous over GMRES. A computational model for the GPU is developed using hardware specific parameters, providing insight towards how the qualitative behavior of BGMRES changes as the number of RHS increase, and this model also helps explain the phenomena observed in the experiments. [ABSTRACT FROM AUTHOR]

In this paper, we study a τ-preconditioner for non-symmetric linear system arising from a steady-state multi-dimensional Riemann-Liouville (RL) fractional diffusion equation. The generalized minimal residual (GMRES) method is applied to solve the preconditioned linear system. Theoretically, we show that the GMRES solver for the preconditioned linear system has a convergence rate independent of discretization stepsizes. To the best of our knowledge, this is the first iterative solver with stepsize-independent convergence rate for the non-symmetric linear system. The proposed τ-preconditioner is diagonalizable by the sine transform matrix, thanks to which the matrix-vector multiplication in each iteration step can be fast implemented by the fast sine transform (FST). Hence, the total operation cost of the proposed solver for the non-symmetric problem is linearithmic. Numerical results are reported to show the efficiency of the proposed preconditioner. [ABSTRACT FROM AUTHOR]

In this paper, by using a modulus-based matrix splitting method as a smoother, a new modulus-based cascadic multigrid method is presented for solving elliptic variational inequality problems. Convergence of the new method is analyzed. Numerical experiments confirm the theoretical analysis and show the efficiency of the proposed method. [ABSTRACT FROM AUTHOR]

Because the expense of estimating the optimal value of the relaxation parameter in the successive over-relaxation (SOR) method is usually prohibitive, the parameter is often adaptively controlled. In this paper, new adaptive SOR methods are presented that are applicable to a variety of symmetric positive definite linear systems and do not require additional matrix-vector products when updating the parameter. To this end, we regard the SOR method as an algorithm for minimising a certain objective function, which yields an interpretation of the relaxation parameter as the step size following a certain change of variables. This interpretation enables us to adaptively control the step size based on some line search techniques, such as the Wolfe conditions. Numerical examples demonstrate the favourable behaviour of the proposed methods. [ABSTRACT FROM AUTHOR]

In this paper, we exploit the Strang splitting technique for solving the multidimensional Allen-Cahn equations with anisotropic spatial fractional Riesz derivatives. Fully discrete numerical methods are proposed using exponential Strang's splitting schemes for the time integration with finite difference discretization in space. It is proved that the proposed methods can preserve the discrete maximum principle unconditionally. Furthermore, the fully discrete methods are theoretically confirmed to be convergent with second-order accuracy in both of time and space. In practical implementation, the proposed algorithms require to compute the matrix exponential associated with only one-dimensional discretized matrices that possess Toeplitz structure. Meanwhile, a fast algorithm is further developed for evaluating the product of the Toeplitz matrix exponential with a vector. Numerical examples are presented to verify the theoretical analysis and demonstrate the efficiency of the proposed methods. [ABSTRACT FROM AUTHOR]

PageRank is a widespread model for analysing the relative relevance of nodes within large graphs arising in several applications. In the current paper, we present a cost-effective Hessenberg-type method built upon the Hessenberg process for the solution of difficult PageRank problems. The new method is very competitive with other popular algorithms in this field, such as Arnoldi-type methods, especially when the damping factor is close to 1 and the dimension of the search subspace is large. The convergence and the complexity of the proposed algorithm are investigated. Numerical experiments are reported to show the efficiency of the new solver for practical PageRank computations. [ABSTRACT FROM AUTHOR]

In the past decade, robust principal component analysis (RPCA) and low-rank matrix completion (LRMC), as two very important optimization problems with the view of recovering original low-rank matrix from sparsely and highly corrupted observations or a subset of its entries, have already been successfully adopted in image denoising, video processing, web search, biological information, etc. This paper proposes an efficient and effective algorithm, named the alternating direction and step size minimization (ADSM) algorithm, which employs the alternating direction minimization idea to solve the general relaxed model that can describe small noise (e.g., Gaussian noise). The coupling of sparse noise and small noise makes low-rank matrix recovery more challenging than that of RPCA. We make use of Taylor expansion, singular value decomposition and shrinkage operator as the alternating direction minimization method to deduce iterative direction matrices. A continuous technology is incorporated into ADSM to accelerate convergence. Similarly, the Taylor expansion and step size minimization (TESM) algorithm for LRMC is designed by the above way, but the alternating direction minimization idea needs to be ruled out since there is not a sparse matrix in it. Theoretically, it is proved that the two algorithms globally converge to their respective optimal points based on some conditions. The numerical results are reported, illustrating that ADSM and TESM are quite efficient and effective for recovering large-scale low-rank matrix problems at many cases. [ABSTRACT FROM AUTHOR]

This paper describes a new MATLAB software package of iterative regularization methods and test problems for large-scale linear inverse problems. The software package, called IR TOOLS, serves two related purposes: we provide implementations of a range of iterative solvers, including several recently proposed methods that are not available elsewhere, and we provide a set of large-scale test problems in the form of discretizations of 2D linear inverse problems. The solvers include iterative regularization methods where the regularization is due to the semi-convergence of the iterations, Tikhonov-type formulations where the regularization is explicitly formulated in the form of a regularization term, and methods that can impose bound constraints on the computed solutions. All the iterative methods are implemented in a very flexible fashion that allows the problem's coefficient matrix to be available as a (sparse) matrix, a function handle, or an object. The most basic call to all of the various iterative methods requires only this matrix and the right hand side vector; if the method uses any special stopping criteria, regularization parameters, etc., then default values are set automatically by the code. Moreover, through the use of an optional input structure, the user can also have full control of any of the algorithm parameters. The test problems represent realistic large-scale problems found in image reconstruction and several other applications. Numerical examples illustrate the various algorithms and test problems available in this package. [ABSTRACT FROM AUTHOR]

In this paper, the block generalized product-type bi-conjugate gradient (GPBi-CG) method for solving large, sparse nonsymmetric linear systems of equations with multiple right-hand sides is proposed. The new algorithm is based on the block BiCG process. We analyze the convergence behavior of this method and present a bound for the residual norm of block GPBi-CG according to the residual norm of Bl-GMRES method. In addition, we prove that convergence is guaranteed when A is positive real. The numerical experiments show the efficiency of the new method and confirm the theoretical results. [ABSTRACT FROM AUTHOR]

Discrete ill-posed inverse problems arise in various areas of science and engineering. The presence of noise in the data often makes it difficult to compute an accurate approximate solution. To reduce the sensitivity of the computed solution to the noise, one replaces the original problem by a nearby well-posed minimization problem, whose solution is less sensitive to the noise in the data than the solution of the original problem. This replacement is known as regularization. We consider the situation when the minimization problem consists of a fidelity term, that is defined in terms of a p-norm, and a regularization term, that is defined in terms of a q-norm. We allow 0 < p,q ≤ 2. The relative importance of the fidelity and regularization terms is determined by a regularization parameter. This paper develops an automatic strategy for determining the regularization parameter for these minimization problems. The proposed approach is based on a new application of generalized cross validation. Computed examples illustrate the performance of the method proposed. [ABSTRACT FROM AUTHOR]

Low-rank Krylov methods are one of the few options available in the literature to address the numerical solution of large-scale general linear matrix equations. These routines amount to well-known Krylov schemes that have been equipped with a couple of low-rank truncations to maintain a feasible storage demand in the overall solution procedure. However, such truncations may affect the convergence properties of the adopted Krylov method. In this paper we show how the truncation steps have to be performed in order to maintain the convergence of the Krylov routine. Several numerical experiments validate our theoretical findings. [ABSTRACT FROM AUTHOR]

Solving large-scale l1 minimization problem is very important and has attracted much attentions in recent years. In this paper, we present an efficient method for solving this problem. To this end, we first reformulate this problem as a nonnegative constrained minimization problem. Then we propose a modulus-based iterative method to solve the nonnegative constrained minimization problem. Convergence analysis and the choice of near optimal parameter are presented. Experimental results are given to illustrate feasibility and effectiveness of our proposed method. [ABSTRACT FROM AUTHOR]

In this paper, we study the spectral distributions of the saddle point matrices arising from the discretization and linearization of the Navier-Stokes equations, where the (1,1) block is nonsymmetric positive definite. In this paper, we derive the lower and upper bounds of the real and imaginary parts of all the eigenvalues of the saddle point matrices. We then propose a new class of block triangle preconditioners for solving the saddle point problems, and analyze the spectral properties of the preconditioned systems. Some numerical experiments with the preconditioned restarted generalized minimal residual method are reported to demonstrate the effectiveness and feasibility of these block triangle preconditioners. [ABSTRACT FROM AUTHOR]

A large number of free boundary problems can be formulated as linear-complementarity problems. In this paper, we propose an inexact alternating direction method of multipliers for solving linear complementarity problem arising from free boundary problems by using the special structure of these problems. The convergence of our proposed method is proved. Numerical results show that the proposed method is feasible and effective, and it is significantly faster than modified alternating direction implicit algorithm and many other methods, especially when dimension of the problem being solved is large. [ABSTRACT FROM AUTHOR]

In this paper, by extrapolating the modified Hermitian and skew-Hermitian splitting (MHSS) iteration method with a complex relaxation parameter, a complex-extrapolated MHSS (CMHSS) iteration method is present for solving a class of complex singular symmetric of linear equations. We study the semi-convergence properties of the CMHSS iteration method and the extent of the optimal iterative parameters. Furthermore, the convergence conditions also hold for solving nonsingular complex systems. Numerical experiments are given to verify the effectiveness of the CMHSS iteration method for solving both singular and nonsingular complex symmetric systems. [ABSTRACT FROM AUTHOR]