Search

In recent years, there has been a renewed interest in preconditioning for multilevel Toeplitz systems, a research field that has been extensively explored over the past several decades. This work introduces novel preconditioning strategies using multilevel $\tau$ matrices for both symmetric and nonsymmetric multilevel Toeplitz systems. Our proposals constitute a general framework, as they are constructed solely based on the generating function of the multilevel Toeplitz coefficient matrix, when it can be defined. We begin with nonsymmetric systems, where we employ a symmetrization technique by permuting the coefficient matrix to produce a real symmetric multilevel Hankel structure. We propose a multilevel $\tau$ preconditioner tailored to the symmetrized system and prove that the eigenvalues of the preconditioned matrix sequence cluster at $\pm 1$, leading to rapid convergence when using the preconditioned minimal residual method. The high effectiveness of this approach is demonstrated through its application in solving space fractional diffusion equations. Next, for symmetric systems we introduce another multilevel $\tau$ preconditioner and show that the preconditioned conjugate gradient method can achieve an optimal convergence rate, namely a rate that is independent of the matrix size, when employed for a class of ill-conditioned multilevel Toeplitz systems. Numerical examples are provided to critically assess the effectiveness of our proposed preconditioners compared to several leading existing preconditioned solvers, highlighting their superior performance.

Preconditioning for multilevel Toeplitz systems has long been a focal point of research in numerical linear algebra. In this work, we develop a novel preconditioning method for a class of nonsymmetric multilevel Toeplitz systems, which includes the all-at-once systems that arise from evolutionary partial differential equations. These systems have recently garnered considerable attention in the literature. To further illustrate our proposed preconditioning strategy, we specifically consider the application of solving a wide range of non-local, time-dependent partial differential equations in a parallel-in-time manner. For these equations, we propose a symmetric positive definite multilevel Tau preconditioner that is not only efficient to implement but can also be adapted as an optimal preconditioner. In this context, the proposed preconditioner is optimal in the sense that it enables mesh-independent convergence when using the preconditioned generalized minimal residual method. Numerical examples are provided to critically analyze the results and underscore the effectiveness of our preconditioning strategy.

In this study, the $\theta$-method is used for discretizing a class of evolutionary partial differential equations. Then, we transform the resultant all-at-once linear system and introduce a novel one-sided preconditioner, which can be fast implemented in a parallel-in-time way. By introducing an auxiliary two-sided preconditioned system, we provide theoretical insights into the relationship between the residuals of the generalized minimal residual (GMRES) method when applied to both one-sided and two-sided preconditioned systems. Moreover, we show that the condition number of the two-sided preconditioned matrix is uniformly bounded by a constant that is independent of the matrix size, which in turn implies that the convergence behavior of the GMRES method for the one-sided preconditioned system is guaranteed. Numerical experiments confirm the efficiency and robustness of the proposed preconditioning approach.

The complex-shifted Laplacian systems arising in a wide range of applications. In this work, we propose an absolute-value based preconditioner for solving the complex-shifted Laplacian system. In our approach, the complex-shifted Laplacian system is equivalently rewritten as a $2\times 2$ block real linear system. With the Toeplitz structure of uniform-grid discretization of the constant-coefficient Laplacian operator, the absolute value of the block real matrix is fast invertible by means of fast sine transforms. For more general coefficient function, we then average the coefficient function and take the absolute value of the averaged matrix as our preconditioner. With assumptions on the complex shift, we theoretically prove that the eigenvalues of the preconditioned matrix in absolute value are upper and lower bounded by constants independent of matrix size, indicating a matrix-size independent linear convergence rate of MINRES solver. Interestingly, numerical results show that the proposed preconditioner is still efficient even if the assumptions on the complex shift are not met. The fast invertibility of the proposed preconditioner and the robust convergence rate of the preconditioned MINRES solver lead to a linearithmic (nearly optimal) complexity of the proposed solver. The proposed preconditioner is compared with several state-of-the-art preconditioners via several numerical examples to demonstrate the efficiency of the proposed preconditioner.

In this work, we develop a novel multilevel Tau matrix-based preconditioned method for a class of non-symmetric multilevel Toeplitz systems. This method not only accounts for but also improves upon an ideal preconditioner pioneered by [J. Pestana. Preconditioners for symmetrized Toeplitz and multilevel Toeplitz matrices. SIAM J. Matrix Anal. Appl., 40(3):870-887, 2019]. The ideal preconditioning approach was primarily examined numerically in that study, and an effective implementation was not included. To address these issues, we first rigorously show in this study that this ideal preconditioner can indeed achieve optimal convergence when employing the MINRES method, with a convergence rate is that independent of the mesh size. Then, building on this preconditioner, we develop a practical and optimal preconditioned MINRES method. To further illustrate its applicability and develop a fast implementation strategy, we consider solving Riemann-Liouville fractional diffusion equations as an application. Specifically, following standard discretization on the equation, the resultant linear system is a non-symmetric multilevel Toeplitz system, affirming the applicability of our preconditioning method. Through a simple symmetrization strategy, we transform the original linear system into a symmetric multilevel Hankel system. Subsequently, we propose a symmetric positive definite multilevel Tau preconditioner for the symmetrized system, which can be efficiently implemented using discrete sine transforms. Theoretically, we demonstrate that mesh-independent convergence can be achieved. In particular, we prove that the eigenvalues of the preconditioned matrix are bounded within disjoint intervals containing $\pm 1$, without any outliers. Numerical examples are provided to critically discuss the results, showcase the spectral distribution, and support the efficacy of our strategy.

In this work, we propose a class of novel preconditioned Krylov subspace methods for solving an optimal control problem of parabolic equations. Namely, we develop a family of block $\omega$-circulant based preconditioners for the all-at-once linear system arising from the concerned optimal control problem, where both first order and second order time discretization methods are considered. The proposed preconditioners can be efficiently diagonalized by fast Fourier transforms in a parallel-in-time fashion, and their effectiveness is theoretically shown in the sense that the eigenvalues of the preconditioned matrix are clustered around $\pm 1$, which leads to rapid convergence when the minimal residual method is used. When the generalized minimal residual method is deployed, the efficacy of the proposed preconditioners are justified in the way that the singular values of the preconditioned matrices are proven clustered around unity. Numerical results are provided to demonstrate the effectiveness of our proposed solvers.

A novel fourth-order finite difference formula coupling the Crank-Nicolson explicit linearized method is proposed to solve Riesz space fractional nonlinear reaction-diffusion equations in two dimensions. Theoretically, under the Lipschitz assumption on the nonlinear term, the proposed high-order scheme is proved to be unconditionally stable and convergent in the discrete $L_2$-norm. Moreover, a $\tau$-matrix based preconditioner is developed to speed up the convergence of the conjugate gradient method with an optimal convergence rate (a convergence rate independent of mesh sizes) for solving the symmetric discrete linear system. Theoretical analysis shows that the spectra of the preconditioned matrices are uniformly bounded in the open interval $(3/8,2)$. To the best of our knowledge, this is the first attempt to develop a preconditioned iterative solver with a mesh-independent convergence rate for the linearized high-order scheme. Numerical examples are given to validate the accuracy of the scheme and the effectiveness of the proposed preconditioned solver.

{In [X. L. Lin, M. K. Ng, and Y. Zhi. {\it J. Comput. Phys.}, 434 (2021), pp. 110221] and [Y. L. Zhao, J. Wu, X. M. Gu, and H. Li. {\it Comput. Math. Appl.}, 148(2023), pp. 200--210]}, two-sided preconditioning techniques are proposed for non-local evolutionary equations, which possesses (i) mesh-size independent theoretical bound of condition number of the two-sided preconditioned matrix; (ii) small and stable iteration numbers in numerical tests. In this paper, we modify the two-sided preconditioning by multiplying the left-sided and the right-sided preconditioners together as a single-sided preconditioner. Such a single-sided preconditioner essentially derives from approximating the spatial matrix with a fast diagonalizable matrix and keeping the temporal matrix unchanged. Clearly, the matrix-vector multiplication of the single-sided preconditioning is faster to compute than that of the two-sided one, since the single-sided preconditioned matrix has a simpler structure. More importantly, we show theoretically that the single-sided preconditioned generalized minimal residual (GMRES) method has a convergence rate no worse than the two-sided preconditioned one. As a result, the one-sided preconditioned GMRES solver requires less computational time than the two-sided preconditioned GMRES solver in total. Numerical results are reported to show the efficiency of the proposed single-sided preconditioning technique.

In this work, we propose a novel preconditioned Krylov subspace method for solving an optimal control problem of wave equations, after explicitly identifying the asymptotic spectral distribution of the involved sequence of linear coefficient matrices from the optimal control problem. Namely, we first show that the all-at-once system stemming from the wave control problem is associated to a structured coefficient matrix-sequence possessing an eigenvalue distribution. Then, based on such a spectral distribution of which the symbol is explicitly identified, we develop an ideal preconditioner and two parallel-in-time preconditioners for the saddle point system composed of two block Toeplitz matrices. For the ideal preconditioner, we show that the eigenvalues of the preconditioned matrix-sequence all belong to the set $\left(-\frac{3}{2},-\frac{1}{2}\right)\bigcup \left(\frac{1}{2},\frac{3}{2}\right)$ well separated from zero, leading to mesh-independent convergence when the minimal residual method is employed. The proposed {parallel-in-time} preconditioners can be implemented efficiently using fast Fourier transforms or discrete sine transforms, and their effectiveness is theoretically shown in the sense that the eigenvalues of the preconditioned matrix-sequences are clustered around $\pm 1$, which leads to rapid convergence. When these parallel-in-time preconditioners are not fast diagonalizable, we further propose modified versions which can be efficiently inverted. Several numerical examples are reported to verify our derived localization and spectral distribution result and to support the effectiveness of our proposed preconditioners and the related advantages with respect to the relevant literature.

In this study, a novel preconditioner based on the absolute-value block $\alpha$-circulant matrix approximation is developed, specifically designed for nonsymmetric dense block lower triangular Toeplitz (BLTT) systems that emerge from the numerical discretization of evolutionary equations. Our preconditioner is constructed by taking an absolute-value of a block $\alpha$-circulant matrix approximation to the BLTT matrix. To apply our preconditioner, the original BLTT linear system is converted into a symmetric form by applying a time-reversing permutation transformation. Then, with our preconditioner, the preconditioned minimal residual method (MINRES) solver is employed to solve the symmetrized linear system. With properly chosen $\alpha$, the eigenvalues of the preconditioned matrix are proven to be clustered around $\pm1$ without any significant outliers. With the clustered spectrum, we show that the preconditioned MINRES solver for the preconditioned system has a convergence rate independent of system size. To the best of our knowledge, this is the first preconditioned MINRES method with size-independent convergence rate for the dense BLTT system. The efficacy of the proposed preconditioner is corroborated by our numerical experiments, which reveal that it attains optimal convergence.

In this work, we propose an absolute value block $\alpha$-circulant preconditioner for the minimal residual (MINRES) method to solve an all-at-once system arising from the discretization of wave equations. Motivated by the absolute value block circulant preconditioner proposed in [E. McDonald, J. Pestana, and A. Wathen. SIAM J. Sci. Comput., 40(2):A1012-A1033, 2018], we propose an absolute value version of the block $\alpha$-circulant preconditioner. Since the original block $\alpha$-circulant preconditioner is non-Hermitian in general, it cannot be directly used as a preconditioner for MINRES. Our proposed preconditioner is the first Hermitian positive definite variant of the block $\alpha$-circulant preconditioner for the concerned wave equations, which fills the gap between block $\alpha$-circulant preconditioning and the field of preconditioned MINRES solver. The matrix-vector multiplication of the preconditioner can be fast implemented via fast Fourier transforms. Theoretically, we show that for a properly chosen $\alpha$ the MINRES solver with the proposed preconditioner achieves a linear convergence rate independent of the matrix size. To the best of our knowledge, this is the first attempt to generalize the original absolute value block circulant preconditioner in the aspects of both theory and performance the concerned problem. Numerical experiments are given to support the effectiveness of our preconditioner, showing that the expected optimal convergence can be achieved.

In this work, we propose a simple yet generic preconditioned Krylov subspace method for a large class of nonsymmetric block Toeplitz all-at-once systems arising from discretizing evolutionary partial differential equations. Namely, our main result is to propose two novel symmetric positive definite preconditioners, which can be efficiently diagonalized by the discrete sine transform matrix. More specifically, our approach is to first permute the original linear system to obtain a symmetric one, and subsequently develop desired preconditioners based on the spectral symbol of the modified matrix. Then, we show that the eigenvalues of the preconditioned matrix sequences are clustered around $\pm 1$, which entails rapid convergence when the minimal residual method is devised. Alternatively, when the conjugate gradient method on the normal equations is used, we show that our preconditioner is effective in the sense that the eigenvalues of the preconditioned matrix sequence are clustered around unity. An extension of our proposed preconditioned method is given for high-order backward difference time discretization schemes, which can be applied on a wide range of time-dependent equations. Numerical examples are given, also in the variable-coefficient setting, to demonstrate the effectiveness of our proposed preconditioners, which consistently outperforms an existing block circulant preconditioner discussed in the relevant literature.

We establish in this work approximation results of deep neural networks for smooth functions measured in Sobolev norms, motivated by recent development of numerical solvers for partial differential equations using deep neural networks. {Our approximation results are nonasymptotic in the sense that the error bounds are explicitly characterized in terms of both the width and depth of the networks simultaneously with all involved constants explicitly determined.} Namely, for $f\in C^s([0,1]^d)$, we show that deep ReLU networks of width $\mathcal{O}(N\log{N})$ and of depth $\mathcal{O}(L\log{L})$ can achieve a nonasymptotic approximation rate of $\mathcal{O}(N^{-2(s-1)/d}L^{-2(s-1)/d})$ with respect to the $\mathcal{W}^{1,p}([0,1]^d)$ norm for $p\in[1,\infty)$. If either the ReLU function or its square is applied as activation functions to construct deep neural networks of width $\mathcal{O}(N\log{N})$ and of depth $\mathcal{O}(L\log{L})$ to approximate $f\in C^s([0,1]^d)$, the approximation rate is $\mathcal{O}(N^{-2(s-n)/d}L^{-2(s-n)/d})$ with respect to the $\mathcal{W}^{n,p}([0,1]^d)$ norm for $p\in[1,\infty)$.

Given a Lebesgue integrable function $f$ over $[0,2\pi]$, we consider the sequence of matrices $\{Y_nT_n[f]\}_n$, where $T_n[f]$ is the $n$-by-$n$ Toeplitz matrix generated by $f$ and $Y_n$ is the flip permutation matrix, also called the anti-identity matrix. Because of the unitary character of $Y_n$, the singular values of $T_n[f]$ and $Y_n T_n[f]$ coincide. However, the eigenvalues are affected substantially by the action of the matrix $Y_n$. Under the assumption that the Fourier coefficients are real, we prove that $\{Y_nT_n[f]\}_n$ is distributed in the eigenvalue sense as \[ \phi_g(\theta)=\left\{ \begin{array}{cc} g(\theta), & \theta\in [0,2\pi], -g(-\theta), & \theta\in [-2\pi,0), \end{array} \right.\, \] with $g(\theta)=|f(\theta)|$. We also consider the preconditioning introduced by Pestana and Wathen and, by using the same arguments, we prove that the preconditioned sequence is distributed in the eigenvalue sense as $\phi_1$, under the mild assumption that $f$ is sparsely vanishing. We emphasize that the mathematical tools introduced in this setting have a general character and in fact can be potentially used in different contexts. A number of numerical experiments are provided and critically discussed.

The singular value and spectral distribution of Toeplitz matrix sequences with Lebesgue integrable generating functions is well studied. Early results were provided in the classical Szeg{\H{o}} theorem and the Avram-Parter theorem, in which the singular value symbol coincides with the generating function. More general versions of the theorem were later proved by Zamarashkin and Tyrtyshnikov, and Tilli. Considering (real) nonsymmetric Toeplitz matrix sequences, we first symmetrize them via a simple permutation matrix and then we show that the singular value and spectral distribution of the symmetrized matrix sequence can be obtained analytically, by using the notion of approximating class of sequences. In particular, under the assumption that the symbol is sparsely vanishing, we show that roughly half of the eigenvalues of the symmetrized Toeplitz matrix (i.e. a Hankel matrix) are negative/positive for sufficiently large dimension, i.e. the matrix sequence is symmetric (asymptotically) indefinite.

Circulant preconditioners for functions of matrices have been recently of interest. In particular, several authors proposed the use of the optimal circulant preconditioners as well as the superoptimal circulant preconditioners in this context and numerically illustrated that such preconditioners are effective for certain functions of Toeplitz matrices. Motivated by their results, we propose in this work the absolute value superoptimal circulant preconditioners and provide several theorems that analytically show the effectiveness of such circulant preconditioners for systems defined by functions of Toeplitz matrices. Namely, we show that the eigenvalues of the preconditioned matrices are clustered around $\pm 1$ and rapid convergence of Krylov subspace methods can therefore be expected. Moreover, we show that our results can be extended to functions of block Toeplitz matrices with Toeplitz blocks provided that the optimal block circulant matrices with circulant blocks are used as preconditioners. Numerical examples are given to support our theoretical results.

We propose several circulant preconditioners for systems defined by some functions $g$ of Toeplitz matrices $A_n$. In this paper we are interested in solving $g(A_n)\mathbf{x}=\mathbf{b}$ by the preconditioned conjugate method or the preconditioned minimal residual method, namely in the cases when $g(z)$ are the functions $e^{z}$, $\sin{z}$ and $\cos{z}$. Numerical results are given to show the effectiveness of the proposed preconditioners.