Search

This paper studies deep neural networks for solving extremely large linear systems arising from high-dimensional problems. Because of the curse of dimensionality, it is expensive to store both the solution and right-hand side vector in such extremely large linear systems. Our idea is to employ a neural network to characterize the solution with many fewer parameters than the size of the solution under a matrix-free setting. We present an error analysis of the proposed method, indicating that the solution error is bounded by the condition number of the matrix and the neural network approximation error. Several numerical examples from partial differential equations, queueing problems, and probabilistic Boolean networks are presented to demonstrate that the solutions of linear systems can be learned quite accurately. [ABSTRACT FROM AUTHOR]

In this paper, we study deep neural networks (DNNs) for solving high-dimensional evolution equations with oscillatory solutions. Different from deep least-squares methods that deal with time and space variables simultaneously, we propose a deep adaptive basis Galerkin (DABG) method which employs the spectral-Galerkin method for time variable by a tensor-product basis for oscillatory solutions and the deep neural network method for high-dimensional space variables. The proposed method can lead to a linear system of differential equations having unknown DNNs that can be trained via the loss function. We establish a posteriori estimates of the solution error which is bounded by the minimal loss function and the term O(N-m), where N is the number of basis functions and m characterizes the regularity of the equation, and show that if the true solution is a Barron-type function, the error bound converges to zero as M = O(Np) approaches to infinity where M is the width of the used networks and p is a positive constant. Numerical examples, including high-dimensional linear parabolic and hyperbolic equations, and a nonlinear Allen--Cahn equation are presented to demonstrate that the performance of the proposed DABG method is better than that of existing DNNs. [ABSTRACT FROM AUTHOR]

In this paper, we propose a novel method for solving high-dimensional spectral fractional Laplacian equations. Using the Caffarelli--Silvestre extension, the d-dimensional spectral fractional equation is reformulated as a regular partial differential equation of dimension d+1. We transform the extended equation as a minimal Ritz energy functional problem and search for its minimizer in a special class of deep neural networks. Moreover, based on the approximation property of networks, we establish estimates on the error made by the deep Ritz method. Numerical results are reported to demonstrate the effectiveness of the proposed method for solving fractional Laplacian equations up to 10 dimensions. Technically, in this method, we design a special network-based structure to adapt to the singularity and exponential decaying of the true solution. Also, a hybrid integration technique combining the Monte Carlo method and sinc quadrature is developed to compute the loss function with higher accuracy. [ABSTRACT FROM AUTHOR]

In [McDonald, Pestana, and Wathen, SIAM J. Sci. Comput., 40 (2018), pp. A1012--A1033], a block circulant preconditioner is proposed for all-at-once linear systems arising from evolutionary partial differential equations, in which the preconditioned matrix is proven to be diagonalizable and to have identity-plus-low-rank decomposition in the case of the heat equation. In this paper, we generalize the block circulant preconditioner by introducing a small parameter ϵ>0 into the top-right block of the block circulant preconditioner. The implementation of the generalized preconditioner requires the same computational complexity as that of the block circulant one. Theoretically, we prove that (i) the generalization preserves the diagonalizability and the identity-plus-low-rank decomposition; (ii) all eigenvalues of the new preconditioned matrix are clustered at 1 for sufficiently small\epsilon; (iii) GMRES method for the preconditioned system has a linear convergence rate independent of size of the linear system when\epsilon is taken to be smaller than or comparable to square root of time-step size. Numerical results are reported to confirm the efficiency of the proposed preconditioner and to show that the generalization improves the performance of block circulant preconditioner. [ABSTRACT FROM AUTHOR]

In this paper, we study nonnegative tensor data and propose an orthogonal nonnegative Tucker decomposition (ONTD). We discuss some properties of ONTD and develop a convex relaxation algorithm of the augmented Lagrangian function to solve the optimization problem. The convergence of the algorithm is given. We employ ONTD on the image data sets from the real world applications including face recognition, image representation, and hyperspectral unmixing. Numerical results are shown to illustrate the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

Data assimilation is a process used to fuse information from priors, observations of nature, and numerical models, in order to obtain best estimates of the parameters or state of a physical system of interest. Presence of large errors in some observational data, e.g., data collected from a faulty instrument, negatively affect the quality of the overall assimilation results. This work develops a systematic framework for robust data assimilation. The new algorithms continue to produce good estimates of parameters or state in the presence of observation outliers. The approach is based on replacing the traditional L2 norm formulation of data assimilation problems with formulations based on L1 and Huber norms. Numerical experiments using the Lorenz-96 and the shallow water on the sphere models illustrate how the new algorithms outperform traditional data assimilation approaches in the presence of data outliers. [ABSTRACT FROM AUTHOR]

The phase retrieval problem has drawn considerable attention, as many optical detection devices can only measure magnitudes of the Fourier transform of the underlying object (signal or image). This paper addresses the phase retrieval problem from incomplete data, where only partial magnitudes of Fourier transform are obtained. In particular, we consider structured illuminated patterns in holography and find that noninteger values used in designing such patterns often yield better reconstruction than the conventional integer-valued ones. Furthermore, we demonstrate theoretically and numerically that three diffracted sets of (complete) magnitude data are sufficient to recover the object. To compensate for incomplete information, we incorporate a total variation regularization a priori to guarantee that the reconstructed image satisfies some desirable properties. The proposed model can be solved efficiently by an alternative directional multiplier method with provable convergence. Numerical experiments validate the theoretical finding and demonstrate the effectiveness of the proposed method in recovering objects from noisy and incomplete data. [ABSTRACT FROM AUTHOR]

In this paper, we study the linear systems arising from the discretization of timedependent space-fractional diffusion equations. By using a finite difference discretization scheme for the time derivative and a finite volume discretization scheme for the space-fractional derivative, Toeplitz-like linear systems are obtained. We propose using the approximate inverse-circulant preconditioner to deal with such Toeplitz-like matrices, and we show that the spectra of the corresponding preconditioned matrices are clustered around 1. Experimental results on time-dependent and space-fractional diffusion equations are presented to demonstrate that the preconditioned Krylov subspace methods converge very quickly. [ABSTRACT FROM AUTHOR]

Over the past few decades, much attention has been drawn to large-scale incremental data analysis, where researchers are faced with huge amounts of high-dimensional data acquired incrementally. In such a case, conventional algorithms that compute the result from scratch whenever a new sample comes are highly inefficient. To conquer this problem, we propose a new incremental algorithm incremental regularized least squares (IRLS) that incrementally computes the solution to the regularized least squares (RLS) problem with multiple columns on the right-hand side. More specifically, for an RLS problem with c (c > 1) columns on the right-hand side, we update its unique solution by solving an RLS problem with a single column on the right-hand side whenever a new sample arrives, instead of solving an RLS problem with c columns on the right-hand side from scratch. As a direct application of IRLS, we consider the supervised dimensionality reduction of large-scale data and focus on linear discriminant analysis (LDA). We first propose a new batch LDA model that is closely related to the RLS problem, and then apply IRLS to develop a new incremental LDA algorithm. Experimental results on real-world datasets demonstrate the effectiveness and efficiency of our algorithms. [ABSTRACT FROM AUTHOR]

The fractional diffusion equation is discretized by an implicit finite difference scheme with the shifted Grünwald formula, which is unconditionally stable. The coefficient matrix of the discretized linear system is equal to the sum of a scaled identity matrix and two diagonal-times-Toeplitz matrices. Standard circulant preconditioners may not work for such Toeplitz-like linear systems. The main aim of this paper is to propose and develop approximate inverse preconditioners for such Toeplitz-like matrices. An approximate inverse preconditioner is constructed to approximate the inverses of weighted Toeplitz matrices by circulant matrices, and then combine them together row-by-row. Because of Toeplitz structure, both the discretized coefficient matrix and the preconditioner can be implemented very efficiently by using fast Fourier transforms. Theoretically, we show that the spectra of the resulting preconditioned matrices are clustered around one. Thus Krylov subspace methods with the proposed preconditioner converge very fast. Numerical examples are given to demonstrate the effectiveness of the proposed preconditioner and show that its performance is better than the other testing preconditioners. [ABSTRACT FROM AUTHOR]