Search

Shifted CholeskyQR3 is designed to address the QR factorization of ill-conditioned matrices. This algorithm introduces a shift parameter $s$ to prevent failure during the initial Cholesky factorization step. The choice of this shift parameter $s$ is critical for the algorithm's effectiveness. Our goal is to identify a smaller $s$ compared to the traditional selection involving $\|X\|_2$. In this research, we propose a new norm $\|X\|_g$, which is based on the column properties of $X$, to obtain a reduced shift parameter $s$ for the Shifted CholeskyQR3 algorithm. We provide rigorous proofs of orthogonality and residuals for the improved algorithm with our proposed $s$. Numerical experiments confirm the enhanced numerical stability of orthogonality and residuals with the reduced $s$. Furthermore, we compare CPU times with other algorithms to assess performance improvements.

This study explores the application of large language models (LLMs) with callable tools in energy and power engineering domain, focusing on gas path analysis of gas turbines. We developed a dual-agent tool-calling process to integrate expert knowledge, predefined tools, and LLM reasoning. We evaluated various LLMs, including LLama3, Qwen1.5 and GPT. Smaller models struggled with tool usage and parameter extraction, while larger models demonstrated favorable capabilities. All models faced challenges with complex, multi-component problems. Based on the test results, we infer that LLMs with nearly 100 billion parameters could meet professional scenario requirements with fine-tuning and advanced prompt design. Continued development are likely to enhance their accuracy and effectiveness, paving the way for more robust AI-driven solutions.

CholeskyQR is a simple and fast QR decomposition via Cholesky decomposition, while it has been considered highly sensitive to the condition number. In this paper, we provide a randomized preconditioner framework for CholeskyQR algorithm. Under this framework, two methods (randomized LU-CholeskyQR and randomized QR-CholeskyQR) are proposed and discussed. We prove the proposed preconditioners can effectively reduce the condition number, which is also demonstrated by numerical tests. Abundant numerical tests indicate our methods are more stable and faster than all the existing algorithms and have good scalability., Comment: 23 pages, 11 figures, 6 tables

In scientific machine learning, regression networks have been recently applied to approximate solution maps (e.g., potential-ground state map of Schr\"odinger equation). In this paper, we aim to reduce the generalization error without spending more time in generating training samples. However, to reduce the generalization error, the regression network needs to be fit on a large number of training samples (e.g., a collection of potential-ground state pairs). The training samples can be produced by running numerical solvers, which takes much time in many applications. In this paper, we aim to reduce the generalization error without spending more time in generating training samples. Inspired by few-shot learning techniques, we develop the Multi-Level Fine-Tuning algorithm by introducing levels of training: we first train the regression network on samples generated at the coarsest grid and then successively fine-tune the network on samples generated at finer grids. Within the same amount of time, numerical solvers generate more samples on coarse grids than on fine grids. We demonstrate a significant reduction of generalization error in numerical experiments on challenging problems with oscillations, discontinuities, or rough coefficients. Further analysis can be conducted in the Neural Tangent Kernel regime and we provide practical estimators to the generalization error. The number of training samples at different levels can be optimized for the smallest estimated generalization error under the constraint of budget for training data. The optimized distribution of budget over levels provides practical guidance with theoretical insight as in the celebrated Multi-Level Monte Carlo algorithm.

This paper proposes a multiscale method for solving the numerical solution of mean field games which accelerates the convergence and addresses the problem of determining the initial guess. Starting from an approximate solution at the coarsest level, the method constructs approximations on successively finer grids via alternating sweeping, which not only allows for the use of classical time marching numerical schemes but also enables applications to both local and nonlocal problems. At each level, numerical relaxation is used to stabilize the iterative process. A second-order discretization scheme is derived for higher-order convergence. Numerical examples are provided to demonstrate the efficiency of the proposed method in both local and nonlocal, 1-dimensional and 2-dimensional cases., Comment: 20 pages, 24 figures; v2: final version accepted at Journal of Computational Physics

We make a brief historical review to the moment model reduction to the kinetic equations, particularly the Grad's moment method for Boltzmann equation. The focus is on the hyperbolicity of the reduced model, which is essential to the existence of its classical solution as a Cauchy problem. The theory of the framework we developed in last years is then introduced, which may preserve the hyperbolic nature of the kinetic equations with high universality. Some lastest progress on the comparison between models with/without hyperbolicity is presented to validate the hyperbolic moment models for rarefied gases.

This paper proposes a neural network approach for solving two classical problems in the two-dimensional inverse wave scattering: far field pattern problem and seismic imaging. The mathematical problem of inverse wave scattering is to recover the scatterer field of a medium based on the boundary measurement of the scattered wave from the medium, which is high-dimensional and nonlinear. For the far field pattern problem under the circular experimental setup, a perturbative analysis shows that the forward map can be approximated by a vectorized convolution operator in the angular direction. Motivated by this and filtered back-projection, we propose an effective neural network architecture for the inverse map using the recently introduced BCR-Net along with the standard convolution layers. Analogously for the seismic imaging problem, we propose a similar neural network architecture under the rectangular domain setup with a depth-dependent background velocity. Numerical results demonstrate the efficiency of the proposed neural networks., Comment: 17 pages, 11 figures. arXiv admin note: substantial text overlap with arXiv:1911.11636

This paper introduces a neural network approach for solving two-dimensional traveltime tomography (TT) problems based on the eikonal equation. The mathematical problem of TT is to recover the slowness field of a medium based on the boundary measurement of the traveltimes of waves going through the medium. This inverse map is high-dimensional and nonlinear. For the circular tomography geometry, a perturbative analysis shows that the forward map can be approximated by a vectorized convolution operator in the angular direction. Motivated by this and filtered back-projection, we propose an effective neural network architecture for the inverse map using the recently proposed BCR-Net, with weights learned from training datasets. Numerical results demonstrate the efficiency of the proposed neural networks., Comment: 17 pages, 7 figures. arXiv admin note: text overlap with arXiv:1910.04756

In this paper, we introduce a new reduced basis methodology for accelerating the computation of large parameterized systems of high-fidelity integral equations. Core to our methodology is the use of coarse-proxy models (i.e., lower resolution variants of the underlying high-fidelity equations) to identify important samples in the parameter space from which a high quality reduced basis is then constructed. Unlike the more traditional POD or greedy methods for reduced basis construction, our methodology has the benefit of being both easy to implement and embarrassingly parallel. We apply our methodology to the under-served area of integral equations, where the density of the underlying integral operators has traditionally made reduced basis methods difficult to apply. To handle this difficulty, we introduce an operator interpolation technique, based on random sub-sampling, that is aimed specifically at integral operators. To demonstrate the effectiveness of our techniques, we present two numerical case studies, based on the Radiative Transport Equation and a boundary integral formation of the Laplace Equation respectively, where our methodology provides a significant improvement in performance over the underlying high-fidelity models for a wide range of error tolerances. Moreover, we demonstrate that for these problems, as the coarse-proxy selection threshold is made more aggressive, the approximation error of our method decreases at an approximately linear rate., Comment: 24 pages, 8 figures, 2 tables

Linear models for the radiative transfer equation have been well developed, while nonlinear models are seldom investigated even for slab geometry due to some essential difficulties. We have proposed a moment model in MPN for slab geometry which combines the ideas of the classical PN and MN model. Though the model is far from perfect, it was demonstrated to be quite efficient in numerically approximating the solution of the radiative transfer equation, that we are motivated to further improve this model. Consequently we propose in this paper a new model following the chartmap in MPN with some significant theoretic progresses. The new model is derived with global hyperbolicity, and meanwhile some necessary physical properties are preserved. We give a complete analysis to the characteristic structure and propose a numerical scheme for the new model. Numerical examples are presented to demonstrate the numerical performance of the new model.

This paper presents a neural network approach for solving two-dimensional optical tomography (OT) problems based on the radiative transfer equation. The mathematical problem of OT is to recover the optical properties of an object based on the albedo operator that is accessible from boundary measurements. Both the forward map from the optical properties to the albedo operator and the inverse map are high-dimensional and nonlinear. For the circular tomography geometry, a perturbative analysis shows that the forward map can be approximated by a vectorized convolution operator in the angular direction. Motivated by this, we propose effective neural network architectures for the forward and inverse maps based on convolution layers, with weights learned from training datasets. Numerical results demonstrate the efficiency of the proposed neural networks., Comment: 16 pages, 8 figures

This paper introduces a meta-learning approach for parameterized pseudo-differential operators with deep neural networks. With the help of the nonstandard wavelet form, the pseudo-differential operators can be approximated in a compressed form with a collection of vectors. The nonlinear map from the parameter to this collection of vectors and the wavelet transform are learned together from a small number of matrix-vector multiplications of the pseudo-differential operator. Numerical results for Green's functions of elliptic partial differential equations and the radiative transfer equations demonstrate the efficiency and accuracy of the proposed approach., Comment: 21 pages, 9 figures

This paper introduces a new approach for solving electrical impedance tomography (EIT) problems using deep neural networks. The mathematical problem of EIT is to invert the electrical conductivity from the Dirichlet-to-Neumann (DtN) map. Both the forward map from the electrical conductivity to the DtN map and the inverse map are high-dimensional and nonlinear. Motivated by the linear perturbative analysis of the forward map and based on a numerically low-rank property, we propose compact neural network architectures for the forward and inverse maps for both 2D and 3D problems. Numerical results demonstrate the efficiency of the proposed neural networks., Comment: 22 pages, 17 figures

This paper is concerned with the approximation of the radiative transfer equation for a grey medium in the slab geometry by the moment method. We develop a novel moment model inspired by the classical $P_N$ model and $M_N$ model. The new model takes the ansatz of the $M_1$ model as the weight function and follows the primary idea of the $P_N$ model to approximate the specific intensity by expanding it around the weight function in terms of orthogonal polynomials. The weight function uses the information of the first two moments, which brings the new model the capability to approximate an anisotropic distribution. Mathematical properties of the moment model are investigated, and particularly the hyperbolicity and the characteristic structure of the Riemann problem of the model with three moments are studied in detail. Some numerical simulations demonstrate its numerical efficiency and show its superior in comparison to the $P_N$ model.

This paper proposes a novel neural network architecture inspired by the nonstandard form proposed by Beylkin, Coifman, and Rokhlin in [Communications on Pure and Applied Mathematics, 44(2), 141-183]. The nonstandard form is a highly effective wavelet-based compression scheme for linear integral operators. In this work, we first represent the matrix-vector product algorithm of the nonstandard form as a linear neural network where every scale of the multiresolution computation is carried out by a locally connected linear sub-network. In order to address nonlinear problems, we propose an extension, called BCR-Net, by replacing each linear sub-network with a deeper and more powerful nonlinear one. Numerical results demonstrate the efficiency of the new architecture by approximating nonlinear maps that arise in homogenization theory and stochastic computation., Comment: 17 pages and 9 figures

We develop a spectral method for the spatially homogeneous Boltzmann equation using Burnett polynomials in the basis functions. Using the sparsity of the coefficients in the expansion of the collision term, the computational cost is reduced by one order of magnitude for general collision kernels and by two orders of magnitude for Maxwell molecules. The proposed method can couple seamlessly with the BGK-type modelling techniques to make future applications affordable. The implementation of the algorithm is discussed in detail, including a numerical scheme to compute all the coefficients accurately, and the design of the data structure to achieve high cache hit ratio. Numerical examples are provided to demonstrate the accuracy and efficiency of our method, Comment: 22 pages, 15 picutres

In recent years, deep learning has led to impressive results in many fields. In this paper, we introduce a multi-scale artificial neural network for high-dimensional non-linear maps based on the idea of hierarchical nested bases in the fast multipole method and the $\mathcal{H}^2$-matrices. This approach allows us to efficiently approximate discretized nonlinear maps arising from partial differential equations or integral equations. It also naturally extends our recent work based on the generalization of hierarchical matrices [Fan et al. arXiv:1807.01883] but with a reduced number of parameters. In particular, the number of parameters of the neural network grows linearly with the dimension of the parameter space of the discretized PDE. We demonstrate the properties of the architecture by approximating the solution maps of non-linear Schr{\"o}dinger equation, the radiative transfer equation, and the Kohn-Sham map., Comment: 21 figures, 25 pages. arXiv admin note: text overlap with arXiv:1807.01883

In this work we introduce a new multiscale artificial neural network based on the structure of $\mathcal{H}$-matrices. This network generalizes the latter to the nonlinear case by introducing a local deep neural network at each spatial scale. Numerical results indicate that the network is able to efficiently approximate discrete nonlinear maps obtained from discretized nonlinear partial differential equations, such as those arising from nonlinear Schr\"odinger equations and the Kohn-Sham density functional theory., Comment: 26 pages, 11 figures

We investigate integral formulations and fast algorithms for the steady-state radiative transfer equation with isotropic and anisotropic scattering. When the scattering term is a smooth convolution on the unit sphere, a model reduction step in the angular domain using the Fourier transformation in 2D and the spherical harmonic transformation in 3D significantly reduces the number of degrees of freedoms. The resulting Fourier coefficients or spherical harmonic coefficients satisfy a Fredholm integral equation of the second kind. We study the uniqueness of the equation and proved an a priori estimate. For a homogeneous medium, the integral equation can be solved efficiently using the FFT and iterative methods. For an inhomogeneous medium, the recursive skeletonization factorization method is applied instead. Numerical simulations demonstrate the efficiency of the proposed algorithms in both homogeneous and inhomogeneous cases and for both transport and diffusion regimes., Comment: 24 pages, 3 figures

In this paper, we investigate the effect of the filter for the hyperbolic moment equations(HME) [15] of the Vlasov-Poisson equations and propose a novel quasi time-consistent filter to suppress the numerical recurrence effect. By taking properties of HME into consideration, the filter preserves a lot of physical properties of HME, including Galilean invariance and the conservation of mass, momentum and energy. We present two viewpoints, collisional viewpoint and dissipative viewpoint, to dissect the filter, and show that the filtered hyperbolic moment method can be treated as a solver of Vlasov equation. Numerical simulations of the linear Landau damping and two stream instability are tested to demonstrate the effectiveness of the filter in restraining recurrence arising from particle streaming. Both the analysis and the numerical results indicate that the filtered HME can capture the evolution of the Vlasov equation, even when phase mixing and filamentation are dominant.

We propose an entropic Fourier method for the numerical discretization of the Boltzmann collision operator. The method, which is obtained by modifying a Fourier Galerkin method to match the form of the discrete velocity method, can be viewed both as a discrete velocity method and as a Fourier method. As a discrete velocity method, it preserves the positivity of the solution and satisfies a discrete version of the H-theorem. As a Fourier method, it allows one to readily apply the FFT-based fast algorithms. A second-order convergence rate is validated by numerical experiments, Comment: 26 pages, 23 figures

We model the Knudsen layer in Kramers' problem by linearized high order hyperbolic moment system. Due to the hyperbolicity, the boundary conditions of the moment system is properly reduced from the kinetic boundary condition. For Kramers' problem, we give the analytical solutions of moment systems. With the order increasing of the moment model, the solutions are approaching to the solution of the linearized BGK kinetic equation. The velocity profile in the Knudsen layer is captured with improved accuracy for a wide range of accommodation coefficients.

Grad's moment models for Boltzmann equation were recently regularized to globally hyperbolic systems, and thus the regularized models attain local well-posedness for Cauchy data. The hyperbolic regularization is only related to the convection term in Boltzmann equation. We in this paper studied the regularized models with the presentation of collision terms. It is proved that the regularized models are linear stability at the local equilibrium and satisfy Yong's first stability condition with commonly used approximate collision terms, and particularly with Boltzmann's binary collision model., Comment: Dedicated to Prof. Zhen-Huan Teng's 80-th birthday

We point out that the quantum Grad's 13-moment system [R. Yano, Physica A: Statistical Mechanics and its Applications, 416:231-241, 2014] is lack of global hyperbolicity, and even worse, the thermodynamic equilibrium is not an interior point of the hyperbolicity region of the system. To remedy this problem, by fully considering Grad's expansion, we split the expansion into the equilibrium part and the non-equilibrium part, and propose a regularization for the system with the help of the new theory developed in [Z. Cai et al., SIAM J. Appl. Math., 75(5):2001-2023, 2015, Y. Fan, J. Stat. Phys., 161(4), 2015]. This provides us a new model which is hyperbolic for all admissible thermodynamic states, and meanwhile preserves the approximate accuracy of the original system. It should be noted that this procedure is not a trivial application of the theory in [Z. Cai et al., SIAM J. Appl. Math., 75(5):2001-2023, 2015, Y. Fan, J. Stat. Phys., 161(4), 2015]., Comment: 23 pages and 12 figures

By a further study of the mechanism of the hyperbolic regularization of the moment system for Boltzmann equation proposed in [Z. Cai, Y. Fan, R. Li, Comm. Math. Sci. 11(2): 547-571, 2013], we point out that the key point is treating the time and space derivative in the same way. Based on this understanding, a uniform framework to derive globally hyperbolic moment systems from kinetic equations using an operator projection method is proposed. The framework is so concise and clear that it can be treated as an algorithm with four inputs to derive hyperbolic moment system by routine calculations. Almost all existing globally hyperbolic moment system can be included in the framework, as well as some new moment system including globally hyperbolic regularized versions of Grad ordered moment system and a multidimensional extension of the quadrature-based moment system., Comment: 32 pages, 2 figures