Search

We study two-layer neural networks whose domain and range are Banach spaces with separable preduals. In addition, we assume that the image space is equipped with a partial order, i.e. it is a Riesz space. As the nonlinearity we choose the lattice operation of taking the positive part; in case of $\mathbb R^d$-valued neural networks this corresponds to the ReLU activation function. We prove inverse and direct approximation theorems with Monte-Carlo rates for a certain class of functions, extending existing results for the finite-dimensional case. In the second part of the paper, we study, from the regularisation theory viewpoint, the problem of finding optimal representations of such functions via signed measures on a latent space from a finite number of noisy observations. We discuss regularity conditions known as source conditions and obtain convergence rates in a Bregman distance for the representing measure in the regime when both the noise level goes to zero and the number of samples goes to infinity at appropriate rates.

Loss functions with a large number of saddle points are one of the major obstacles for training modern machine learning (ML) models efficiently. First-order methods such as gradient descent (GD) are usually the methods of choice for training ML models. However, these methods converge to saddle points for certain choices of initial guesses. In this paper, we propose a modification of the recently proposed Laplacian smoothing gradient descent (LSGD) [Osher et al., arXiv:1806.06317], called modified LSGD (mLSGD), and demonstrate its potential to avoid saddle points without sacrificing the convergence rate. Our analysis is based on the attraction region, formed by all starting points for which the considered numerical scheme converges to a saddle point. We investigate the attraction region’s dimension both analytically and numerically. For a canonical class of quadratic functions, we show that the dimension of the attraction region for mLSGD is $\lfloor (n-1)/2\rfloor$ , and hence it is significantly smaller than that of GD whose dimension is $n-1$ .

We consider approximating the solution of the Helmholtz exterior Dirichlet problem for a nontrapping obstacle, with boundary data coming from plane-wave incidence, by the solution of the corresponding boundary value problem where the exterior domain is truncated and a local absorbing boundary condition coming from a Pad\'e approximation (of arbitrary order) of the Dirichlet-to-Neumann map is imposed on the artificial boundary (recall that the simplest such boundary condition is the impedance boundary condition). We prove upper- and lower-bounds on the relative error incurred by this approximation, both in the whole domain and in a fixed neighbourhood of the obstacle (i.e. away from the artificial boundary). Our bounds are valid for arbitrarily-high frequency, with the artificial boundary fixed, and show that the relative error is bounded away from zero, independent of the frequency, and regardless of the geometry of the artificial boundary., Comment: 86 pages

Rejuvenation in particle filters is necessary to prevent the collapse of the weights when the number of particles is insufficient to properly sample the high-probability regions of the state space. Rejuvenation is often implemented in a heuristic manner by the addition of random noise that widens the support of the ensemble. This work aims at improving canonical rejuvenation methodology by the introduction of additional prior information obtained from climatological samples; the dynamical particles used for importance sampling are augmented with samples obtained from stochastic covariance shrinkage. A localized variant of the proposed method is developed. Numerical experiments with the Lorenz '63 model show that modified filters significantly improve the analyses for low dynamical ensemble sizes. Furthermore, localization experiments with the Lorenz '96 model show that the proposed methodology is extendable to larger systems.

This work proposes an adaptive structure-preserving model order reduction method for finite-dimensional parametrized Hamiltonian systems modeling non-dissipative phenomena. To overcome the slowly decaying Kolmogorov width typical of transport problems, the full model is approximated on local reduced spaces that are adapted in time using dynamical low-rank approximation techniques. The reduced dynamics is prescribed by approximating the symplectic projection of the Hamiltonian vector field in the tangent space to the local reduced space. This ensures that the canonical symplectic structure of the Hamiltonian dynamics is preserved during the reduction. In addition, accurate approximations with low-rank reduced solutions are obtained by allowing the dimension of the reduced space to change during the time evolution. Whenever the quality of the reduced solution, assessed via an error indicator, is not satisfactory, the reduced basis is augmented in the parameter direction that is worst approximated by the current basis. Extensive numerical tests involving wave interactions, nonlinear transport problems, and the Vlasov equation demonstrate the superior stability properties and considerable runtime speedups of the proposed method as compared to global and traditional reduced basis approaches.

We provide a framework to prove convergence rates for discretizations of kinetic Langevin dynamics for $M$-$\nabla$Lipschitz $m$-log-concave densities. Our approach provides convergence rates of $\mathcal{O}(m/M)$, with explicit stepsize restrictions, which are of the same order as the stability threshold for Gaussian targets and are valid for a large interval of the friction parameter. We apply this methodology to various integration methods which are popular in the molecular dynamics and machine learning communities. Finally we introduce the property ``$\gamma$-limit convergent" (GLC) to characterise underdamped Langevin schemes that converge to overdamped dynamics in the high friction limit and which have stepsize restrictions that are independent of the friction parameter; we show that this property is not generic by exhibiting methods from both the class and its complement.

In this contribution, we extend the hybridization framework for the Hodge Laplacian (Awanou et al., Hybridization and postprocessing in finite element exterior calculus, 2023) to port-Hamiltonian systems. To this aim, a general dual field continuous Galerkin discretization is introduced, in which one variable is approximated via conforming finite element spaces, whereas the second is completely local. This scheme retains a discrete power balance and discrete conservation laws and is directly amenable to hybridization. The hybrid formulation is equivalent to the continuous Galerkin formulation and to a power preserving interconnection of port-Hamiltonian systems, thus providing a system theoretic interpretation of finite element assembly. The hybrid system can be efficiently solved using a static condensation procedure in discrete time. The size reduction achieved thanks to the hybridization is greater than the one obtained for the Hodge Laplacian as one field is completely discarded. Numerical experiments on the 3D wave and Maxwell equations show the equivalence of the continuous and hybrid formulation and the computational gain achieved by the latter., 34 pages, 14 figures

To overcome topological constraints and improve the expressiveness of normalizing ow architectures, Wu, Kohler, and Noe introduced stochastic normalizing ows which combine deterministic, learnable ow transformations with stochastic sampling methods. In this paper, we consider stochastic normalizing ows from a Markov chain point of view. In particular, we replace transition densities by general Markov kernels and establish proofs via Radon-Nikodym derivatives, which allows us to incorporate distributions without densities in a sound way. Further, we generalize the results for sampling from posterior distributions as required in inverse problems. The performance of the proposed conditional stochastic normalizing ow is demonstrated by numerical examples.

Biomimicry is a powerful science that takes advantage of nature's remarkable ability to devise innovative solutions to challenging problems. In this work, we use asymptotic methods to develop the mathematical foundations for the exchange of design inspiration and features between biological hearing systems, signal processing algorithms and acoustic metamaterials. Our starting point is a concise asymptotic analysis of high-contrast acoustic metamaterials. We are able to fine tune this graded structure to mimic the biomechanical properties of the cochlea, at the same scale. We then turn our attention to developing a biomimetic signal processing algorithm. We use the response of the cochlea-like metamaterial as an initial filtering layer and then add additional biomimetic processing stages, designed to mimic the human auditory system's ability to recognise the global properties of natural sounds. This demonstrates the three-way exchange of ideas that, thanks to our analysis, is possible between signal processing, metamaterials and biology.

The Stochastic Primal-Dual Hybrid Gradient or SPDHG is an algorithm proposed by Chambolle et al. to efficiently solve a wide class of nonsmooth large-scale optimization problems. In this paper we contribute to its theoretical foundations and prove its almost sure convergence for convex but neither necessarily strongly convex nor smooth functionals, defined on Hilbert spaces of arbitrary dimension. We also prove its convergence for any arbitrary sampling, and for some specific samplings we propose theoretically optimal step size parameters which yield faster convergence. In addition, we propose using SPDHG for parallel Magnetic Resonance Imaging reconstruction, where data from different coils are randomly selected at each iteration. We apply SPDHG using a wide range of random sampling methods. We compare its performance across a range of settings, including mini-batch size, step size parameters, and both convex and strongly convex objective functionals. We show that the sampling can significantly affect the convergence speed of SPDHG. We conclude that for many cases an optimal sampling method can be identified.

The Regularised Inertial Dean-Kawasaki model (RIDK) is a nonlinear stochastic PDE which captures fluctuations around the mean-field limit for large-scale particle systems in both density and momentum. We focus on the following two aspects. Firstly, we set up a discontinuous Galerkin (DG) discretisation scheme for the RIDK model: we provide suitable definitions of numerical fluxes at the interface of the mesh elements which are consistent with the wave-type nature of the RIDK model and grant stability of the simulations, and we quantify the rate of convergence in mean square to the continuous RIDK model. Secondly, we introduce modifications of the RIDK model in order to preserve positivity of the density (such a feature does not hold for the original RIDK model). By means of numerical simulations, we show that the modifications lead to physically realistic and positive density profiles. In one case, subject to additional regularity constraints, we also prove positivity. Finally, we present an application of our methodology to a system of diffusing and reacting particles. Our Python code is available in open-source format.

Mainstream numerical Partial Differential Equation (PDE) solvers require discretizing the physical domain using a mesh. Mesh movement methods aim to improve the accuracy of the numerical solution by increasing mesh resolution where the solution is not well-resolved, whilst reducing unnecessary resolution elsewhere. However, mesh movement methods, such as the Monge-Ampere method, require the solution of auxiliary equations, which can be extremely expensive especially when the mesh is adapted frequently. In this paper, we propose to our best knowledge the first learning-based end-to-end mesh movement framework for PDE solvers. Key requirements of learning-based mesh movement methods are alleviating mesh tangling, boundary consistency, and generalization to mesh with different resolutions. To achieve these goals, we introduce the neural spline model and the graph attention network (GAT) into our models respectively. While the Neural-Spline based model provides more flexibility for large deformation, the GAT based model can handle domains with more complicated shapes and is better at performing delicate local deformation. We validate our methods on stationary and time-dependent, linear and non-linear equations, as well as regularly and irregularly shaped domains. Compared to the traditional Monge-Ampere method, our approach can greatly accelerate the mesh adaptation process, whilst achieving comparable numerical error reduction.

We introduce a deep learning accelerated methodology to solve PDE-based Bayesian inverse problems with guaranteed accuracy. This is motivated by solving the ill-posed problem of inferring a spatio-temporal heat-flux parameter known as the Biot number in a PDE model given temperature data, however the methodology is generalisable to other settings. To achieve accelerated Bayesian inference we develop a novel training scheme that uses data to adaptively train a neural-network surrogate simulating the parametric forward model. By simultaneously identifying an approximate posterior distribution over the Biot number, and weighting a physics-informed training loss according to this, our approach approximates forward and inverse solution together without any need for external solves. Using a random Chebyshev series, we outline how to approximate a Gaussian process prior, and using the surrogate we apply Hamiltonian Monte Carlo (HMC) to sample from the posterior distribution. We derive convergence of the surrogate posterior to the true posterior distribution in the Hellinger metric as our adaptive loss approaches zero. Additionally, we describe how this surrogate-accelerated HMC approach can be combined with traditional PDE solvers in a delayed-acceptance scheme to a-priori control the posterior accuracy. This overcomes a major limitation of deep learning-based surrogate approaches, which do not achieve guaranteed accuracy a-priori due to their non-convex training. Biot number calculations are involved in turbo-machinery design, which is safety critical and highly regulated, therefore it is important that our results have such mathematical guarantees. Our approach achieves fast mixing in high-dimensional parameter spaces, whilst retaining the convergence guarantees of a traditional PDE solver, and without the burden of evaluating this solver for proposals that are likely to be rejected. A range of numerical results are given using real and simulated data that compare adaptive and general training regimes and various gradient-based Markov chain Monte Carlo (MCMC) sampling methods.

In this paper we propose a novel approach to discretize linear port-Hamiltonian systems while preserving the underlying structure. We present a finite element exterior calculus formulation that is able to mimetically represent conservation laws and cope with mixed open boundary conditions using a single computational mesh. The possibility of including open boundary conditions allows for modular composition of complex multi-physical systems whereas the exterior calculus formulation provides a coordinate-free treatment. Our approach relies on a dual-field representation of the physical system that is redundant at the continuous level but eliminates the need of mimicking the Hodge star operator at the discrete level. By considering the Stokes-Dirac structure representing the system together with its adjoint, which embeds the metric information directly in the codifferential, the need for an explicit discrete Hodge star is avoided altogether. By imposing the boundary conditions in a strong manner, the power balance characterizing the Stokes-Dirac structure is then retrieved at the discrete level via symplectic Runge-Kutta integrators based on Gauss-Legendre collocation points. Numerical experiments validate the convergence of the method and the conservation properties in terms of energy balance both for the wave and Maxwell equations in a three dimensional domain. For the latter example, the magnetic and electric fields preserve their divergence free nature at the discrete level., Comment: 44 pages, 15 figures

Strongly interacting electrons in solids are generically described by Hubbardtype models, and the impact of solar light can be modeled by an additional time-dependence. This yields a finite dimensional system of ordinary differential equations (ODE)s of Schr\"odinger type, which can be solved numerically by exponential time integrators of Magnus type. The efficiency may be enhanced by combining these with operator splittings. We will discuss several different approaches of employing exponential-based methods in conjunction with an adaptive Lanczos method for the evaluation of matrix exponentials and compare their accuracy and efficiency. For each integrator, we use defect-based local error estimators to enable adaptive time-stepping. This serves to reliably control the approximation error and reduce the computational effort

This study assesses a Continuous Data Assimilation (CDA) dynamical-downscaling algorithm for enhancing the simulation of the Indian summer monsoon (ISM) system. CDA is a mathematically rigorous technique that has been recently introduced to constrain the large-scale features of high-resolution atmospheric models with coarse spatial scale data. It is similar to spectral nudging but does not require any spectral decomposition for scales separation. This is expected to be particularly relevant for ISM, which involves various interactions between large-scale circulations and regional physical processes. Along with a control simulation, several downscaling simulations were conducted with the Weather Research and Forecasting (WRF) model using CDA, spectral (retaining different wavenumbers) and grid nudging for three ISM seasons: normal (2016), excess (2013), and drought (2009). The simulations are nested within the NCEP Final Analysis and the model outputs are evaluated against the observations. Compared to grid and spectral nudging, the simulations using CDA produce enhanced ISM features over the Indian subcontinent including the low-level jet, tropical easterly jet, easterly wind shear, and rainfall distributions for all investigated ISM seasons. The major ISM processes, in particular the monsoon inversion over the Arabian Sea, tropospheric temperature gradients and moist static energy over central India, and zonal wind shear over the monsoon region, are all better simulated with CDA. Spectral nudging outputs are found to be sensitive to the choice of the wavenumber, requiring careful tuning to provide robust simulations of the ISM system. In contrast, control and grid nudging generally fail to well reproduce some of the main ISM features., Comment: Research Article

Easy to construct and optimally convergent generalisations of B-splines to unstructured meshes are essential for the application of isogeometric analysis to domains with non-trivial topologies. Nonetheless, especially for hexahedral meshes, the construction of smooth and optimally convergent isogeometric analysis basis functions is still an open question. We introduce a simple partition of unity construction that yields smooth blended B-splines, referred to as SB-splines, on semi-structured quadrilateral and hexahedral meshes, namely on mostly structured meshes with a few sufficiently separated unstructured regions. To this end, we first define the mixed smoothness B-splines that are C0 continuous in the unstructured regions of the mesh but have higher smoothness everywhere else. Subsequently, the SB-splines are obtained by smoothly blending in the physical space the mixed smoothness B-splines with Bernstein bases of equal degree. One of the key novelties of our approach is that the required smooth weight functions are assembled from the available smooth B-splines on the unstructured mesh. The SB-splines are globally smooth, non- negative, have no breakpoints within the elements and reduce to conventional B-splines away from the unstructured regions of the mesh. Although we consider only quadratic mixed smoothness B-splines in this paper, the construction generalises to arbitrary degrees. We demonstrate the excellent performance of SB-splines studying Poisson and bihar- monic problems on semi-structured quadrilateral and hexahedral meshes, and numerically establishing their optimal convergence in one and two dimensions.

We study the problem of unbiased estimation of expectations with respect to (w.r.t.) $\pi$ a given, general probability measure on $(\mathbb{R}^d,\mathcal{B}(\mathbb{R}^d))$ that is absolutely continuous with respect to a standard Gaussian measure. We focus on simulation associated to a particular class of diffusion processes, sometimes termed the Schr\"odinger-F\"ollmer Sampler, which is a simulation technique that approximates the law of a particular diffusion bridge process $\{X_t\}_{t\in [0,1]}$ on $\mathbb{R}^d$, $d\in \mathbb{N}_0$. This latter process is constructed such that, starting at $X_0=0$, one has $X_1\sim \pi$. Typically, the drift of the diffusion is intractable and, even if it were not, exact sampling of the associated diffusion is not possible. As a result, \cite{sf_orig,jiao} consider a stochastic Euler-Maruyama scheme that allows the development of biased estimators for expectations w.r.t.~$\pi$. We show that for this methodology to achieve a mean square error of $\mathcal{O}(\epsilon^2)$, for arbitrary $\epsilon>0$, the associated cost is $\mathcal{O}(\epsilon^{-5})$. We then introduce an alternative approach that provides unbiased estimates of expectations w.r.t.~$\pi$, that is, it does not suffer from the time discretization bias or the bias related with the approximation of the drift function. We prove that to achieve a mean square error of $\mathcal{O}(\epsilon^2)$, the associated cost is, with high probability, $\mathcal{O}(\epsilon^{-2}|\log(\epsilon)|^{2+\delta})$, for any $\delta>0$. We implement our method on several examples including Bayesian inverse problems., Comment: 27 pages, 11 figures

Dust explosions are among the most hazardous accidents affecting industrial facilities processing particulate solids. Describing the severity parameters of dust clouds is critical to the safety management and risk assessment of dust explosions. These parameters are determined experimentally in a 20 L spherical vessel, following the ASTM E1226 or UNE 14034 standards. Since their reproducibility depends on the levels of turbulence associated with the dust cloud, a computational model of the multi-phase (gas-solid) flow is used to simulate the dispersion process with the open-source CFD code OpenFOAM. The model is successfully validated against experimental measurements from the literature and numerical results of a commercial CFD code. In addition, this study considers the impact of particle size on the turbulence of the carrier phase, suggesting that particles attenuate its turbulence intensity. Moreover, the model predicts well the formation of a two-vortex flow pattern, which has a negative impact on the distribution of the particle-laden flows with dp≤ 100 μm, as most of the particles concentrate at the near-wall region. Contrarily, an improved homogeneity of dust cloud is observed for a case fed with larger particles (dp= 200 μm), as the increased inertia of these particles allows them to enter into the re-circulation regions.

Exponential integrators are a well-known class of time integration methods that have been the subject of many studies and developments in the past two decades. Surprisingly, there have been limited efforts to analyze their stability and efficiency on non-diffusive equations to date. In this paper we apply linear stability analysis to showcase the poor stability properties of exponential integrators on non-diffusive problems. We then propose a simple repartitioning approach that stabilizes the integrators and enables the efficient solution of stiff, non-diffusive equations. To validate the effectiveness of our approach, we perform several numerical experiments that compare partitioned exponential integrators to unmodified ones. We also compare repartitioning to the well-known approach of adding hyperviscosity to the equation right-hand-side. Overall, we find that the repartitioning restores convergence at large timesteps and, unlike hyperviscosity, it does not require the use of high-order spatial derivatives.