Search

Variational Physics-Informed Neural Networks often suffer from poor convergence when using stochastic gradient-descent-based optimizers. By introducing a Least Squares solver for the weights of the last layer of the neural network, we improve the convergence of the loss during training in most practical scenarios. This work analyzes the computational cost of the resulting hybrid Least-Squares/Gradient-Descent optimizer and explains how to implement it efficiently. In particular, we show that a traditional implementation based on backward-mode automatic differentiation leads to a prohibitively expensive algorithm. To remedy this, we propose using either forward-mode automatic differentiation or an ultraweak-type scheme that avoids the differentiation of trial functions in the discrete weak formulation. The proposed alternatives are up to one hundred times faster than the traditional one, recovering a computational cost-per-iteration similar to that of a conventional gradient-descent-based optimizer alone. To support our analysis, we derive computational estimates and conduct numerical experiments in one- and two-dimensional problems., Comment: 23 pages, 10 figures, preprint

Language identification is a crucial component in the automated production of language resources, particularly in multilingual and big data contexts. However, commonly used language identifiers struggle to differentiate between similar or closely-related languages. This paper introduces FastSpell, a language identifier that combines fastText (a pre-trained language identifier tool) and Hunspell (a spell checker) with the aim of having a refined second-opinion before deciding which language should be assigned to a text. We provide a description of the FastSpell algorithm along with an explanation on how to use and configure it. To that end, we motivate the need of such a tool and present a benchmark including some popular language identifiers evaluated during the development of FastSpell. We show how FastSpell is useful not only to improve identification of similar languages, but also to identify new ones ignored by other tools.

We introduce a Robust version of the Variational Physics-Informed Neural Networks method (RVPINNs). As in VPINNs, we define the quadratic loss functional in terms of a Petrov-Galerkin-type variational formulation of the PDE problem: the trial space is a (Deep) Neural Network (DNN) manifold, while the test space is a finite-dimensional vector space. Whereas the VPINN's loss depends upon the selected basis functions of a given test space, herein, we minimize a loss based on the discrete dual norm of the residual. The main advantage of such a loss definition is that it provides a reliable and efficient estimator of the true error in the energy norm under the assumption of the existence of a local Fortin operator. We test the performance and robustness of our algorithm in several advection-diffusion problems. These numerical results perfectly align with our theoretical findings, showing that our estimates are sharp.

Leveraging Trace Theory, we investigate the efficient parallelization of direct solvers for large linear equation systems. Our focus lies on a multi-frontal algorithm, and we present a methodology for achieving near-optimal scheduling on modern massively parallel machines. By employing trace theory with Diekert Graphs and Foata Normal Form, we rigorously validate the effectiveness of our proposed solution. To establish a strong link between the mesh and elimination tree of the multi-frontal solver, we conduct extensive testing on matrices derived from the Finite Element Method (FEM). Furthermore, we assess the performance of computations on both GPU and CPU platforms, employing practical implementation strategies.

We propose a reliable and efficient a posteriori error estimator for a hybridizable discontinuous Galerkin (HDG) discretization of the Helmholtz equation, with a first-order absorbing boundary condition, based on residual minimization. Such a residual minimization is performed on a local and superconvergent postprocessing scheme of the approximation of the scalar solution provided by the HDG scheme. As a result, in addition to the super convergent approximation for the scalar solution, a residual representative in the Riesz sense, which is further employed to derive the a posteriori estimators. We illustrate our theoretical findings and the behavior of the a posteriori error estimator through two ad-hoc numerical experiments.

The Adaptive Stabilized Finite Element method (AS-FEM) developed in Calo et. al. combines the idea of the residual minimization method with the inf-sup stability offered by the discontinuous Galerkin (dG) frameworks. As a result, the discretizations deliver stabilized approximations and residual representatives in the dG space that can drive automatic adaptivity. We generalize AS FEM by considering continuous test spaces; thus, we propose a residual minimization method on a stable Continuous Interior Penalty (CIP) formulation that considers a C0-conforming trial FEM space and a test space based on the enrichment of the trial space by bubble functions. In our numerical experiments, the test space choice results in a significant reduction of the total degrees of freedom compared to the dG test spaces of Calo et. al. that converge at the same rate. Moreover, as trial and test spaces are C0-conforming, implementing a full dG data structure is unnecessary, simplifying the method's implementation considerably and making it appealing for industrial applications, see Labanda et. al.

We introduce an adaptive superconvergent finite element method for a class of mixed formulations to solve partial differential equations involving a diffusion term. It combines a superconvergent postprocessing technique for the primal variable with an adaptive finite element method via residual minimization. Such a residual minimization procedure is performed on a local postprocessing scheme, commonly used in the context of mixed finite element methods. Given the local nature of that approach, the underlying saddle point problems associated with residual minimizations can be solved with minimal computational effort. We propose and study a posteriori error estimators, including the built-in residual representative associated with residual minimization schemes; and an improved estimator which adds, on the one hand, a residual term quantifying the mismatch between discrete fluxes and, on the other hand, the interelement jumps of the postprocessed solution. We present numerical experiments in two dimensions using Brezzi-Douglas-Marini elements as input for our methodology. The experiments perfectly fit our key theoretical findings and suggest that our estimates are sharp.

We show how to construct the deep neural network (DNN) expert to predict quasi-optimal $hp$-refinements for a given computational problem. The main idea is to train the DNN expert during executing the self-adaptive $hp$-finite element method ($hp$-FEM) algorithm and use it later to predict further $hp$ refinements. For the training, we use a two-grid paradigm self-adaptive $hp$-FEM algorithm. It employs the fine mesh to provide the optimal $hp$ refinements for coarse mesh elements. We aim to construct the DNN expert to identify quasi-optimal $hp$ refinements of the coarse mesh elements. During the training phase, we use the direct solver to obtain the solution for the fine mesh to guide the optimal refinements over the coarse mesh element. After training, we turn off the self-adaptive $hp$-FEM algorithm and continue with quasi-optimal refinements as proposed by the DNN expert trained. We test our method on three-dimensional Fichera and two-dimensional L-shaped domain problems. We verify the convergence of the numerical accuracy with respect to the mesh size. We show that the exponential convergence delivered by the self-adaptive $hp$-FEM can be preserved if we continue refinements with a properly trained DNN expert. Thus, in this paper, we show that from the self-adaptive $hp$-FEM it is possible to train the DNN expert the location of the singularities, and continue with the selection of the quasi-optimal $hp$ refinements, preserving the exponential convergence of the method., Comment: 30 pages, 19 figures

Based on trace theory, we study efficient methods for concurrent integration of B-spline basis functions in IGA-FEM. We consider several scenarios of parallelization for two standard integration methods; the classical one and sum factorization. We aim to efficiently utilize hybrid memory machines, such as modern clusters, by focusing on the non-obvious layer of the shared memory part of concurrency. We estimate the performance of computations on a GPU and provide a strategy for performing such computations in practical implementations.

We study computational complexity aspects for Finite Element formulations considering hypercubic space--time full and time--marching discretization schemes for $h$--refined grids towards singularities. We perform a relatively comprehensive study of comparing the computational time via time complexities of direct and iterative solvers. We focus on the space-time formulation with refined computational grids and on the corresponding time slabs, namely, computational grids obtained by taking the "cross-sections" of the refined space-time mesh. We compare the computational complexity of the space-time formulation and the corresponding time--marching scheme. Our consideration concerns the computational complexity of the multi-frontal solvers, the iterative solvers, as well as the static condensation. Numerical experiments with Octave confirm our theoretical findings., Comment: 36 pages, 13 figures, 9 tables

In order to construct regularizations of continuous linear functionals acting on Sobolev spaces such as $W_0^{1,q}(\Omega)$, where $1

In this paper, we describe a stable finite element formulation for advection-diffusion-reaction problems that allows for robust automatic adaptive strategies to be easily implemented. We consider locally vanishing, heterogeneous, and anisotropic diffusivities, as well as advection-dominated diffusion problems. The general stabilized finite element framework was described and analyzed in arXiv:1907.12605v3 for linear problems in general, and tested for pure advection problems. The method seeks for the discrete solution through a residual minimization process on a proper stable discontinuous Galerkin (dG) dual norm. This technique leads to a saddle-point problem that delivers a stable discrete solution and a robust error estimate that can drive mesh adaptivity. In this work, we demonstrate the efficiency of the method in extreme scenarios, delivering stable solutions. The quality and performance of the solutions are comparable to classical discontinuous Galerkin formulations in the respective discrete space norm on each mesh. Meanwhile, this technique allows us to solve on coarse meshes and adapt the solution to achieve a user-specified solution quality.

We model incompressible flows with an adaptive stabilized finite element method Stokes flows, which solves a discretely stable saddle-point problem to approximate the velocity-pressure pair. Additionally, this saddle-point problem delivers a robust error estimator to guide mesh adaptivity. We analyze the accuracy of different discrete velocity-pressure pairs of continuous finite element spaces, which do not necessarily satisfy the discrete inf-sup condition. We validate the framework's performance with numerical examples.

Under compressive creep, visco-plastic solids experiencing internal mass transfer processes have been recently proposed to accommodate singular cnoidal wave solutions, as material instabilities at the stationary wave limit. These instabilities appear when the loading rate is significantly faster than the capability of the material to diffuse internal perturbations and lead to localized failure features (e.g., cracks and compaction bands). This type of solution, generally found in fluids, has strong nonlinearities and periodic patterns. Due to the singular nature of the solutions, the applicability of the theory is currently limited. Additionally, effective numerical tools require proper regularization to overcome the challenges that singularity induces. We focus on the numerical treatment of the governing equation using a nonlinear approach building on a recent adaptive stabilized finite element method. This method provides a residual representation to drive adaptive mesh refinement, a particularly useful feature for the problem at hand. We compare against analytical and standard finite element solutions to demonstrate the performance of our approach. We then investigate the sensitivity of the diffusivity ratio, main parameter of the problem, and identify multiple possible solutions, with multiple stress peaks. Finally, we show the evolution of the spacing between peaks for all solutions as a function of that parameter.

We devise a stabilized method to weakly enforce bound constraints in the discrete solution of advection-dominated diffusion problems. This method combines a nonlinear penalty formulation with a discontinuous Galerkin-based residual minimization method. We illustrate the efficiency of this scheme for both uniform and adaptive meshes through proper numerical examples.

We propose a goal-oriented mesh-adaptive algorithm for a finite element method stabilized via residual minimization on dual discontinuous-Galerkin norms. By solving a saddle-point problem, this residual minimization delivers a stable continuous approximation to the solution on each mesh instance and a residual projection onto a broken polynomial space, which is a robust error estimator to minimize the discrete energy norm via automatic mesh refinement. In this work, we propose and analyze a goal-oriented adaptive algorithm for this stable residual minimization. We solve the primal and adjoint problems considering the same saddle-point formulation and different right-hand sides. By solving a third stable problem, we obtain two efficient error estimates to guide goal-oriented adaptivity. We illustrate the performance of this goal-oriented adaptive strategy on advection-diffusion-reaction problems.

We design and analyze a new adaptive stabilized finite element method. We construct a discrete approximation of the solution in a continuous trial space by minimizing the residual measured in a dual norm of a discontinuous test space that has inf-sup stability. We formulate this residual minimization as a stable saddle-point problem which delivers a stabilized discrete solution and a residual representation that drives the adaptive mesh refinement. Numerical results on an advection-reaction model problem show competitive error reduction rates when compared to discontinuous Galerkin methods on uniformly refined meshes and smooth solutions. Moreover, the technique leads to optimal decay rates for adaptive mesh refinement and solutions having sharp layers.

Most variational forms of isogeometric analysis use highly-continuous basis functions for both trial and test spaces. For a partial differential equation with a smooth solution, isogeometric analysis with highly-continuous basis functions for trial space results in excellent discrete approximations of the solution. However, we observe that high continuity for test spaces is not necessary. In this work, we present a framework which uses highly-continuous B-splines for the trial spaces and basis functions with minimal regularity and possibly lower order polynomials for the test spaces. To realize this goal, we adopt the residual minimization methodology. We pose the problem in a mixed formulation, which results in a system governing both the solution and a Riesz representation of the residual. We present various variational formulations which are variationally-stable and verify their equivalence numerically via numerical tests.