Search

The training of artificial neural networks (ANNs) with rectified linear unit (ReLU) activation via gradient descent (GD) type optimization schemes is nowadays a common industrially relevant procedure. GD type optimization schemes can be regarded as temporal discretization methods for the gradient flow (GF) differential equations associated to the considered optimization problem and, in view of this, it seems to be a natural direction of research to first aim to develop a mathematical convergence theory for time-continuous GF differential equations and, thereafter, to aim to extend such a time-continuous convergence theory to implementable time-discrete GD type optimization methods. In this article we establish two basic results for GF differential equations in the training of fully-connected feedforward ANNs with one hidden layer and ReLU activation. In the first main result of this article we establish in the training of such ANNs under the assumption that the probability distribution of the input data of the considered supervised learning problem is absolutely continuous with a bounded density function that every GF differential equation admits for every initial value a solution which is also unique among a suitable class of solutions. In the second main result of this article we prove in the training of such ANNs under the assumption that the target function and the density function of the probability distribution of the input data are piecewise polynomial that every non-divergent GF trajectory converges with an appropriate rate of convergence to a critical point and that the risk of the non-divergent GF trajectory converges with rate 1 to the risk of the critical point. We establish this result by proving that the considered risk function is semialgebraic and, consequently, satisfies the Kurdyka-Łojasiewicz inequality , which allows us to show convergence of every non-divergent GF trajectory. [ABSTRACT FROM AUTHOR]

In this paper we consider a semilinear parabolic equation in an infinite cylinder with shrinking cross section. The portion where the domain is not a straight half-cylinder is assumed to be compact. The spatially varying nonlinearity is such that it connects two (spatially independent) bistable nonlinearities in a compact set in space. We prove that, given such a setting, a traveling wave obeying the equation with the one bistable nonlinearity and starting at the respective side of the decreasing cylinder, will converge to a traveling wave solution prescribed by the nonlinearity on the other side. [ABSTRACT FROM AUTHOR]

We consider a Fokker–Planck equation which is coupled to an externally given time-dependent constraint on its first moment. This constraint introduces a Lagrange-multiplier which renders the equation nonlocal and nonlinear. In this paper we exploit an interpretation of this equation as a Wasserstein gradient flow of a free energy F on a time-constrained manifold. First, we prove existence of solutions by passing to the limit in an implicit Euler scheme obtained by minimizing h F ( ϱ ) + W 2 2 ( ϱ 0 , ϱ ) among all ϱ satisfying the constraint for some ϱ 0 and time-step h > 0 . Second, we provide quantitative estimates for the rate of convergence to equilibrium when the constraint converges to a constant. The proof is based on the investigation of a suitable relative entropy with respect to minimizers of the free energy chosen according to the constraint. The rate of convergence can be explicitly expressed in terms of constants in suitable logarithmic Sobolev inequalities. [ABSTRACT FROM AUTHOR]

In this paper we give an explicit formula for the homogenization limit of Poisson's equation for a wide range of non-periodic problems including self-similarly ramified domains. This work was motivated by the modelling of the diffusion of medical sprays in lungs, which can be approximated by a self-similarly ramified domain. Such motivation also led us to consider the influence that a continuous scaling of the size of holes towards a chosen direction (e.g. towards the fractal boundary) has on the homogenization limit. It turned out that our strategy to explicitly calculate a formula for the homogenization limit could be applied beyond self-similar perforations as presented in this article. [ABSTRACT FROM AUTHOR]

In this paper we give a full classification of global solutions of the obstacle problem for the fractional Laplacian (including the thin obstacle problem) with compact coincidence set and at most polynomial growth in dimension N ≥ 3. We do this in terms of a bijection onto a set of polynomials describing the asymptotics of the solution. Furthermore we prove that coincidence sets of global solutions that are compact are also convex if the solution has at most quadratic growth. [ABSTRACT FROM AUTHOR]

In this paper we derive quantitative conditions under which a compactly supported drift term depending on the direction of propagation blocks a traveling wave solution or lets it pass almost unchanged. We give explicit conditions on the drift term for blocking as well as almost unchanged propagation in one spatial dimension. [ABSTRACT FROM AUTHOR]