Search

We study global solutions to the thin obstacle problem with at most quadratic growth at infinity. We show that every ellipsoid can be realized as the contact set of such a solution. On the other hand, if such a solution has a compact contact set, we show that it must be an ellipsoid.

The training of artificial neural networks (ANNs) is nowadays a highly relevant algorithmic procedure with many applications in science and industry. Roughly speaking, ANNs can be regarded as iterated compositions between affine linear functions and certain fixed nonlinear functions, which are usually multidimensional versions of a one-dimensional so-called activation function. The most popular choice of such a one-dimensional activation function is the rectified linear unit (ReLU) activation function which maps a real number to its positive part $ \mathbb{R} \ni x \mapsto \max\{ x, 0 \} \in \mathbb{R} $. In this article we propose and analyze a modified variant of the standard training procedure of such ReLU ANNs in the sense that we propose to restrict the negative gradient flow dynamics to a large submanifold of the ANN parameter space, which is a strict $ C^{ \infty } $-submanifold of the entire ANN parameter space that seems to enjoy better regularity properties than the entire ANN parameter space but which is also sufficiently large and sufficiently high dimensional so that it can represent all ANN realization functions that can be represented through the entire ANN parameter space. In the special situation of shallow ANNs with just one-dimensional ANN layers we also prove for every Lipschitz continuous target function that every gradient flow trajectory on this large submanifold of the ANN parameter space is globally bounded. For the standard gradient flow on the entire ANN parameter space with Lipschitz continuous target functions it remains an open problem of research to prove or disprove the global boundedness of gradient flow trajectories even in the situation of shallow ANNs with just one-dimensional ANN layers., 26 pages, 1 figure

The training of artificial neural networks (ANNs) with rectified linear unit (ReLU) activation via gradient descent (GD) type optimization schemes is nowadays a common industrially relevant procedure. GD type optimization schemes can be regarded as temporal discretization methods for the gradient flow (GF) differential equations associated to the considered optimization problem and, in view of this, it seems to be a natural direction of research to first aim to develop a mathematical convergence theory for time-continuous GF differential equations and, thereafter, to aim to extend such a time-continuous convergence theory to implementable time-discrete GD type optimization methods. In this article we establish two basic results for GF differential equations in the training of fully-connected feedforward ANNs with one hidden layer and ReLU activation. In the first main result of this article we establish in the training of such ANNs under the assumption that the probability distribution of the input data of the considered supervised learning problem is absolutely continuous with a bounded density function that every GF differential equation admits for every initial value a solution which is also unique among a suitable class of solutions. In the second main result of this article we prove in the training of such ANNs under the assumption that the target function and the density function of the probability distribution of the input data are piecewise polynomial that every non-divergent GF trajectory converges with an appropriate rate of convergence to a critical point and that the risk of the non-divergent GF trajectory converges with rate 1 to the risk of the critical point. We establish this result by proving that the considered risk function is semialgebraic and, consequently, satisfies the Kurdyka-Łojasiewicz inequality, which allows us to show convergence of every non-divergent GF trajectory., Finanziert durch den Open-Access-Publikationsfonds der Westfälischen Wilhelms-Universität Münster (WWU Münster).

In this thesis we investigate global solutions of the classical obstacle problem. We give a partial result towards a conjecture by H. Shahgholian ('92) saying that the coincidence sets of global solutions of the obstacle problem are in the closure of ellipsoids, i.e. ellipsoids, paraboloids, cylinders with one of the two as basis, or half-spaces. We give a short proof of the known result that bounded coincidence sets of global solutions of the obstacle problem that have non-empty interior are ellipsoids. Our main and new result is that in dimensions greater or equal to 6 coincidence sets of global solutions, that are not constant in any direction and have a blow-down that is independent of exactly one direction, are paraboloids if they have non-empty interior. The proof rests on a careful analysis of the asymptotics of solutions at infinity, the Newton-potential expansion of the solution and a comparison argument that only requires two solutions to be compared on a sufficiently large portion of huge spheres., In dieser Arbeit untersuchen wir globale Lösungen des klassischen Hindernisproblems. Wir zeigen einen Teil einer Vermutung von H. Shahgholian (’92), die besagt, dass die Koinzidenzmenge globaler Lösungen des Hindernisproblems im Abschluss von Ellipsoiden liegt, d.h. ein Ellipsoid, ein Paraboloid, ein Zylinder mit einer der beiden Mengen als Basis oder ein Halbraum ist. Wir geben zunächst einen kurzen Beweis des klassischen Resultats, dass die Koinzidenzmenge globaler Lösungen des Hindernisproblems, wenn sie beschränkt ist und nicht-leeres Inneres hat, ein Ellipsoid sein muss. Unser neues Resultat und das Hauptresultat dieser Arbeit ist, dass in Dimensionen größer oder gleich sechs die Koinzidenzmenge einer globalen Lösung der Hindernisproblems, die in keine Richtung konstant ist und einen blow-down hat, der in genau eine Richtung verschwindet, wenn sie nicht-leeres Inneres hat, ein Paraboloid ist. Der Beweis basiert auf einer eingehenden Untersuchung des asymptotischen Verhaltens der Lösung im Unendlichen, der Newton-Potential-Entwicklung der Lösung und einem Vergleich zweier Lösungen, der nur benötigt, dass die beiden Lösungen auf ausreichend großen Bereichen großer Sphären vergleichbar sind.