Search

The training of artificial neural networks (ANNs) with rectified linear unit (ReLU) activation via gradient descent (GD) type optimization schemes is nowadays a common industrially relevant procedure. GD type optimization schemes can be regarded as temporal discretization methods for the gradient flow (GF) differential equations associated to the considered optimization problem and, in view of this, it seems to be a natural direction of research to first aim to develop a mathematical convergence theory for time-continuous GF differential equations and, thereafter, to aim to extend such a time-continuous convergence theory to implementable time-discrete GD type optimization methods. In this article we establish two basic results for GF differential equations in the training of fully-connected feedforward ANNs with one hidden layer and ReLU activation. In the first main result of this article we establish in the training of such ANNs under the assumption that the probability distribution of the input data of the considered supervised learning problem is absolutely continuous with a bounded density function that every GF differential equation admits for every initial value a solution which is also unique among a suitable class of solutions. In the second main result of this article we prove in the training of such ANNs under the assumption that the target function and the density function of the probability distribution of the input data are piecewise polynomial that every non-divergent GF trajectory converges with an appropriate rate of convergence to a critical point and that the risk of the non-divergent GF trajectory converges with rate 1 to the risk of the critical point. We establish this result by proving that the considered risk function is semialgebraic and, consequently, satisfies the Kurdyka-Łojasiewicz inequality, which allows us to show convergence of every non-divergent GF trajectory., Finanziert durch den Open-Access-Publikationsfonds der Westfälischen Wilhelms-Universität Münster (WWU Münster).

In this thesis we investigate global solutions of the classical obstacle problem. We give a partial result towards a conjecture by H. Shahgholian ('92) saying that the coincidence sets of global solutions of the obstacle problem are in the closure of ellipsoids, i.e. ellipsoids, paraboloids, cylinders with one of the two as basis, or half-spaces. We give a short proof of the known result that bounded coincidence sets of global solutions of the obstacle problem that have non-empty interior are ellipsoids. Our main and new result is that in dimensions greater or equal to 6 coincidence sets of global solutions, that are not constant in any direction and have a blow-down that is independent of exactly one direction, are paraboloids if they have non-empty interior. The proof rests on a careful analysis of the asymptotics of solutions at infinity, the Newton-potential expansion of the solution and a comparison argument that only requires two solutions to be compared on a sufficiently large portion of huge spheres., In dieser Arbeit untersuchen wir globale Lösungen des klassischen Hindernisproblems. Wir zeigen einen Teil einer Vermutung von H. Shahgholian (’92), die besagt, dass die Koinzidenzmenge globaler Lösungen des Hindernisproblems im Abschluss von Ellipsoiden liegt, d.h. ein Ellipsoid, ein Paraboloid, ein Zylinder mit einer der beiden Mengen als Basis oder ein Halbraum ist. Wir geben zunächst einen kurzen Beweis des klassischen Resultats, dass die Koinzidenzmenge globaler Lösungen des Hindernisproblems, wenn sie beschränkt ist und nicht-leeres Inneres hat, ein Ellipsoid sein muss. Unser neues Resultat und das Hauptresultat dieser Arbeit ist, dass in Dimensionen größer oder gleich sechs die Koinzidenzmenge einer globalen Lösung der Hindernisproblems, die in keine Richtung konstant ist und einen blow-down hat, der in genau eine Richtung verschwindet, wenn sie nicht-leeres Inneres hat, ein Paraboloid ist. Der Beweis basiert auf einer eingehenden Untersuchung des asymptotischen Verhaltens der Lösung im Unendlichen, der Newton-Potential-Entwicklung der Lösung und einem Vergleich zweier Lösungen, der nur benötigt, dass die beiden Lösungen auf ausreichend großen Bereichen großer Sphären vergleichbar sind.