Search

A popular method to perform adversarial attacks on neuronal networks is the so-called fast gradient sign method and its iterative variant. In this paper, we interpret this method as an explicit Euler discretization of a differential inclusion, where we also show convergence of the discretization to the associated gradient flow. To do so, we consider the concept of p-curves of maximal slope in the case $p=\infty$. We prove existence of $\infty$-curves of maximum slope and derive an alternative characterization via differential inclusions. Furthermore, we also consider Wasserstein gradient flows for potential energies, where we show that curves in the Wasserstein space can be characterized by a representing measure on the space of curves in the underlying Banach space, which fulfill the differential inclusion. The application of our theory to the finite-dimensional setting is twofold: On the one hand, we show that a whole class of normalized gradient descent methods (in particular signed gradient descent) converge, up to subsequences, to the flow, when sending the step size to zero. On the other hand, in the distributional setting, we show that the inner optimization task of adversarial training objective can be characterized via $\infty$-curves of maximum slope on an appropriate optimal transport space.

We analyze a recently proposed algorithm for the problem of sampling from probability distributions $\mu^\ast$ in $\mathbb{R}^d$ with a Lebesgue density of the form $\mu^\ast(x) \propto \exp(-f(Kx)-g(x))$, where $K$ is a linear operator and $f,g$ convex and non-smooth. The algorithm is a generalization of the primal-dual hybrid gradient (PDHG) convex optimization algorithm to a sampling scheme. We analyze the method's continuous time limit, an SDE in the joint primal-dual variable. We give mild conditions under which the corresponding Fokker-Planck equation converges to a unique stationary state, which however does not concentrate in the dual variable and consequently does not have $\mu^\ast$ as its primal marginal. Under a smoothness assumption on $f$, we show that the scheme converges to the purely primal overdamped Langevin diffusion in the limit of small primal and large dual step sizes. We further prove that the target can never be the primal marginal of the invariant solution for any modification of the SDE with space-homogeneous diffusion coefficient. A correction with inhomogeneous diffusion coefficient and the correct invariant solution is proposed, but the scheme requires the same smoothness assumptions on $f$ and is numerically inferior to overdamped Langevin diffusion. We demonstrate our findings numerically, first on small-scale examples in which we can exactly verify the theoretical results, and subsequently on typical examples of larger scale from Bayesian imaging inverse problems.

As a generalization of graphs, hypergraphs are widely used to model higher-order relations in data. This paper explores the benefit of the hypergraph structure for the interpolation of point cloud data that contain no explicit structural information. We define the $\varepsilon_n$-ball hypergraph and the $k_n$-nearest neighbor hypergraph on a point cloud and study the $p$-Laplacian regularization on the hypergraphs. We prove the variational consistency between the hypergraph $p$-Laplacian regularization and the continuum $p$-Laplacian regularization in a semisupervised setting when the number of points $n$ goes to infinity while the number of labeled points remains fixed. A key improvement compared to the graph case is that the results rely on weaker assumptions on the upper bound of $\varepsilon_n$ and $k_n$. To solve the convex but non-differentiable large-scale optimization problem, we utilize the stochastic primal-dual hybrid gradient algorithm. Numerical experiments on data interpolation verify that the hypergraph $p$-Laplacian regularization outperforms the graph $p$-Laplacian regularization in preventing the development of spikes at the labeled points., Comment: 33 pages

This paper models gas networks as metric graphs, with isothermal Euler equations at the edges, Kirchhoff's law at interior vertices and time-(in)dependent boundary conditions at boundary vertices. For this setup, a generalized $p$-Wasserstein metric in a dynamic formulation is introduced and utilized to derive $p$-Wasserstein gradient flows, specifically focusing on the non-standard case $p = 3$., Comment: 43 pages, 5 figures, submitted to EJAM (Evolution Equations on Graphs: Analysis and Applications)

This paper studies the $p$-biharmonic equation on graphs, which arises in point cloud processing and can be interpreted as a natural extension of the graph $p$-Laplacian from the perspective of hypergraph. The asymptotic behavior of the solution is investigated when the random geometric graph is considered and the number of data points goes to infinity. We show that the continuum limit is an appropriately weighted $p$-biharmonic equation with homogeneous Neumann boundary conditions. The result relies on the uniform $L^p$ estimates for solutions and gradients of nonlocal and graph Poisson equations. The $L^\infty$ estimates of solutions are also obtained as a byproduct., Comment: 20 pages

We investigate an interacting particle model to simulate a foraging colony of ants, where each ant is represented as an active Brownian particle. The interactions among ants are mediated through chemotaxis, aligning their orientations with the upward gradient of the pheromone field. Unlike conventional models, our study introduces a parameter that enables the reproduction of two distinctive behaviors: the well-known Keller--Segel aggregation into spots and the formation of traveling clusters, without relying on external constraints such as food sources or nests. We consider the associated mean-field limit partial differential equation (PDE) of this system and establish the analytical and numerical foundations for understanding these particle behaviors. Remarkably, the mean-field PDE not only supports aggregation spots and lane formation but also unveils a bistable region where these two behaviors compete. The patterns associated with these phenomena are elucidated by the shape of the growing eigenfunctions derived from linear stability analysis. This study not only contributes to our understanding of complex ant colony dynamics but also introduces a novel parameter-dependent perspective on pattern formation in collective systems.

In this chapter we provide a theoretically founded investigation of state-of-the-art learning approaches for inverse problems from the point of view of spectral reconstruction operators. We give an extended definition of regularization methods and their convergence in terms of the underlying data distributions, which paves the way for future theoretical studies. Based on a simple spectral learning model previously introduced for supervised learning, we investigate some key properties of different learning paradigms for inverse problems, which can be formulated independently of specific architectures. In particular we investigate the regularization properties, bias, and critical dependence on training data distributions. Moreover, our framework allows to highlight and compare the specific behavior of the different paradigms in the infinite-dimensional limit.

This paper presents a method for finding a sparse representation of Barron functions. Specifically, given an $L^2$ function $f$, the inverse scale space flow is used to find a sparse measure $\mu$ minimising the $L^2$ loss between the Barron function associated to the measure $\mu$ and the function $f$. The convergence properties of this method are analysed in an ideal setting and in the cases of measurement noise and sampling bias. In an ideal setting the objective decreases strictly monotone in time to a minimizer with $\mathcal{O}(1/t)$, and in the case of measurement noise or sampling bias the optimum is achieved up to a multiplicative or additive constant. This convergence is preserved on discretization of the parameter space, and the minimizers on increasingly fine discretizations converge to the optimum on the full parameter space., Comment: 30 pages, 0 figures

We consider time-dependent inverse problems in a mathematical setting using Lebesgue-Bochner spaces. Such problems arise when one aims to recover parameters from given observations where the parameters or the data depend on time. There are various important applications being subject of current research that belong to this class of problems. Typically inverse problems are ill-posed in the sense that already small noise in the data causes tremendous errors in the solution. In this article we present two different concepts of ill-posedness: temporally (pointwise) ill-posedness and uniform ill-posedness with respect to the Lebesgue-Bochner setting. We investigate the two concepts by means of a typical setting consisting of a time-depending observation operator composed by a compact operator. Furthermore we develop regularization methods that are adapted to the respective class of ill-posedness., Comment: 21 pages, no figures

We prove the existence of solutions to a non-linear, non-local, degenerate equation which was previously derived as the formal hydrodynamic limit of an active Brownian particle system, where the particles are endowed with a position and an orientation. This equation incorporates diffusion in both the spatial and angular coordinates, as well as a non-linear non-local drift term, which depends on the angle-independent density. The spatial diffusion is non-linear degenerate and also comprises diffusion of the angle-independent density, which one may interpret as cross-diffusion with infinitely many species. Our proof relies on interpreting the equation as the perturbation of a gradient flow in a Wasserstein-type space. It generalizes the boundedness-by-entropy method to this setting and makes use of a gain of integrability due to the angular diffusion. For this latter step, we adapt a classical interpolation lemma for function spaces depending on time. We also prove uniqueness in the particular case where the non-local drift term is null, and provide existence and uniqueness results for stationary equilibrium solutions., Comment: 34 pages

This paper introduces gradient, adjoint, and $p$-Laplacian definitions for oriented hypergraphs as well as differential and averaging operators for unoriented hypergraphs. These definitions are used to define gradient flows in the form of diffusion equations with applications in modelling group dynamics and information flow in social networks as well as performing local and non-local image processing., Comment: 33 pages, 5 figures, submitted to Scale Space and Variational Methods, part of it published in International Conference on Scale Space and Variational Methods in Computer Vision proceedings

In this paper we investigate the use of Fourier Neural Operators (FNOs) for image classification in comparison to standard Convolutional Neural Networks (CNNs). Neural operators are a discretization-invariant generalization of neural networks to approximate operators between infinite dimensional function spaces. FNOs - which are neural operators with a specific parametrization - have been applied successfully in the context of parametric PDEs. We derive the FNO architecture as an example for continuous and Fr\'echet-differentiable neural operators on Lebesgue spaces. We further show how CNNs can be converted into FNOs and vice versa and propose an interpolation-equivariant adaptation of the architecture.

We study a variant of the dynamical optimal transport problem in which the energy to be minimised is modulated by the covariance matrix of the distribution. Such transport metrics arise naturally in mean-field limits of certain ensemble Kalman methods for solving inverse problems. We show that the transport problem splits into two coupled minimization problems: one for the evolution of mean and covariance of the interpolating curve and one for its shape. The latter consists in minimising the usual Wasserstein length under the constraint of maintaining fixed mean and covariance along the interpolation. We analyse the geometry induced by this modulated transport distance on the space of probabilities as well as the dynamics of the associated gradient flows. Those show better convergence properties in comparison to the classical Wasserstein metric in terms of exponential convergence rates independent of the Gaussian target. On the level of the gradient flows a similar splitting into the evolution of moments and shapes of the distribution can be observed., Comment: 84 pages, 4 figures. Comments are welcome

We investigate the convergence of solutions of a recently proposed diffuse interface/phase field model for cell blebbing by means of matched asymptotic expansions. It is a biological phenomenon that increasingly attracts attention by both experimental and theoretical communities. Key to understanding the process of cell blebbing mechanically are proteins that link the cell cortex and the cell membrane. Another important model component is the bending energy of the cell membrane and cell cortex which accounts for differential equations up to sixth order. Both aspects pose interesting mathematical challenges that will be addressed in this work like showing non-singularity formation for the pressure at boundary layers, deriving equations for asymptotic series coefficients of uncommonly high order, and dealing with a highly coupled system of equations., Comment: 32 pages, 1 figure

The reconstruction of images from their corresponding noisy Radon transform is a typical example of an ill-posed linear inverse problem as arising in the application of computerized tomography (CT). As the (naive) solution does not depend on the measured data continuously, regularization is needed to re-establish a continuous dependence. In this work, we investigate simple, but yet still provably convergent approaches to learning linear regularization methods from data. More specifically, we analyze two approaches: One generic linear regularization that learns how to manipulate the singular values of the linear operator in an extension of our previous work, and one tailored approach in the Fourier domain that is specific to CT-reconstruction. We prove that such approaches become convergent regularization methods as well as the fact that the reconstructions they provide are typically much smoother than the training data they were trained on. Finally, we compare the spectral as well as the Fourier-based approaches for CT-reconstruction numerically, discuss their advantages and disadvantages and investigate the effect of discretization errors at different resolutions.

We present an analysis of total-variation (TV) on non-Euclidean parameterized surfaces, a natural representation of the shapes used in 3D graphics. Our work explains recent experimental findings in shape spectral TV [Fumero et al., 2020] and adaptive anisotropic spectral TV [Biton and Gilboa, 2022]. A new way to generalize set convexity from the plane to surfaces is derived by characterizing the TV eigenfunctions on surfaces. Relationships between TV, area, eigenvalue, eigenfunctions and their discontinuities are discovered. Further, we expand the shape spectral TV toolkit to include versatile zero-homogeneous flows demonstrated through smoothing and exaggerating filters. Last but not least, we propose the first TV-based method for shape deformation, characterized by deformations along geometrical bottlenecks. We show these bottlenecks to be aligned with eigenfunction discontinuities. This research advances the field of spectral TV on surfaces and its application in 3D graphics, offering new perspectives for shape filtering and deformation., Comment: 19 pages, 20 figures

In this paper we investigate a generalisation of a Boltzmann mean field game (BMFG) for knowledge growth, originally introduced by the economists Lucas and Moll. In BMFG the evolution of the agent density with respect to their knowledge level is described by a Boltzmann equation. Agents increase their knowledge through binary interactions with others; their increase is modulated by the interaction and learning rate: Agents with similar knowledge learn more in encounters, while agents with very different levels benefit less from learning interactions. The optimal fraction of time spent on learning is calculated by a Bellman equation, resulting in a highly nonlinear forward-backward in time PDE system. The structure of solutions to the Boltzmann and Bellman equation depends strongly on the learning rate in the Boltzmann collision kernel as well as the utility function in the Bellman equation. In this paper we investigate the monotonicity behavior of solutions for different learning and utility functions, show existence of solutions and investigate how they impact the existence of so-called balanced growth path solutions, that relate to exponential growth of the overall economy. Furthermore we corroborate and illustrate our analytical results with computational experiments.

In this paper we provide a novel approach to the analysis of kinetic models for label switching, which are used for particle systems that can randomly switch between gradient flows in different energy landscapes. Besides problems in biology and physics, we also demonstrate that stochastic gradient descent, the most popular technique in machine learning, can be understood in this setting, when considering a time-continuous variant. Our analysis is focusing on the case of evolution in a collection of external potentials, for which we provide analytical and numerical results about the evolution as well as the stationary problem.

This paper studies the derivation of the quadratic porous medium equation and a class of cross-diffusion systems from nonlocal interactions. We prove convergence of solutions of a nonlocal interaction equation, resp. system, to solutions of the quadratic porous medium equation, resp. cross-diffusion system, in the limit of a localising interaction kernel. The analysis is carried out at the level of the (nonlocal) partial differential equations and we use gradient flow techniques to derive bounds on energy, second order moments, and logarithmic entropy. The dissipation of the latter yields sufficient regularity to obtain compactness results and pass to the limit in the localised convolutions. The strategy we propose relies on a discretisation scheme, which can be slightly modified in order to extend our result to PDEs without gradient flow structure. In particular, it does not require convexity of the associated energies. Our analysis allows to treat the case of limiting weak solutions of the non-viscous porous medium equation at relevant low regularity, assuming the initial value to have finite energy and entropy.

This paper discusses basic results and recent developments on variational regularization methods, as developed for inverse problems. In a typical setup we review basic properties needed to obtain a convergent regularization scheme and further discuss the derivation of quantitative estimates respectively needed ingredients such as Bregman distances for convex functionals. In addition to the approach developed for inverse problems we will also discuss variational regularization in machine learning and work out some connections to the classical regularization theory. In particular we will discuss a reinterpretation of machine learning problems in the framework of regularization theory and a reinterpretation of variational methods for inverse problems in the framework of risk minimization. Moreover, we establish some previously unknown connections between error estimates in Bregman distances and generalization errors.

We propose a general strategy for solving nonlinear integro-differential evolution problems with periodic boundary conditions, where no direct maximum/minimum principle is available. This is motivated by the study of recent macroscopic models for active Brownian particles with repulsive interactions, consisting of advection-diffusion processes in the space of particle position and orientation. We focus on one of such models, namely a semilinear parabolic equation with a nonlinear active drift term, whereby the velocity depends on the particle orientation and angle-independent overall particle density (leading to a nonlocal term by integrating out the angular variable). The main idea of the existence analysis is to exploit a-priori estimates from (approximate) entropy dissipation. The global existence and uniqueness of weak solutions is shown using a two-step Galerkin approximation with appropriate cutoff in order to obtain nonnegativity, an upper bound on the overall density and preserve a-priori estimates. Our anyalysis naturally includes the case of finite systems, corresponding to the case of a finite number of directions. The Duhamel principle is then used to obtain additional regularity of the solution, namely continuity in time-space. Motivated by the class of initial data relevant for the application, which includes perfectly aligned particles (same orientation), we extend the well-posedness result to very weak solutions allowing distributional initial data with low regularity., Comment: 35 pages

Variational regularization has remained one of the most successful approaches for reconstruction in imaging inverse problems for several decades. With the emergence and astonishing success of deep learning in recent years, a considerable amount of research has gone into data-driven modeling of the regularizer in the variational setting. Our work extends a recently proposed method, referred to as adversarial convex regularization (ACR), that seeks to learn data-driven convex regularizers via adversarial training in an attempt to combine the power of data with the classical convex regularization theory. Specifically, we leverage the variational source condition (SC) during training to enforce that the ground-truth images minimize the variational loss corresponding to the learned convex regularizer. This is achieved by adding an appropriate penalty term to the ACR training objective. The resulting regularizer (abbreviated as ACR-SC) performs on par with the ACR, but unlike ACR, comes with a quantitative convergence rate estimate., Comment: Accepted to the NeurIPS-2021 Workshop on Deep Learning and Inverse Problems

We revisit a classical continuum model for the diffusion of multiple species with size-exclusion constraint, which leads to a degenerate nonlinear cross-diffusion system. The purpose of this article is twofold: first, it aims at a systematic study of the question of existence of weak solutions and their long-time asymptotic behaviour. Second, it provides a weak-strong stability estimate for a wide range of coefficients, which had been missing so far. In order to achieve the results mentioned above, we exploit the formal gradient-flow structure of the model with respect to a logarithmic entropy, which leads to best estimates in the full-interaction case, where all cross-diffusion coefficients are non-zero. Those are crucial to obtain the minimal Sobolev regularity needed for a weak-strong stability result. For meaningful cases when some of the coefficients vanish, we provide a novel existence result based on approximation by the full-interaction case.

Goal oriented autonomous operation of space rovers has been known to increase scientific output of a mission. In this work we present an algorithm, called the RoI Prioritised Sampling (RPS), that prioritises Region-of-Interests (RoIs) in an exploration scenario in order to utilise the limited resources of the imaging instrument on the rover effectively. This prioritisation is based on an estimator that evaluates the change in information content at consecutive spatial scales of the RoIs without calculating the finer scale reconstruction. The estimator, called the Refinement Indicator (RI), is motivated and derived. Multi-scale acquisition approaches, based on classical and multilevel compressed sensing, with respect to the single pixel camera architecture are discussed. The performance of the algorithm is verified on remote sensing images and compared with the state-of-the-art multi-resolution reconstruction algorithms. At the considered sub-sampling rates the RPS is shown to better utilise the system resources for reconstructing the RoIs.

The aim of this paper is to develop suitable models for the phenomenon of cell blebbing, which allow for computational predictions of mechanical effects including the crucial interaction of the cell membrane and the actin cortex. For this sake we resort to a two phase-field model that uses diffuse descriptions of both the membrane and the cortex, which in particular allows for a suitable description of the interaction via linker protein densities. Besides the detailed modelling we discuss some energetic aspects of the models and present a numerical scheme, which allows to carry out several computational studies. In those we demonstrate that several effects found in experiments can be reproduced, in particular bleb formation by cortex rupture, which was not possible by previous models without the linker dynamics.

This chapter focuses on the mathematical modelling of active particles (or agents) in crowded environments. We discuss several microscopic models found in literature and the derivation of the respective macroscopic partial differential equations for the particle density. The macroscopic models share common features, such as cross diffusion or degenerate mobilities. We then take the diversity of macroscopic models to a uniform structure and work out potential similarities and differences. Moreover, we discuss boundary effects and possible applications in life and social sciences. This is complemented by numerical simulations that highlight the effects of different boundary conditions.

We propose a novel strategy for Neural Architecture Search (NAS) based on Bregman iterations. Starting from a sparse neural network our gradient-based one-shot algorithm gradually adds relevant parameters in an inverse scale space manner. This allows the network to choose the best architecture in the search space which makes it well-designed for a given task, e.g., by adding neurons or skip connections. We demonstrate that using our approach one can unveil, for instance, residual autoencoders for denoising, deblurring, and classification tasks. Code is available at https://github.com/TimRoith/BregmanLearning.

This chapter describes how gradient flows and nonlinear power methods in Banach spaces can be used to solve nonlinear eigenvector-dependent eigenvalue problems, and how convergence of (discretized) approximations can be verified. We review several flows from literature, which were proposed to compute nonlinear eigenfunctions, and show that they all relate to normalized gradient flows. Furthermore, we show that the implicit Euler discretization of gradient flows gives rise to a nonlinear power method of the proximal operator and prove their convergence to nonlinear eigenfunctions. Finally, we prove that $\Gamma$-convergence of functionals implies convergence of their ground states, which is important for discrete approximations., Comment: To appear in Handbook of Numerical Analysis, Numerical Control: Part A, Volume 23

We propose a learning framework based on stochastic Bregman iterations, also known as mirror descent, to train sparse neural networks with an inverse scale space approach. We derive a baseline algorithm called LinBreg, an accelerated version using momentum, and AdaBreg, which is a Bregmanized generalization of the Adam algorithm. In contrast to established methods for sparse training the proposed family of algorithms constitutes a regrowth strategy for neural networks that is solely optimization-based without additional heuristics. Our Bregman learning framework starts the training with very few initial parameters, successively adding only significant ones to obtain a sparse and expressive network. The proposed approach is extremely easy and efficient, yet supported by the rich mathematical theory of inverse scale space methods. We derive a statistically profound sparse parameter initialization strategy and provide a rigorous stochastic convergence analysis of the loss decay and additional convergence proofs in the convex regime. Using only 3.4% of the parameters of ResNet-18 we achieve 90.2% test accuracy on CIFAR-10, compared to 93.6% using the dense network. Our algorithm also unveils an autoencoder architecture for a denoising task. The proposed framework also has a huge potential for integrating sparse backpropagation and resource-friendly training., Comment: 43 pages, 5 figures, some minor modifications, weakened assumptions

The aim of this paper is to derive macroscopic equations for processes on large co-evolving networks, examples being opinion polarization with the emergence of filter bubbles or other social processes such as norm development. This leads to processes on graphs (or networks), where both the states of particles in nodes as well as the weights between them are updated in time. In our derivation we follow the basic paradigm of statistical mechanics: We start from paradigmatic microscopic models and derive a Liouville-type equation in a high-dimensional space including not only the node states in the network (corresponding to positions in mechanics), but also the edge weights between them. We then derive a natural (finite) marginal hierarchy and pass to an infinite limit. We will discuss the closure problem for this hierarchy and see that a simple mean-field solution can only arise if the weight distributions between nodes of equal states are concentrated. In a more interesting general case we propose a suitable closure at the level of a two-particle distribution (including the weight between them) and discuss some properties of the arising kinetic equations. Moreover, we highlight some structure-preserving properties of this closure and discuss its analysis in a minimal model. We discuss the application of our theory to some agent-based models in literature and discuss some open mathematical issues.

The susceptibility of deep neural networks to untrustworthy predictions, including out-of-distribution (OOD) data and adversarial examples, still prevent their widespread use in safety-critical applications. Most existing methods either require a re-training of a given model to achieve robust identification of adversarial attacks or are limited to out-of-distribution sample detection only. In this work, we propose a geometric gradient analysis (GGA) to improve the identification of untrustworthy predictions without retraining of a given model. GGA analyzes the geometry of the loss landscape of neural networks based on the saliency maps of their respective input. To motivate the proposed approach, we provide theoretical connections between gradients' geometrical properties and local minima of the loss function. Furthermore, we demonstrate that the proposed method outperforms prior approaches in detecting OOD data and adversarial attacks, including state-of-the-art and adaptive attacks.

We consider a statistical inverse learning problem, where the task is to estimate a function $f$ based on noisy point evaluations of $Af$, where $A$ is a linear operator. The function $Af$ is evaluated at i.i.d. random design points $u_n$, $n=1,...,N$ generated by an unknown general probability distribution. We consider Tikhonov regularization with general convex and $p$-homogeneous penalty functionals and derive concentration rates of the regularized solution to the ground truth measured in the symmetric Bregman distance induced by the penalty functional. We derive concrete rates for Besov norm penalties and numerically demonstrate the correspondence with the observed rates in the context of X-ray tomography., Comment: 35 pages, 4 figures

We study a dynamic optimal transport problem on a network. Despite the cost for transport along the edges, an additional cost, scaled with a parameter $\kappa$, has to be paid for interchanging mass between edges and vertices. We show existence f minimisers using duality and discuss the relationship of the model to other metrics such as Fisher-Rao and the classical Wasserstein metric. Finally, we examine the limiting behaviour of the model in terms of the parameter $\kappa$.

The vulnerability of deep neural networks to small and even imperceptible perturbations has become a central topic in deep learning research. Although several sophisticated defense mechanisms have been introduced, most were later shown to be ineffective. However, a reliable evaluation of model robustness is mandatory for deployment in safety-critical scenarios. To overcome this problem we propose a simple yet effective modification to the gradient calculation of state-of-the-art first-order adversarial attacks. Normally, the gradient update of an attack is directly calculated for the given data point. This approach is sensitive to noise and small local optima of the loss function. Inspired by gradient sampling techniques from non-convex optimization, we propose Dynamically Sampled Nonlocal Gradient Descent (DSNGD). DSNGD calculates the gradient direction of the adversarial attack as the weighted average over past gradients of the optimization history. Moreover, distribution hyperparameters that define the sampling operation are automatically learned during the optimization scheme. We empirically show that by incorporating this nonlocal gradient information, we are able to give a more accurate estimation of the global descent direction on noisy and non-convex loss surfaces. In addition, we show that DSNGD-based attacks are on average 35% faster while achieving 0.9% to 27.1% higher success rates compared to their gradient descent-based counterparts.

We propose a mean-field optimal control problem for the parameter identification of a given pattern. The cost functional is based on the Wasserstein distance between the probability measures of the modeled and the desired patterns. The first-order optimality conditions corresponding to the optimal control problem are derived using a Lagrangian approach on the mean-field level. Based on these conditions we propose a gradient descent method to identify relevant parameters such as angle of rotation and force scaling which may be spatially inhomogeneous. We discretize the first-order optimality conditions in order to employ the algorithm on the particle level. Moreover, we prove a rate for the convergence of the controls as the number of particles used for the discretization tends to infinity. Numerical results for the spatially homogeneous case demonstrate the feasibility of the approach.

The aim of this paper is to investigate superresolution in deconvolution driven by sparsity priors. The observed signal is a convolution of an original signal with a continuous kernel.With the prior knowledge that the original signal can be considered as a sparse combination of Dirac delta peaks, we seek to estimate the positions and amplitudes of these peaks by solving a finite dimensional convex problem on a computational grid. Because, the support of the original signal may or may not be on this grid, by studying the discrete deconvolution of sparse peaks using L1-norm sparsity prior, we confirm recent observations that canonically the discrete reconstructions will result in multiple peaks at grid points adjacent to the location of the true peak. Owning to the complexity of this problem, we analyse carefully the de-convolution of single peaks on a grid and gain a strong insight about the dependence of the reconstructed magnitudes on the exact peak location. This in turn allows us to infer further information on recovering the location of the exact peaks i.e. to perform super-resolution. We analyze in detail the possible cases that can appear and based on our theoretical findings, we propose an self-driven adaptive grid approach that allows to perform superresolution in one-dimensional and multi-dimensional spaces. With the view that the current study can provide a further step in the development of more robust algorithms for the detection of single molecules in fluorescence microscopy or identification of characteristic frequencies in spectral analysis, we demonstrate how the proposed approach can recover sparse signals using simulated clusters of point sources (peaks) of low-resolution in one and two-dimensional spaces., Comment: It can help to improve super-resolution algorithms that use sparsity constraints

The aim of this paper is to study the derivation of appropriate meso- and macroscopic models for interactions as appearing in social processes. There are two main characteristics the models take into account, namely a network structure of interactions, which we treat by an appropriate mesoscopic description, and a different role of interacting agents. The latter differs from interactions treated in classical statistical mechanics in the sense that the agents do not have symmetric roles, but there is rather an active and a passive agent. We will demonstrate how a certain form of kinetic equations can be obtained to describe such interactions at a mesoscopic level and moreover obtain macroscopic models from monokinetics solutions of those. The derivation naturally leads to systems of nonlocal reaction-diffusion equations (or in a suitable limit local versions thereof), which can explain spatial phase separation phenomena found to emerge from the microscopic interactions. We will highlight the approach in three examples, namely the evolution and coarsening of dialects in human language, the construction of social norms, and the spread of an epidemic.

This work applies Bayesian experimental design to selecting optimal projection geometries in (discretized) parallel beam X-ray tomography assuming the prior and the additive noise are Gaussian. The introduced greedy exhaustive optimization algorithm proceeds sequentially, with the posterior distribution corresponding to the previous projections serving as the prior for determining the design parameters, i.e. the imaging angle and the lateral position of the source-receiver pair, for the next one. The algorithm allows redefining the region of interest after each projection as well as adapting parameters in the (original) prior to the measured data. Both A and D-optimality are considered, with emphasis on efficient evaluation of the corresponding objective functions. Two-dimensional numerical experiments demonstrate the functionality of the approach., Comment: 23 pages, 8 figures

We study variational regularisation methods for inverse problems with imperfect forward operators whose errors can be modelled by order intervals in a partial order of a Banach lattice. We carry out analysis with respect to existence and convex duality for general data fidelity terms and regularisation functionals. Both for a-priori and a-posteriori parameter choice rules, we obtain convergence rates of the regularized solutions in terms of Bregman distances. Our results apply to fidelity terms such as Wasserstein distances, f-divergences, norms, as well as sums and infimal convolutions of those.

The aim of this paper is to analyze a model for chemotaxis based on a local sensing mechanism instead of the gradient sensing mechanism used in the celebrated minimal Keller-Segel model. The model we study has the same entropy as the minimal Keller-Segel model, but a different dynamics to minimize this entropy. Consequently, the conditions on the mass for the existence of stationary solutions or blow-up are the same, however we make the interesting observation that with the local sensing mechanism the blow-up in the case of supercritical mass is delayed to infinite time. Our observation is made rigorous from a mathematical point via a proof of global existence of weak solutions for arbitrary large masses and space dimension. The key difference of our model to the minimal Keller-Segel model is that the structure of the equation allows for a duality estimate that implies a bound on the $(H^1)'$-norm of the solutions, which can only grow with a square-root law in time. This additional $(H^1)'$-bound implies a lower bound on the entropy, which contrasts markedly with the minimal Keller-Segel model for which it is unbounded from below in the supercritical case. Besides, regularity and uniqueness of solutions are also studied.

The propagation of gradient flow structures from microscopic to macroscopic models is a topic of high current interest. In this paper we discuss this propagation in a model for the diffusion of particles interacting via hard-core exclusion or short-range repulsive potentials. We formulate the microscopic model as a high-dimensional gradient flow in the Wasserstein metric for an appropriate free-energy functional. Then we use the JKO approach to identify the asymptotics of the metric and the free-energy functional beyond the lowest order for single particle densities in the limit of small particle volumes by matched asymptotic expansions. While we use a propagation of chaos assumption at far distances, we consider correlations at small distance in the expansion. In this way we obtain a clear picture of the emergence of a macroscopic gradient structure incorporating corrections in the free energy functional due to the volume exclusion.

In this work we analyse the functional ${\cal J}(u)=\|\nabla u\|_\infty$ defined on Lipschitz functions with homogeneous Dirichlet boundary conditions. Our analysis is performed directly on the functional without the need to approximate with smooth $p$-norms. We prove that its ground states coincide with multiples of the distance function to the boundary of the domain. Furthermore, we compute the $L^2$-subdifferential of ${\cal J}$ and characterize the distance function as unique non-negative eigenfunction of the subdifferential operator. We also study properties of general eigenfunctions, in particular their nodal sets. Furthermore, we prove that the distance function can be computed as asymptotic profile of the gradient flow of ${\cal J}$ and construct analytic solutions of fast marching type. In addition, we give a geometric characterization of the extreme points of the unit ball of ${\cal J}$. Finally, we transfer many of these results to a discrete version of the functional defined on a finite weighted graph. Here, we analyze properties of distance functions on graphs and their gradients. The main difference between the continuum and discrete setting is that the distance function is not the unique non-negative eigenfunction on a graph., Comment: Accepted for publication at Pure and Applied Analysis

We introduce a first order Total Variation type regulariser that decomposes a function into a part with a given Lipschitz constant (which is also allowed to vary spatially) and a jump part. The kernel of this regulariser contains all functions whose Lipschitz constant does not exceed a given value, hence by locally adjusting this value one can determine how much variation is the reconstruction allowed to have. We prove regularising properties of this functional, study its connections to other Total Variation type regularisers and propose a primal dual optimisation scheme. Our numerical experiments demonstrate that the proposed first order regulariser can achieve reconstruction quality similar to that of second order regularisers such as Total Generalised Variation, while requiring significantly less computational time.

This work is concerned with the gradient flow of absolutely $p$-homogeneous convex functionals on a Hilbert space, which we show to exhibit finite ($p<2$) or infinite extinction time ($p \geq 2$). We give upper bounds for the finite extinction time and establish convergence rates of the flow. Moreover, we study next order asymptotics and prove that asymptotic profiles of the solution are eigenfunctions of the subdifferential operator of the functional. To this end, we compare with solutions of an ordinary differential equation which describes the evolution of eigenfunction under the flow. Our work applies, for instance, to local and nonlocal versions of PDEs like $p$-Laplacian evolution equations, the porous medium equation, and fast diffusion equations, herewith generalizing many results from the literature to an abstract setting. We also demonstrate how our theory extends to general homogeneous evolution equations which are not necessarily a gradient flow. Here we discover an interesting integrability condition which characterizes whether or not asymptotic profiles are eigenfunctions., Comment: Version as published in Journal of Evolution Equations

The aim of this paper is to investigate the use of an entropic projection method for the iterative regularization of linear ill-posed problems. We derive a closed form solution for the iterates and analyze their convergence behaviour both in a case of reconstructing general nonnegative unknowns as well as for the sake of recovering probability distributions. Moreover, we discuss several variants of the algorithm and relations to other methods in the literature. The effectiveness of the approach is studied numerically in several examples.

We consider the problem of estimating the density of buyers and vendors in a nonlinear parabolic price formation model using measurements of the price and the transaction rate. Our approach is based on a work by Puel et al., see \cite{Puel2002}, and results in a optimal control problem. We analyse this problems and provide stability estimates for the controls as well as the unknown density in the presence of measurement errors. Our analytic findings are supported with numerical experiments.

The aim of this paper is to further develop mathematical models for bleb formation in cells, including cell-membrane interactions with linker proteins. This leads to nonlinear reaction-diffusion equations on a surface coupled to fluid dynamics in the bulk. We provide a detailed mathematical analysis and investigate some singular limits of the model, connecting it to previous literature. Moreover, we provide numerical simulations in different scenarios, confirming that the model can reproduce experimental results on bleb initation.

Controlling large particle systems in collective dynamics by a few agents is a subject of high practical importance, e.g., in evacuation dynamics. In this paper we study an instantaneous control approach to steer an interacting particle system into a certain spatial region by repulsive forces from a few external agents, which might be interpreted as shepherd dogs leading sheep to their home. We introduce an appropriate mathematical model and the corresponding optimization problem. In particular, we are interested in the interaction of numerous particles, which can be approximated by a mean-field equation. Due to the high-dimensional phase space this will require a tailored optimization strategy. The arising control problems are solved using adjoint information to compute the descent directions. Numerical results on the microscopic and the macroscopic level indicate the convergence of optimal controls and optimal states in the mean-field limit,i.e., for an increasing number of particles., Comment: arXiv admin note: substantial text overlap with arXiv:1610.01325

The aim of this paper is to discuss some advanced aspects of image reconstruction in single-pixel cameras, focusing in particular on detectors in the THz regime. We discuss the reconstruction problem from a computational imaging perspective and provide a comparison of the effects of several state-of-the art regularization techniques. Moreover, we focus on some advanced aspects arising in practice with THz cameras, which lead to nonlinear reconstruction problems: the calibration of the beam reminiscent of the Retinex problem in imaging and phase recovery problems. Finally we provide an outlook to future challenges in the area.

We introduce a new regularizer in the total variation family that promotes reconstructions with a given Lipschitz constant (which can also vary spatially). We prove regularizing properties of this functional and investigate its connections to total variation and infimal convolution type regularizers TVLp and, in particular, establish topological equivalence. Our numerical experiments show that the proposed regularizer can achieve similar performance as total generalized variation while having the advantage of a very intuitive interpretation of its free parameter, which is just a local estimate of the norm of the gradient. It also provides a natural approach to spatially adaptive regularization., Comment: 12 pages, 4 figures, accepted for publication in SSVM conference proceedings 2019