Search

Many fields of physics use quantum Monte Carlo techniques, but struggle to estimate dynamic spectra via the analytic continuation of imaginary-time quantum Monte Carlo data. One of the most ubiquitous approaches to analytic continuation is the maximum entropy method (MEM). We supply a dual Newton optimization algorithm to be used within the MEM and provide analytic bounds for the algorithm's error. The MEM is typically used with Bryan's controversial algorithm [Rothkopf, "Bryan's Maximum Entropy Method" Data 5.3 (2020)]. We present new theoretical issues that are not yet in the literature. Our algorithm has all the theoretical benefits of Bryan's algorithm without these theoretical issues. We compare the MEM with Bryan's optimization to the MEM with our dual Newton optimization on test problems from lattice quantum chromodynamics and plasma physics. These comparisons show that in the presence of noise the dual Newton algorithm produces better estimates and error bars; this indicates the limits of Bryan's algorithm's applicability. We use the MEM to investigate authentic quantum Monte Carlo data for the uniform electron gas at warm dense matter conditions and further substantiate the roton-type feature in the dispersion relation., Comment: 20 pages, 8 figures, 1 appendix

We establish the theoretical framework for implementing the maximumn entropy on the mean (MEM) method for linear inverse problems in the setting of approximate (data-driven) priors. We prove a.s. convergence for empirical means and further develop general estimates for the difference between the MEM solutions with different priors $\mu$ and $\nu$ based upon the epigraphical distance between their respective log-moment generating functions. These estimates allow us to establish a rate of convergence in expectation for empirical means. We illustrate our results with denoising on MNIST and Fashion-MNIST data sets., Comment: 25 pages, 13 figures

In this paper, we study Lipschitz continuity of the solution mappings of regularized least-squares problems for which the convex regularizers have (Fenchel) conjugates that are $\mathcal{C}^2$-cone reducible. Our approach, by using Robinson's strong regularity on the dual problem, allows us to obtain new characterizations of Lipschitz stability that rely solely on first-order information, thus bypassing the need to explore second-order information (curvature) of the regularizer. We show that these solution mappings are automatically Lipschitz continuous around the points in question whenever they are locally single-valued. We leverage our findings to obtain new characterizations of full stability and tilt stability for a broader class of convex additive-composite problems.

In this paper we prove the existence of H\"{o}lder continuous terminal embeddings of any desired $X \subseteq \mathbb{R}^d$ into $\mathbb{R}^{m}$ with $m=\mathcal{O}(\varepsilon^{-2}\omega(S_X)^2)$, for arbitrarily small distortion $\varepsilon$, where $\omega(S_X)$ denotes the Gaussian width of the unit secants of $X$. More specifically, when $X$ is a finite set we provide terminal embeddings that are locally $\frac{1}{2}$-H\"{o}lder almost everywhere, and when $X$ is infinite with positive reach we give terminal embeddings that are locally $\frac{1}{4}$-H\"{o}lder everywhere sufficiently close to $X$ (i.e., within all tubes around $X$ of radius less than $X$'s reach). When $X$ is a compact $d$-dimensional submanifold of $\mathbb{R}^N$, an application of our main results provides terminal embeddings into $\tilde{\mathcal{O}}(d)$-dimensional space that are locally H\"{o}lder everywhere sufficiently close to the manifold., Comment: 1 figure, 23 pages

Minimizing the difference of two submodular (DS) functions is a problem that naturally occurs in various machine learning problems. Although it is well known that a DS problem can be equivalently formulated as the minimization of the difference of two convex (DC) functions, existing algorithms do not fully exploit this connection. A classical algorithm for DC problems is called the DC algorithm (DCA). We introduce variants of DCA and its complete form (CDCA) that we apply to the DC program corresponding to DS minimization. We extend existing convergence properties of DCA, and connect them to convergence properties on the DS problem. Our results on DCA match the theoretical guarantees satisfied by existing DS algorithms, while providing a more complete characterization of convergence properties. In the case of CDCA, we obtain a stronger local minimality guarantee. Our numerical results show that our proposed algorithms outperform existing baselines on two applications: speech corpus selection and feature selection., Comment: Removed minor errors in Proposition 2.7, Theorem 4.3 and Corollary 4.4. Key results unchanged (see Erratum on p.4). Also fixed typos

This paper studies well-posedness and parameter sensitivity of the Square Root LASSO (SR-LASSO), an optimization model for recovering sparse solutions to linear inverse problems in finite dimension. An advantage of the SR-LASSO (e.g., over the standard LASSO) is that the optimal tuning of the regularization parameter is robust with respect to measurement noise. This paper provides three point-based regularity conditions at a solution of the SR-LASSO: the weak, intermediate, and strong assumptions. It is shown that the weak assumption implies uniqueness of the solution in question. The intermediate assumption yields a directionally differentiable and locally Lipschitz solution map (with explicit Lipschitz bounds), whereas the strong assumption gives continuous differentiability of said map around the point in question. Our analysis leads to new theoretical insights on the comparison between SR-LASSO and LASSO from the viewpoint of tuning parameter sensitivity: noise-robust optimal parameter choice for SR-LASSO comes at the "price" of elevated tuning parameter sensitivity. Numerical results support and showcase the theoretical findings.

We explore a method of statistical estimation called Maximum Entropy on the Mean (MEM) which is based on an information-driven criterion that quantifies the compliance of a given point with a reference prior probability measure. At the core of this approach lies the MEM function which is a partial minimization of the Kullback-Leibler divergence over a linear constraint. In many cases, it is known that this function admits a simpler representation (known as the Cram\'er rate function). Via the connection to exponential families of probability distributions, we study general conditions under which this representation holds. We then address how the associated MEM estimator gives rise to a wide class of MEM-based regularized linear models for solving inverse problems. Finally, we propose an algorithmic framework to solve these problems efficiently based on the Bregman proximal gradient method, alongside proximal operators for commonly used reference distributions. The article is complemented by a software package for experimentation and exploration of the MEM approach in applications.

In this paper we establish necessary and sufficient conditions for the existence of line segments (or flats) in the sphere of the nuclear norm via the notion of simultaneous polarization and a refined expression for the subdifferential of the nuclear norm. This is then leveraged to provide (point-based) necessary and sufficient conditions for uniqueness of solutions for minimizing the nuclear norm over an affine manifold. We further establish an alternative set of sufficient conditions for uniqueness, based on the interplay of the subdifferential of the nuclear norm and the range of the problem-defining linear operator. Finally, using convex duality, we show how to transfer the uniqueness results for the original problem to a whole class of nuclear norm-regularized minimization problems with a strictly convex fidelity term.

This paper provides a variational analysis of the unconstrained formulation of the LASSO problem, ubiquitous in statistical learning, signal processing, and inverse problems. In particular, we establish smoothness results for the optimal value as well as Lipschitz properties of the optimal solution as functions of the right-hand side (or measurement vector) and the regularization parameter. Moreover, we show how to apply the proposed variational analysis to study the sensitivity of the optimal solution to the tuning parameter in the context of compressed sensing with subgaussian measurements. Our theoretical findings are validated by numerical experiments.

We study the question as to when the closed convex hull of a K-convex map equals its K-epigraph. In particular, we shed light onto the smallest cone K such that a given map has convex and closed K-epigraph, respectively. We apply our findings to several examples in matrix space as well as to convex composite functions.

The projection onto the epigraph or a level set of a closed proper convex function can be achieved by finding a root of a scalar equation that involves the proximal operator as a function of the proximal parameter. This paper develops the variational analysis of this scalar equation. The approach is based on a study of the variational-analytic properties of general convex optimization problems that are (partial) infimal projections of the the sum of the function in question and the perspective map of a convex kernel. When the kernel is the Euclidean norm squared, the solution map corresponds to the proximal map, and thus the variational properties derived for the general case apply to the proximal case. Properties of the value function and the corresponding solution map -- including local Lipschitz continuity, directional differentiability, and semismoothness -- are derived. An SC$^1$ optimization framework for computing epigraphical and level-set projections is thus established. Numerical experiments on 1-norm projection illustrate the effectiveness of the approach as compared with specialized algorithms

Mathematical programs with vanishing constraints (MPVCs) are a class of nonlinear optimization problems with applications to various engineering problems such as truss topology design and robot motion planning. MPVCs are difficult problems from both a theoretical and numerical perspective: the combinatorial nature of the vanishing constraints often prevents standard constraint qualifications and optimality conditions from being attained; moreover, the feasible set is inherently nonconvex, and often has no interior around points of interest. In this paper, we therefore study and compare four regularization methods for the numerical solution of MPVCS. Each method depends on a single regularization parameter, which is used to embed the original MPVC into a sequence of standard nonlinear programs. Convergence results for these methods based on both exact and approximate stationary of the subproblems are established under weak assumptions. The improved regularity of the subproblems is studied by providing sufficient conditions for the existence of KKT multipliers. Numerical experiments, based on applications in truss topology design and an optimal control problem from aerothermodynamics, complement the theoretical analysis and comparison of the regularization methods. The computational results highlight the benefit of using regularization over applying a standard solver directly, and they allow us to identify two promising regularization schemes., Comment: 41 pages, 5 figures

Image deblurring is a notoriously challenging ill-posed inverse problem. In recent years, a wide variety of approaches have been proposed based upon regularization at the level of the image or on techniques from machine learning. We propose an alternative approach, shifting the paradigm towards regularization at the level of the probability distribution on the space of images. Our method is based upon the idea of maximum entropy on the mean wherein we work at the level of the probability density function of the image whose expectation is our estimate of the ground truth. Using techniques from convex analysis and probability theory, we show that the method is computationally feasible and amenable to very large blurs. Moreover, when images are imbedded with symbology (a known pattern), we show how our method can be applied to approximate the unknown blur kernel with remarkable effects. While our method is stable with respect to small amounts of noise, it does not actively denoise. However, for moderate to large amounts of noise, it performs well by preconditioned denoising with a state of the art method., Comment: 34 pages, 7 figures

Deep neural networks perform well on real world data but are prone to adversarial perturbations: small changes in the input easily lead to misclassification. In this work, we propose an attack methodology not only for cases where the perturbations are measured by $\ell_p$ norms, but in fact any adversarial dissimilarity metric with a closed proximal form. This includes, but is not limited to, $\ell_1, \ell_2$, and $\ell_\infty$ perturbations; the $\ell_0$ counting "norm" (i.e. true sparseness); and the total variation seminorm, which is a (non-$\ell_p$) convolutional dissimilarity measuring local pixel changes. Our approach is a natural extension of a recent adversarial attack method, and eliminates the differentiability requirement of the metric. We demonstrate our algorithm, ProxLogBarrier, on the MNIST, CIFAR10, and ImageNet-1k datasets. We consider undefended and defended models, and show that our algorithm easily transfers to various datasets. We observe that ProxLogBarrier outperforms a host of modern adversarial attacks specialized for the $\ell_0$ case. Moreover, by altering images in the total variation seminorm, we shed light on a new class of perturbations that exploit neighboring pixel information.

In this note we provide a full conjugacy and subdifferential calculus for convex convex-composite functions in finite-dimensional space. Our approach, based on infimal convolution and cone-convexity, is straightforward and yields the desired results under a verifiable Slater-type condition, with relaxed monotonicity and without lower semicontinuity assumptions on the functions in play. The versatility of our findings is illustrated by a series of applications in optimization and matrix analysis, including conic programming, matrix-fractional, variational Gram, and spectral functions., Comment: 30 pages

This paper is devoted to the study of the metric subregularity constraint qualification (MSCQ) for general optimization problems, with the emphasis on the nonconvex setting. We elaborate on notions of directional pseudo- and quasi-normality, recently introduced by Bai et al. (SIAM J. Opt., 2019), which combine the standard approach via pseudo- and quasi-normality with modern tools of directional variational analysis. We focus on applications to disjunctive programs, where (directional) pseudo-normality is characterized via an extremal condition. This, in turn, yields efficient tools to verify pseudo-normality and MSCQ, which include, but are not limited to, Robinson's result on polyhedral multifunctions and Gfrerer's second-order sufficient condition for metric subregularity. Finally, we refine our study by defining the new class of ortho-disjunctive programs which comprises prominent optimization problems such as mathematical programs with complementarity, vanishing or switching constraints., Comment: 27 pages

We show that many important convex matrix functions can be represented as the partial infimal projection of the generalized matrix fractional (GMF) and a relatively simple convex function. This representation provides conditions under which such functions are closed and proper as well as formulas for the ready computation of both their conjugates and subdifferentials. Special attention is given to support and indicator functions. Particular instances yield all weighted Ky Fan norms and squared gauges on $\mathbb R^{n\times m}$, and as an example we show that all variational Gram functions are representable as squares of gauges. Other instances yield weighted sums of the Frobenius and nuclear norms. The scope of applications is large and the range of variational properties and insight is fascinating and fundamental. An important byproduct of these representations is that they lay the foundation for a smoothing approach to many matrix functions on the interior of the domain of the GMF function, which opens the door to a range of unexplored optimization methods.