Search

Convex sets arising in a variety of applications are well-defined for every relevant dimension. Examples include the simplex and the spectraplex that correspond to probability distributions and to quantum states; combinatorial polytopes and their relaxations such as the cut polytope and the elliptope in integer programming; and unit balls of regularizers such as the $\ell_p$ and Schatten norms in inverse problems. Moreover, these sets are often specified using conic descriptions that can be obviously instantiated in any dimension. We develop a systematic framework to study such free descriptions of convex sets. We show that free descriptions arise from a recently-identified phenomenon in algebraic topology called representation stability, which relates invariants across dimensions in a sequence of group representations. Our framework yields structural results for free descriptions pertaining to the relations between the sets they describe across dimensions, extendability of a single set in a given dimension to a freely-described sequence, and continuous limits of such sequences. We also develop a procedure to obtain parametric families of freely-described convex sets whose structure is adapted to a given application; illustrations are provided via examples that arise in the literature as well as new families that are derived using our procedure. We demonstrate the utility of our framework in two contexts. First, we develop an algorithm for a free analog of the convex regression problem, where a convex function is fit to input-output data; by searching over our parametric families, we can fit a function to low-dimensional inputs and extend it to any other dimension. Second, we prove that many sequences of symmetric conic programs can be solved in constant time, which unifies and strengthens several results in the literature., Comment: Substantial re-organization of the paper and strengthening of several results

Traditional supervised learning aims to learn an unknown mapping by fitting a function to a set of input-output pairs with a fixed dimension. The fitted function is then defined on inputs of the same dimension. However, in many settings, the unknown mapping takes inputs in any dimension; examples include graph parameters defined on graphs of any size and physics quantities defined on an arbitrary number of particles. We leverage a newly-discovered phenomenon in algebraic topology, called representation stability, to define equivariant neural networks that can be trained with data in a fixed dimension and then extended to accept inputs in any dimension. Our approach is user-friendly, requiring only the network architecture and the groups for equivariance, and can be combined with any training procedure. We provide a simple open-source implementation of our methods and offer preliminary numerical experiments., Comment: 21 pages, 2 figures

We consider the problem of provably finding a stationary point of a smooth function to be minimized on the variety of bounded-rank matrices. This turns out to be unexpectedly delicate. We trace the difficulty back to a geometric obstacle: On a nonsmooth set, there may be sequences of points along which standard measures of stationarity tend to zero, but whose limit points are not stationary. We name such events apocalypses, as they can cause optimization algorithms to converge to non-stationary points. We illustrate this explicitly for an existing optimization algorithm on bounded-rank matrices. To provably find stationary points, we modify a trust-region method on a standard smooth parameterization of the variety. The method relies on the known fact that second-order stationary points on the parameter space map to stationary points on the variety. Our geometric observations and proposed algorithm generalize beyond bounded-rank matrices. We give a geometric characterization of apocalypses on general constraint sets, which implies that Clarke-regular sets do not admit apocalypses. Such sets include smooth manifolds, manifolds with boundaries, and convex sets. Our trust-region method supports parameterization by any complete Riemannian manifold., Comment: The arXiv version contains a few details omitted in the published version Math. Program. (2022)

The properties of gradient techniques for the phase retrieval problem have received a considerable attention in recent years. In almost all applications, however, the phase retrieval problem is solved using a family of algorithms that can be interpreted as variants of Douglas-Rachford splitting. In this work, we establish a connection between Douglas-Rachford and gradient algorithms. Specifically, we show that in some cases a generalization of Douglas-Rachford, called relaxed-reflect-reflect (RRR), can be viewed as gradient descent on a certain objective function. The solutions coincide with the critical points of that objective, which---in contrast to standard gradient techniques---are not its minimizers. Using the objective function, we give simple proofs of some basic properties of the RRR algorithm. Specifically, we describe its set of solutions, show a local convexity around any solution, and derive stability guarantees. Nevertheless, in its present state, the analysis does not elucidate the remarkable empirical performance of RRR and its global properties.

We consider the multi-target detection problem of recovering a set of signals that appear multiple times at unknown locations in a noisy measurement. In the low noise regime, one can estimate the signals by first detecting occurrences, then clustering and averaging them. In the high noise regime however, neither detection nor clustering can be performed reliably, so that strategies along these lines are destined to fail. Notwithstanding, using autocorrelation analysis, we show that the impossibility to detect and cluster signal occurrences in the presence of high noise does not necessarily preclude signal estimation. Specifically, to estimate the signals, we derive simple relations between the autocorrelations of the observation and those of the signals. These autocorrelations can be estimated accurately at any noise level given a sufficiently long measurement. To recover the signals from the observed autocorrelations, we solve a set of polynomial equations through nonlinear least-squares. We provide analysis regarding well-posedness of the task, and demonstrate numerically the effectiveness of the method in a variety of settings. The main goal of this work is to provide theoretical and numerical support for a recently proposed framework to image 3-D structures of biological macromolecules using cryo-electron microscopy in extreme noise levels., Comment: arXiv admin note: text overlap with arXiv:1810.00226

Single-particle cryo-electron microscopy (cryo-EM) has recently joined X-ray crystallography and NMR spectroscopy as a high-resolution structural method to resolve biological macromolecules. In a cryo-EM experiment, the microscope produces images called micrographs. Projections of the molecule of interest are embedded in the micrographs at unknown locations, and under unknown viewing directions. Standard imaging techniques first locate these projections (detection) and then reconstruct the 3-D structure from them. Unfortunately, high noise levels hinder detection. When reliable detection is rendered impossible, the standard techniques fail. This is a problem, especially for small molecules. In this paper, we pursue a radically different approach: we contend that the structure could, in principle, be reconstructed directly from the micrographs, without intermediate detection. The aim is to bring small molecules within reach for cryo-EM. To this end, we design an autocorrelation analysis technique that allows to go directly from the micrographs to the sought structures. This involves only one pass over the micrographs, allowing online, streaming processing for large experiments. We show numerical results and discuss challenges that lay ahead to turn this proof-of-concept into a complementary approach to state-of-the-art algorithms., Comment: Older citations to this paper refer to version arXiv:1810.00226v1, parts of which now appear in: Tamir Bendory, Nicolas Boumal, William Leeb, Eitan Levin, and Amit Singer. "Multi-target detection with application to cryo-electron microscopy." Inverse Problems 35, no. 10 (2019): 104003

Accurate determination of microscopic transport and magnetization currents is of central importance for the study of the electric properties of low dimensional materials and interfaces, of superconducting thin films and of electronic devices. Current distribution is usually derived from the measurement of the perpendicular component of the magnetic field above the surface of the sample, followed by numerical inversion of the Biot-Savart law. The inversion is commonly obtained by deriving the current stream function $g$, which is then differentiated in order to obtain the current distribution. However, this two-step procedure requires filtering at each step and, as a result, oversmoothes the solution. To avoid this oversmoothing we develop a direct procedure for inversion of the magnetic field that avoids use of the stream function. This approach provides enhanced accuracy of current reconstruction over a wide range of noise levels. We further introduce a reflection procedure that allows for the reconstruction of currents that cross the boundaries of the measurement window. The effectiveness of our approach is demonstrated by several numerical examples.

Single-Particle Reconstruction (SPR) in Cryo-Electron Microscopy (cryo-EM) is the task of estimating the 3D structure of a molecule from a set of noisy 2D projections, taken from unknown viewing directions. Many algorithms for SPR start from an initial reference molecule, and alternate between refining the estimated viewing angles given the molecule, and refining the molecule given the viewing angles. This scheme is called iterative refinement. Reliance on an initial, user-chosen reference introduces model bias, and poor initialization can lead to slow convergence. Furthermore, since no ground truth is available for an unsolved molecule, it is difficult to validate the obtained results. This creates the need for high quality ab initio models that can be quickly obtained from experimental data with minimal priors, and which can also be used for validation. We propose a procedure to obtain such an ab initio model directly from raw data using Kam's autocorrelation method. Kam's method has been known since 1980, but it leads to an underdetermined system, with missing orthogonal matrices. Until now, this system has been solved only for special cases, such as highly symmetric molecules or molecules for which a homologous structure was already available. In this paper, we show that knowledge of just two clean projections is sufficient to guarantee a unique solution to the system. This system is solved by an optimization-based heuristic. For the first time, we are then able to obtain a low-resolution ab initio model of an asymmetric molecule directly from raw data, without 2D class averaging and without tilting. Numerical results are presented on both synthetic and experimental data.

We propose a new stopping criterion for Krylov subspace iterative regularization of large-scale ill-posed inverse problems. Our stopping criterion accurately filters the data using a generalization of the Picard parameter that was originally introduced for direct regularization of small-scale problems. In the one dimension we filter the data in the discrete Fourier transform (DFT) basis using the Picard parameter, which separates noise-dominated Fourier coefficients from the signal-dominated ones. For two-dimensional problems we propose a novel vectorization scheme of the Fourier coefficients of the data based on the Kronecker product structure of the two-dimensional DFT matrix, which effectively reduces the problem to one dimension. At each iteration we compute the distance between the data reconstructed from the iterated solution and the filtered data, terminating the iterations once this distance begins to increase or to level off. The accuracy and robustness of the proposed method is demonstrated by several numerical examples and a MATLAB-based implementation is provided.

Accurate determination of the regularization parameter in inverse problems still represents an analytical challenge, owing mainly to the considerable difficulty to separate the unknown noise from the signal. We present a new approach for determining the parameter for the general-form Tikhonov regularization of linear ill-posed problems. In our approach the parameter is found by approximate minimization of the distance between the unknown noiseless data and the data reconstructed from the regularized solution. We approximate this distance by employing the Picard parameter to separate the noise from the data in the coordinate system of the generalized SVD. A simple and reliable algorithm for the estimation of the Picard parameter enables accurate implementation of the above procedure. We demonstrate the effectiveness of our method on several numerical examples. A MATLAB-based implementation of the proposed algorithms can be found at https://www.weizmann.ac.il/condmat/superc/software/, Comment: Accepted to SIAM Journal on Scientific Computing