Search

In recent years, the proximal gradient method and its variants have been generalized to Riemannian manifolds for solving optimization problems with an additively separable structure, i.e., $f + h$, where $f$ is continuously differentiable, and $h$ may be nonsmooth but convex with computationally reasonable proximal mapping. In this paper, we generalize the proximal Newton method to embedded submanifolds for solving the type of problem with $h(x) = \mu \|x\|_1$. The generalization relies on the Weingarten and semismooth analysis. It is shown that the Riemannian proximal Newton method has a local superlinear convergence rate under certain reasonable assumptions. Moreover, a hybrid version is given by concatenating a Riemannian proximal gradient method and the Riemannian proximal Newton method. It is shown that if the objective function satisfies the Riemannian KL property and the switch parameter is chosen appropriately, then the hybrid method converges globally and also has a local superlinear convergence rate. Numerical experiments on random and synthetic data are used to demonstrate the performance of the proposed methods.

We consider the problem of minimizing a differentiable function with locally Lipschitz continuous gradient on a stratified set and present a first-order algorithm designed to find a stationary point of that problem. Our assumptions on the stratified set are satisfied notably by the determinantal variety (i.e., matrices of bounded rank), its intersection with the cone of positive-semidefinite matrices, and the set of nonnegative sparse vectors. The iteration map of the proposed algorithm applies a step of projected-projected gradient descent with backtracking line search, as proposed by Schneider and Uschmajew (2015), to its input but also to a projection of the input onto each of the lower strata to which it is considered close, and outputs a point among those thereby produced that maximally reduces the cost function. Under our assumptions on the stratified set, we prove that this algorithm produces a sequence whose accumulation points are stationary, and therefore does not follow the so-called apocalypses described by Levin, Kileel, and Boumal (2022). We illustrate the apocalypse-free property of our method through a numerical experiment on the determinantal variety.

Orthogonality constraints naturally appear in many machine learning problems, from Principal Components Analysis to robust neural network training. They are usually solved using Riemannian optimization algorithms, which minimize the objective function while enforcing the constraint. However, enforcing the orthogonality constraint can be the most time-consuming operation in such algorithms. Recently, Ablin & Peyré (2022) proposed the Landing algorithm, a method with cheap iterations that does not enforce the orthogonality constraint but is attracted towards the manifold in a smooth manner. In this article, we provide new practical and theoretical developments for the landing algorithm. First, the method is extended to the Stiefel manifold, the set of rectangular orthogonal matrices. We also consider stochastic and variance reduction algorithms when the cost function is an average of many functions. We demonstrate that all these methods have the same rate of convergence as their Riemannian counterparts that exactly enforce the constraint. Finally, our experiments demonstrate the promise of our approach to an array of machine-learning problems that involve orthogonality constraints.

Quadratic hypersurfaces are a natural generalization of affine subspaces, and projections are elementary blocks of algorithms in optimization and machine learning. It is therefore intriguing that no proper studies and tools have been developed to tackle this nonconvex optimization problem. The quadproj package is a user-friendly and documented software that is dedicated to project a point onto a non-cylindrical central quadratic hypersurface.

We address the problem of computing the smallest symplectic eigenvalues and the corresponding eigenvectors of symmetric positive-definite matrices in the sense of Williamson's theorem. It is formulated as minimizing a trace cost function over the symplectic Stiefel manifold. We first investigate various theoretical aspects of this optimization problem such as characterizing the sets of critical points, saddle points, and global minimizers as well as proving that non-global local minimizers do not exist. Based on our recent results on constructing Riemannian structures on the symplectic Stiefel manifold and the associated optimization algorithms, we then propose solving the symplectic eigenvalue problem in the framework of Riemannian optimization. Moreover, a connection of the sought solution with the eigenvalues of a special class of Hamiltonian matrices is discussed. Numerical examples are presented., Comment: 24 pages, 2 figures

We propose the SH model, a simplified version of the well-known SIR compartmental model of infectious diseases. With optimized parameters and initial conditions, this time-invariant two-parameter two-dimensional model is able to fit COVID-19 hospitalization data over several months with high accuracy (e.g., the root relative squared error is below 10% for Belgium over the period from 2020-03-15 to 2020-07-15). Moreover, we observed that, when the model is trained on a suitable three-week period around the first hospitalization peak for Belgium, it forecasts the subsequent two months with mean absolute percentage error (MAPE) under 4%. We repeated the experiment for each French department and found 14 of them where the MAPE was below 20%. However, when the model is trained in the increase phase, it is less successful at forecasting the subsequent evolution., Comment: Paper home page: https://sites.uclouvain.be/absil/2020.05

We show that H��lder continuity of the gradient is not only a sufficient condition, but also a necessary condition for the existence of a global upper bound on the error of the first-order Taylor approximation. We also relate this global upper bound to the H��lder constant of the gradient. This relation is expressed as an interval, depending on the H��lder constant, in which the error of the first-order Taylor approximation is guaranteed to be. We show that, for the Lipschitz continuous case, the interval cannot be reduced. An application to the norms of quadratic forms is proposed, which allows us to derive a novel characterization of Euclidean norms.

We consider a low-rank tensor completion (LRTC) problem which aims to recover a tensor from incomplete observations. LRTC plays an important role in many applications such as signal processing, computer vision, machine learning, and neuroscience. A widely used approach is to combine the tensor completion data fitting term with a regularizer based on a convex relaxation of the multilinear ranks of the tensor. For the data fitting function, we model the tensor variable by using the Canonical Polyadic (CP) decomposition and for the low-rank promoting regularization function, we consider a graph Laplacian-based function which exploits correlations between the rows of the matrix unfoldings. For solving our LRTC model, we propose an efficient alternating minimization algorithm. Furthermore, based on the Kurdyka-{\L}ojasiewicz property, we show that the sequence generated by the proposed algorithm globally converges to a critical point of the objective function. Besides, an alternating direction method of multipliers algorithm is also developed for the LRTC model. Extensive numerical experiments on synthetic and real data indicate that the proposed algorithms are effective and efficient.

The Dynamic Economic Dispatch (DED) problem consists in satisfying a certain demand for electric power among scheduled generating units over a certain interval of time while satisfying the operating constraints of these units. The consideration of the valve-point effect (VPE) makes the problem more practical but also more challenging due to the non-linear and non-smooth constraints that are required for representing the model. We present a method, based on a sequence of piecewise linear approximations, which produces a feasible solution along with a lower bound on the global solution. In this way, this deterministic approach can trade off the speed which characterizes certain heuristics that are usually used to solve the DED-VPE for a better solution and insights about the problem. The method is applied to a widely used case study and provides a lower solution objective than the best known solution to date.

We present a method to compute a fitting curve B to a set of data points d0,...,dm lying on a manifold M. That curve is obtained by blending together Euclidean B\'ezier curves obtained on different tangent spaces. The method guarantees several properties among which B is C1 and is the natural cubic smoothing spline when M is the Euclidean space. We show examples on the sphere S2 as a proof of concept., Comment: in Proceedings of iTWIST'18, Paper-ID: 18, Marseille, France, November, 21-23, 2018

This paper presents a 3D circuit model capable of rapidly and accurately evaluating substrate noise coupling in the context of 3D integration. Since TSVs are large and noisy structures, the evaluation of electromagnetic coupling to and from TSVs has become crucial to the design of three-dimensional integrated circuits. In this work, we present a fast and accurate 3D circuit model to this end. The model is verified against full-wave simulations and wideband Sparameter measurements of three different TSV structures, including guard rings and up to four TSVs. The model maintains the accuracy of full-wave simulations while presenting a speed increase of 2 to 3 orders of magnitude. This features enables the model to investigate the coupling parameters and crosstalk voltages in very large and complex 3D volumes. Our powerful model provides new opportunities in large scale analysis, and as an example an 11x11 TSV array is simulated in both frequency-and time-domain over a wide frequency band including 121 TSVs and circuit nodes.

The third edition of the "international - Traveling Workshop on Interactions between Sparse models and Technology" (iTWIST) took place in Aalborg, the 4th largest city in Denmark situated beautifully in the northern part of the country, from the 24th to 26th of August 2016. The workshop venue was at the Aalborg University campus. One implicit objective of this biennial workshop is to foster collaboration between international scientific teams by disseminating ideas through both specific oral/poster presentations and free discussions. For this third edition, iTWIST'16 gathered about 50 international participants and features 8 invited talks, 12 oral presentations, and 12 posters on the following themes, all related to the theory, application and generalization of the "sparsity paradigm": Sparsity-driven data sensing and processing (e.g., optics, computer vision, genomics, biomedical, digital communication, channel estimation, astronomy); Application of sparse models in non-convex/non-linear inverse problems (e.g., phase retrieval, blind deconvolution, self calibration); Approximate probabilistic inference for sparse problems; Sparse machine learning and inference; "Blind" inverse problems and dictionary learning; Optimization for sparse modelling; Information theory, geometry and randomness; Sparsity? What's next? (Discrete-valued signals; Union of low-dimensional spaces, Cosparsity, mixed/group norm, model-based, low-complexity models, ...); Matrix/manifold sensing/processing (graph, low-rank approximation, ...); Complexity/accuracy tradeoffs in numerical methods/optimization; Electronic/optical compressive sensors (hardware)., Comment: 69 pages, 22 extended abstracts, iTWIST'16 website: http://www.itwist16.es.aau.dk

Optimal distribution of power among generating units to meet a specific demand subject to system constraints is an ongoing research topic in the power system community. The problem, even in a static setting, turns out to be hard to solve with conventional optimization methods owing to the consideration of valve-point effects which make the cost function nonsmooth and nonconvex. This difficulty gave rise to the proliferation of population-based global heuristics in order to address the multi-extremal and nonsmooth problem. In this paper, we address the economic load dispatch problem (ELDP) with valve-point effect in its classic formulation where the cost function for each generator is expressed as the sum of a quadratic term and a rectified sine term. We propose two methods that resort to piecewise-quadratic surrogate cost functions, yielding surrogate problems that can be handled by mixed-integer quadratic programming (MIQP) solvers. The first method shows that the global solution of the ELDP can often be found by using a fixed and very limited number of quadratic pieces in the surrogate cost function. The second method adaptively builds piecewise-quadratic surrogate under-estimations of the ELDP cost function, yielding a sequence of surrogate MIQP problems. It is shown that any limit point of the sequence of MIQP solutions is a global solution of the ELDP. Moreover, numerical experiments indicate that the proposed methods outclass the state-of-the-art algorithms in terms of minimization value and computation time on practical instances., Comment: 8 pages

We study the problem of separating audio sources from a single linear mixture. The goal is to find a decomposition of the single channel spectrogram into a sum of individual contributions associated to a certain number of sources. In this paper, we consider an informed source separation problem in which the input spectrogram is partly annotated. We propose a convex formulation that relies on a nuclear norm penalty to induce low rank for the contributions. We show experimentally that solving this model with a simple subgradient method outperforms a previously introduced nonnegative matrix factorization (NMF) technique, both in terms of source separation quality and computation time., Comment: Submitted to ESANN 2013 conference

Synchronization of rotations is the problem of estimating a set of rotations R_i in SO(n), i = 1, ..., N, based on noisy measurements of relative rotations R_i R_j^T. This fundamental problem has found many recent applications, most importantly in structural biology. We provide a framework to study synchronization as estimation on Riemannian manifolds for arbitrary n under a large family of noise models. The noise models we address encompass zero-mean isotropic noise, and we develop tools for Gaussian-like as well as heavy-tail types of noise in particular. As a main contribution, we derive the Cram��r-Rao bounds of synchronization, that is, lower-bounds on the variance of unbiased estimators. We find that these bounds are structured by the pseudoinverse of the measurement graph Laplacian, where edge weights are proportional to measurement quality. We leverage this to provide interpretation in terms of random walks and visualization tools for these bounds in both the anchored and anchor-free scenarios. Similar bounds previously established were limited to rotations in the plane and Gaussian-like noise.

We propose an algorithm for solving nonlinear convex programs defined in terms of a symmetric positive semidefinite matrix variable $X$. This algorithm rests on the factorization $X=Y Y^T$, where the number of columns of Y fixes the rank of $X$. It is thus very effective for solving programs that have a low rank solution. The factorization $X=Y Y^T$ evokes a reformulation of the original problem as an optimization on a particular quotient manifold. The present paper discusses the geometry of that manifold and derives a second order optimization method. It furthermore provides some conditions on the rank of the factorization to ensure equivalence with the original problem. The efficiency of the proposed algorithm is illustrated on two applications: the maximal cut of a graph and the sparse principal component analysis problem., Comment: submitted

In male rodents, the arginine-vasopressin-immunoreactive (AVP-ir) neurones of the bed nucleus of the stria terminalis (BNST) and medial amygdala are controlled by plasma testosterone levels (decreased after castration and restored by exogenous testosterone). AVP transcription in these nuclei is increased in adulthood by a synergistic action of the androgenic and oestrogenic metabolites of testosterone and, accordingly, androgen and oestrogen receptors are present in both BNST and medial amygdala. We used knockout mice lacking a functional aromatase enzyme (ArKO) to investigate the effects of a chronic depletion of oestrogens on the sexually dimorphic AVP system. Wild-type (WT) and ArKO male mice were perfused 48 h after an i.c.v. colchicine injection and brain sections were then processed for AVP immunocytochemistry. A prominent decrease (but not a complete suppression) of AVP-ir structures was observed in the BNST and medial amygdala of ArKO mice by comparison with the WT. Similarly, AVP-ir fibres were reduced in the lateral septum of ArKO mice and but not in the medial preoptic area, a region where the AVP system is not sexually dimorphic in rats. No change was detected in the supraoptic and suprachiasmatic nuclei. However, a decrease in AVP-ir cell numbers was however, detected in one subregion of the paraventricular nucleus. These data support the hypothesis that the steroid-sensitive sexually dimorphic AVP system of the mouse forebrain is mainly under the control of aromatized metabolites of testosterone.

Coherent beam combining (CBC) of fiber lasers provide an attractive mean of reaching high output laser power by scaling up the available energy while keeping fiber intrinsic advantages of compactness, reliability, efficiency, and beam quality. In CBC architectures, the power of a master oscillator (MO) is divided into N fibers that are amplified individually. The phase perturbations between channels can be measured using various techniques [1-3] and are corrected by individual phase modulators placed on each fiber before the amplification. In this Communication, we present a Silicon PIC integrating a 1:16 channels splitting network and thermal phase modulators array with low electrical power consumption and a bandwidth compatible with CBC requirements.

Optimization on manifolds is a rapidly developing branch of nonlinear optimization. Its focus is on problems where the smooth geometry of the search space can be leveraged to design effcient numerical algorithms. In particular, optimization on manifolds is well-suited to deal with rank and orthogonality constraints. Such structured constraints appear pervasively in machine learning applications, including low-rank matrix completion, sensor network localization, camera network registration, independent component analysis, metric learning, dimensionality reduction and so on. The Manopt toolbox, available at www.manopt.org, is a user-friendly, documented piece of software dedicated to simplify experimenting with state of the art Riemannian optimization algorithms. By dealing internally with most of the differential geometry, the package aims particularly at lowering the entrance barrier. © 2014 Nicolas Boumal.

Dark current degradation mechanisms in Ge VPIN photodetectors were studied. A methodology to estimate the failure percentages has been developed and applied. Degradation/recovery processes and Ea-decrease of Idark after stress suggest increased TAT during degradation.