1. Parallel Selective Algorithms for Nonconvex Big Data Optimization
- Author
-
Gesualdo Scutari, Simone Sagratella, and Francisco Facchinei
- Subjects
Mathematical optimization ,Parallel optimization ,variables selection ,distributed methods ,Jacobi method ,LASSO ,sparse solution ,MathematicsofComputing_NUMERICALANALYSIS ,Function (mathematics) ,Separable space ,Statistics::Machine Learning ,symbols.namesake ,Quadratic equation ,Lasso (statistics) ,Signal Processing ,Convergence (routing) ,symbols ,Differentiable function ,Electrical and Electronic Engineering ,Algorithm ,Block (data storage) ,Mathematics - Abstract
We propose a decomposition framework for the parallel optimization of the sum of a differentiable (possibly nonconvex) function and a (block) separable nonsmooth, convex one. The latter term is usually employed to enforce structure in the solution, typically sparsity. Our framework is very flexible and includes both fully parallel Jacobi schemes and Gauss–Seidel (i.e., sequential) ones, as well as virtually all possibilities “in between” with only a subset of variables updated at each iteration. Our theoretical convergence results improve on existing ones, and numerical results on LASSO, logistic regression, and some nonconvex quadratic problems show that the new method consistently outperforms existing algorithms.
- Published
- 2015