Search

Many problems of substantial current interest in machine learning, statistics, and data science can be formulated as sparse and low-rank optimization problems. In this paper, we present the nonconvex exterior-point optimization solver (NExOS)—a first-order algorithm tailored to sparse and low-rank optimization problems. We consider the problem of minimizing a convex function over a nonconvex constraint set, where the set can be decomposed as the intersection of a compact convex set and a nonconvex set involving sparse or low-rank constraints. Unlike the convex relaxation approaches, NExOS finds a locally optimal point of the original problem by solving a sequence of penalized problems with strictly decreasing penalty parameters by exploiting the nonconvex geometry. NExOS solves each penalized problem by applying a first-order algorithm, which converges linearly to a local minimum of the corresponding penalized formulation under regularity conditions. Furthermore, the local minima of the penalized problems converge to a local minimum of the original problem as the penalty parameter goes to zero. We then implement and test NExOS on many instances from a wide variety of sparse and low-rank optimization problems, empirically demonstrating that our algorithm outperforms specialized methods. [ABSTRACT FROM AUTHOR]

In this paper, we propose a Riemannian version of the difference of convex algorithm (DCA) to solve a minimization problem involving the difference of convex (DC) function. The equivalence between the classical and simplified Riemannian versions of the DCA is established. We also prove that under mild assumptions the Riemannian version of the DCA is well defined and every cluster point of the sequence generated by the proposed method, if any, is a critical point of the objective DC function. Some duality relations between the DC problem and its dual are also established. To illustrate the algorithm's effectiveness, some numerical experiments are presented. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new adaptive method for solving the non-convex quadratic minimization problem subject to box constraints, where the associated matrix is indefinite, in particular with one negative eigenvalue. We investigate the derived sufficient global optimality conditions by exploiting the particular form of the Moreau envelope (L-subdifferential) of the quadratic function and abstract convexity, also to develop a new algorithm for solving the original problem without transforming it, that we call adaptive global algorithm, which can effectively find one global minimizer of the problem. Furthermore, the research of the convex support of the objective function allows us to characterize the global optimum and reduce the complexity of the big size problems. We give some theoretical aspects of global optimization and present numerical examples with test problems for illustrating our approach. [ABSTRACT FROM AUTHOR]

Decentralized algorithms solve multi-agent problems over a connected network, where the information can only be exchanged with the accessible neighbors. Though there exist several decentralized optimization algorithms, there are still gaps in convergence conditions and rates between decentralized and centralized algorithms. In this paper, we fill some gaps by considering two decentralized algorithms: EXTRA and NIDS. They both converge linearly with strongly convex objective functions. We will answer two questions regarding them. What are the optimal upper bounds for their stepsizes? Do decentralized algorithms require more properties on the functions for linear convergence than centralized ones? More specifically, we relax the required conditions for linear convergence of both algorithms. For EXTRA, we show that the stepsize is comparable to that of centralized algorithms. For NIDS, the upper bound of the stepsize is shown to be exactly the same as the centralized ones. In addition, we relax the requirement for the objective functions and the mixing matrices. We provide the linear convergence results for both algorithms under the weakest conditions. [ABSTRACT FROM AUTHOR]

Jin et al. (in J. Optim. Theory Appl. 155:1073-1083, ) proposed homogeneous self-dual algorithms for stochastic semidefinite programs with finite event space. In this paper, we utilize their work to derive homogeneous self-dual algorithms for stochastic second-order cone programs with finite event space. We also show how the structure in the stochastic second-order cone programming problems may be exploited so that the algorithms developed for these problems have less complexity than the algorithms developed for stochastic semidefinite programs mentioned above. [ABSTRACT FROM AUTHOR]

The purpose of this paper is to introduce a new instance of the Mesh Adaptive Direct Search (Mads) class of algorithms, which utilizes a more uniform distribution of poll directions than do other common instances, such as OrthoMads and LtMads. Our new implementation, called QrMads, bases its poll directions on an equal area partitioning of the n-dimensional unit sphere and the QR decomposition to obtain an orthogonal set of directions. While each instance produces directions which are dense in the limit, QrMads directions are more uniformly distributed in the unit sphere. This uniformity is the key to enhanced performance in higher dimensions and for constrained problems. The trade-off is that QrMads is no longer deterministic and at each iteration the set of polling directions is no longer orthogonal. Instead, at each iteration, the poll directions are only ‘nearly orthogonal,’ becoming increasingly closer to orthogonal as the mesh size decreases. Finally, we present a variety of test results on smooth, nonsmooth, unconstrained, and constrained problems and compare them to OrthoMads on the same set of problems. [ABSTRACT FROM AUTHOR]

In this paper a discrete location model for non-essential service facilities planning is described, which seeks the number, location, and size of facilities, that maximizes the total expected demand attracted by the facilities. It is assumed that the demand for service is sensitive to the distance from facilities and to their size. It is also assumed that facilities must satisfy a threshold level of demand (facilities are not economically viable below that level). A Mixed-Integer Nonlinear Programming (MINLP) model is proposed for this problem. A branch-and-bound algorithm is designed for solving this MINLP and its convergence to a global minimum is established. A finite procedure is also introduced to find a feasible solution for the MINLP that reduces the overall search in the binary tree generated by the branch-and-bound algorithm. Some numerical results using a GAMS/MINOS implementation of the algorithm are reported to illustrate its efficacy and efficiency in practice. [ABSTRACT FROM AUTHOR]

In the conic optimization problems, it is well-known that a positive duality gap may occur, and that solving such a problem is numerically difficult or unstable. For such a case, we propose a facial reduction algorithm to find a primal-dual pair of conic optimization problems having the zero duality gap and the optimal value equal to one of the original primal or dual problems. The conic expansion approach is also known as a method to find such a primal-dual pair, and in this paper we clarify the relationship between our facial reduction algorithm and the conic expansion approach. Our analysis shows that, although they can be regarded as dual to each other, our facial reduction algorithm has ability to produce a finer sequence of faces of the cone including the feasible region. A simple proof of the convergence of our facial reduction algorithm for the conic optimization is presented. We also observe that our facial reduction algorithm has a practical impact by showing numerical experiments for graph partition problems; our facial reduction algorithm in fact enhances the numerical stability in those problems. [ABSTRACT FROM AUTHOR]

Many real applications can be formulated as nonlinear minimization problems with a single linear equality constraint and box constraints. We are interested in solving problems where the number of variables is so huge that basic operations, such as the evaluation of the objective function or the updating of its gradient, are very time consuming. Thus, for the considered class of problems (including dense quadratic programs), traditional optimization methods cannot be applied directly. In this paper, we define a decomposition algorithm model which employs, at each iteration, a descent search direction selected among a suitable set of sparse feasible directions. The algorithm is characterized by an acceptance rule of the updated point which on the one hand permits to choose the variables to be modified with a certain degree of freedom and on the other hand does not require the exact solution of any subproblem. The global convergence of the algorithm model is proved by assuming that the objective function is continuously differentiable and that the points of the level set have at least one component strictly between the lower and upper bounds. Numerical results on large-scale quadratic problems arising in the training of support vector machines show the effectiveness of an implemented decomposition scheme derived from the general algorithm model. [ABSTRACT FROM AUTHOR]

Many optimization problems reduce to the solution of a system of linear inequalities (SLI). Some solution methods use relaxed, averaged projections. Others invoke surrogate constraints (typically stemming from aggregation). This paper proposes a blend of these two approaches. A novelty comes from introducing as surrogate constraint a halfspace defined by differences of algorithmic iterates. The first iteration is identical to surrogate constraints methods. In next iterations, for a given approximation x¯, besides the violated constraints in x¯, we also take into consideration the surrogate inequality, which we have obtained in the previous iteration. The motivation for this research comes from the recent work of Scolnik et al. (Appl. Numer. Math. 41, 499—513, 2002), who studied some projection methods for a system of linear equations. [ABSTRACT FROM AUTHOR]