Search

A small improvement in the structure of the material could save the manufactory a lot of money. The free material design can be formulated as an optimization problem. However, due to its large scale, second-order methods cannot solve the free material design problem in reasonable size. We formulate the free material optimization (FMO) problem into a saddle-point form in which the inverse of the stiffness matrix A(E) in the constraint is eliminated. The size of A(E) is generally large, denoted as N by N. This is the first formulation of FMO without A(E). We apply the primal-dual subgradient method [17] to solve the restricted saddle-point formula. This is the first gradient-type method for FMO. Each iteration of our algorithm takes a total of $O(N^2)$ foating-point operations and an auxiliary vector storage of size O(N), compared with formulations having the inverse of A(E) which requires $O(N^3)$ arithmetic operations and an auxiliary vector storage of size $O(N^2)$. To solve the problem, we developed a closed-form solution to a semidefinite least squares problem and an efficient parameter update scheme for the gradient method, which are included in the appendix. We also approximate a solution to the bounded Lagrangian dual problem. The problem is decomposed into small problems each only having an unknown of k by k (k = 3 or 6) matrix, and can be solved in parallel. The iteration bound of our algorithm is optimal for general subgradient scheme. Finally we present promising numerical results., Comment: SIAM Journal on Optimization (accepted)

We consider mixed model of traffic flow distribution in large networks (BMW model, 1954 & Stable Dynamic model, 1999). We build dual problem and consider primal-dual mirror descent method for the dual problem. There are two ways to recover the solution of the primal problem. First technique is based on "models" approach and second one is based on Lagrange multiplyers technique. Both approaches have its own advantages and drawbacks. We discuss them., Comment: 11 pages, in Russian

We propose a new simple variant of Fast Gradient Method that requires only one projection per iteration. We called this method Triangle Method (TM) because it has a corresponding geometric description. We generalize TM for convex and strictly convex composite optimization problems. Then we propose Universal Triangle Method (UTM) for convex and strictly convex composite optimization problems (see Yu. Nesterov, Math. Program. 2015. for more details about what is Universal Fast Gradient Method). Finally, based on mini-batch technique we propose Stochastic Universal Triangle Method (SUTM). SUTM can be applied to stochastic convex and strictly convex composite optimization problems. Denote, that all the methods TM, UTM, SUTM are continuous on strictly convexity parameter and all of them reach known lower bounds. With additional assumption about the structure of the problems these methods work better than the lower bounds., Comment: in Russian

In this paper, we consider a non-convex loss-minimization problem of learning Supervised PageRank models, which can account for some properties not considered by classical approaches such as the classical PageRank model. We propose gradient-based and random gradient-free methods to solve this problem. Our algorithms are based on the concept of an inexact oracle and unlike the state state-of-the-art gradient-based method we manage to provide theoretically the convergence rate guarantees for both of them. In particular, under the assumption of local convexity of the loss function, our random gradient-free algorithm guarantees decrease of the loss function value expectation. At the same time, we theoretically justify that without convexity assumption for the loss function our gradient-based algorithm allows to find a point where the stationary condition is fulfilled with a given accuracy. For both proposed optimization algorithms, we find the settings of hyperparameters which give the lowest complexity (i.e., the number of arithmetic operations needed to achieve the given accuracy of the solution of the loss-minimization problem). The resulting estimates of the complexity are also provided. Finally, we apply proposed optimization algorithms to the web page ranking problem and compare proposed and state-of-the-art algorithms in terms of the considered loss function., Comment: 34 pages, 5 figures

In this paper, we propose three methods to solve the PageRank problem for the transition matrices with both row and column sparsity. Our methods reduce the PageRank problem to the convex optimization problem over the simplex. The first algorithm is based on the gradient descent in L1 norm instead of the Euclidean one. The second algorithm extends the Frank-Wolfe to support sparse gradient updates. The third algorithm stands for the mirror descent algorithm with a randomized projection. We proof converges rates for these methods for sparse problems as well as numerical experiments support their effectiveness., Comment: 26 pages; Accepted to Optimization Methods and Software

In this paper we propose a new efficient approach for numerical calculation of equillibriums in multistage transport problems. In the very core of our approach lies the proper combination of Universal Gradient Method proposed by Yu. Nesterov (2013) and conception of inexact oracle (Devolder--Glineur--Nesterov, 2011). In particular our technique allows us to calculate Wasserstein's Barycenter in a fast manner (this results generalized M. Cuturi et al. (2014))., Comment: 26 pages, in Russian, TRUDY MIPT. 2016. V. 8. no. 3

In this paper, we consider a problem of learning supervised PageRank models, which can account for some properties not considered by classical approaches such as the classical PageRank algorithm. Due to huge hidden dimension of the optimization problem we use random gradient-free methods to solve it. We prove a convergence theorem and estimate the number of arithmetic operations needed to solve it with a given accuracy. We find the best settings of the gradient-free optimization method in terms of the number of arithmetic operations needed to achieve given accuracy of the objective. In the paper, we apply our algorithm to the web page ranking problem. We consider a parametric graph model of users' behavior and evaluate web pages' relevance to queries by our algorithm. The experiments show that our optimization method outperforms the untuned gradient-free method in the ranking quality., Comment: 11 pages

In the article we lead a brief survey of contemporary gradient type methods (with inexact oracle) for stochastic optimization problems., Comment: 60 pages, in Russian

We propose an efficient dual algorithm for ELP based on Fast Gradient Method. The basic idea - to solve properly regularized dual problem., Comment: 19 pages, in Russian

In the paper we consider an application of mirror descent (dual averaging) to the stochastic online convex optimization problems. We compare classical mirror descent (Nemirovski-Yudin, 1979) with dual averaging (Nesterov, 2005) and Grigoriadis-Khachiyan algorithm (1995). Grigoriadis-Khachiyan algorithm has just proved to be a randomized mirror descent (dual averaging) with randomization in KL-projection of (sub)gradient to a unit simplex. We found out that this randomization is an optimal way of solving sparse matrix games and some other problems arising in convex optimization and experts weighting., Comment: 26 pages, in Russian

An attempt to merge into a single model, which reduces to the solution of non-smooth convex optimization problem: calculation model of OD-matrix (entropy model), the mode split model and the model of the equilibrium distribution of flows (Stable dynamic model, Nesterov - de Palma, 2003). To best of our knowledge, this is the first attempt to combine this three models. Previously such attempts were done for other types of equlibrium models, mainly with the BMW-model (1955), the calibration of which is significantly more difficult. We also remark, that our model much better then traditional from computational point of view., Comment: 48 pages, Russian

In this paper we develop a new approach to sparse principal component analysis (sparse PCA). We propose two single-unit and two block optimization formulations of the sparse PCA problem, aimed at extracting a single sparse dominant principal component of a data matrix, or more components at once, respectively. While the initial formulations involve nonconvex functions, and are therefore computationally intractable, we rewrite them into the form of an optimization program involving maximization of a convex function on a compact set. The dimension of the search space is decreased enormously if the data matrix has many more columns (variables) than rows. We then propose and analyze a simple gradient method suited for the task. It appears that our algorithm has best convergence properties in the case when either the objective function or the feasible set are strongly convex, which is the case with our single-unit formulations and can be enforced in the block case. Finally, we demonstrate numerically on a set of random and gene expression test problems that our approach outperforms existing algorithms both in quality of the obtained solution and in computational speed., Comment: Submitted

The joint spectral radius of a set of matrices is a measure of the maximal asymptotic growth rate that can be obtained by forming long products of matrices taken from the set. This quantity appears in a number of application contexts but is notoriously difficult to compute and to approximate. We introduce in this paper a procedure for approximating the joint spectral radius of a finite set of matrices with arbitrary high accuracy. Our approximation procedure is polynomial in the size of the matrices once the number of matrices and the desired accuracy are fixed.