Search

The recent designed non-linear conjugate gradient method of Dai and Kou [SIAM J Optim. 2013;23:296–320] is very efficient currently in solving large-scale unconstrained minimization problems due to its simpler iterative form, lower storage requirement and its closeness to the scaled memoryless BFGS method. Just because of these attractive properties, this method was extended successfully to solve higher dimensional symmetric non-linear equations in recent years. Nevertheless, its numerical performance in solving convex constrained monotone equations has never been explored. In this paper, combining with the projection method of Solodov and Svaiter, we develop a family of non-linear conjugate gradient methods for convex constrained monotone equations. The proposed methods do not require the Jacobian information of equations, and even they do not store any matrix in each iteration. They are potential to solve non-smooth problems with higher dimensions. We prove the global convergence of the class of the proposed methods and establish its R-linear convergence rate under some reasonable conditions. Finally, we also do some numerical experiments to show that the proposed methods are efficient and promising. [ABSTRACT FROM AUTHOR]

CG_DESCENT is a state-of-the-art algorithm to solve large-scale unconstrained minimization problems. However, research activities on CG_DESCENT in some other scenarios are relatively fewer. In this paper, by combining with the projection method of Solodov and Svaiter, we extend CG_DESCENT to solve large-scale nonlinear convex constrained monotone equations. The proposed method does not require the Jacobian information, even though it does not store any matrix at each iteration. It thus has the potential to solve large-scale non-smooth problems. Under some mild conditions, we show that the proposed method converges globally. Primary numerical results illustrate that the proposed method works quite well. Moreover, we also extend this method to solve the -norm regularized problems to decode a sparse signal in compressive sensing. Performance comparisons show that the proposed method is practical, efficient and competitive with the compared ones. [ABSTRACT FROM AUTHOR]

Abstract: Finding sparse solutions to under-determined linear systems of equations have intensively involved in fields of machine learning, signal processing, compressive sensing, linear inverse problems and statistical inference. Generally, the task can be realized by solving -norm regularized minimization problems. However, the resulting problem is challenging due to the non-smoothness of the regularization term. Inspired by Nesterov’s smoothing technique, this paper proposes, analyzes and tests a modified Polak–Ribière–Polyak conjugate gradient method to solve large-scale -norm least squares problem for sparse signal recovery. The per-iteration cost of the proposed algorithm is dominated by three matrix–vector multiplications and the global convergence is guaranteed by results in optimization literature. Moreover, the algorithm is also accelerated by continuation loops as usual. The limited experiments show that this continuation technique benefits to its performance. Numerical experiments which decode a sparse signal from its limited measurements illustrate that the proposed algorithm performs better than NESTA—a recently developed code with Nesterov’s smoothing technique and gradient algorithm. [Copyright &y& Elsevier]

For solving large-scale unconstrained minimization problems, the nonlinear conjugate gradient method is welcome due to its simplicity, low storage, efficiency and nice convergence properties. Among all the methods in the framework, the conjugate gradient descent algorithm — CG_DESCENT is very popular, in which the generated directions descend automatically, and this nice property is independent of any line search used. In this paper, we generalize CG_DESCENT with two Barzilai–Borwein steplength reused cyclically. We show that the resulting algorithm owns attractive sufficient descent property and converges globally under some mild conditions. We test the proposed algorithm by using a large set of unconstrained problems with high dimensions in CUTEr library. The numerical comparisons with the state-of-the-art algorithm CG_DESCENT illustrate that the proposed method is effective, competitive, and promising. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, we propose a family of derivative-free conjugate gradient methods for large-scale nonlinear systems of equations. They come from two modified conjugate gradient methods [W.Y. Cheng, A two term PRP based descent Method, Numer. Funct. Anal. Optim. 28 (2007) 1217–1230; L. Zhang, W.J. Zhou, D.H. Li, A descent modified Polak–Ribiére–Polyak conjugate gradient method and its global convergence, IMA J. Numer. Anal. 26 (2006) 629–640] recently proposed for unconstrained optimization problems. Under appropriate conditions, the global convergence of the proposed method is established. Preliminary numerical results show that the proposed method is promising. [Copyright &y& Elsevier]