Search

This paper proposes a nonmonotone scaled conjugate gradient algorithm for solving large-scale unconstrained optimization problems, which combines the idea of scaled memoryless Broyden-Fletcher-Goldfarb-Shanno preconditioned conjugate gradient method with the nonmonotone technique. An attractive property of the proposed method is that the search direction always provides sufficient descent step at each iteration. This property is independent of the line search used. Under appropriate assumptions, the method is proven to possess global convergence for nonconvex smooth functions, and R-linear convergence for strongly convex functions. Preliminary numerical results and related comparisons show the efficiency of the proposed method in practical computation. [ABSTRACT FROM AUTHOR]

Adaptive regularized framework using cubics has emerged as an alternative to line-search and trust-region algorithms for smooth nonconvex optimization, with an optimal complexity among second-order methods. In this paper, we propose and analyze the use of an iteration dependent scaled norm in the adaptive regularized framework using cubics. Within such a scaled norm, the obtained method behaves as a line-search algorithm along the quasi-Newton direction with a special backtracking strategy. Under appropriate assumptions, the new algorithm enjoys the same convergence and complexity properties as adaptive regularized algorithm using cubics. The complexity for finding an approximate first-order stationary point can be improved to be optimal whenever a second-order version of the proposed algorithm is regarded. In a similar way, using the same scaled norm to define the trust-region neighborhood, we show that the trust-region algorithm behaves as a line-search algorithm. The good potential of the obtained algorithms is shown on a set of large-scale optimization problems. [ABSTRACT FROM AUTHOR]

This paper presents a new nonmonotone quasi-Newton trust-region algorithm of the conic model for the solution of unconstrained optimization problems, where the computation of the horizontal vectors is easier and the approximate Hessian matrices can be maintained positive definite. Under reasonable assumptions, the global convergence, the locally linear and superlinear convergence of the proposed algorithm are developed, respectively. It is well known that in applying trust-region algorithms, the basic issue is how to solve the trust-region subproblem efficiently. To deal with the issue, an approximate solution method is developed in this paper. Note that the approximate solution method not only is computationally cheap, but also preserves the strong convergence properties as the exact solution methods. Numerical results are shown for a number of test problems from the literature. [ABSTRACT FROM AUTHOR]

The paper discusses several versions of the method of shortest residuals, a specific variant of the conjugate gradient algorithm, first introduced by Lemaréchal and Wolfe and discussed by Hestenes in a quadratic case. In the paper we analyze the global convergence of the versions considered. Numerical comparison of these versions of the method of shortest residuals and an implementation of a standard Polak–Ribière conjugate gradient algorithm is also provided. It supports the claim that the method of shortest residuals is a viable technique, competitive to other conjugate gradient algorithms. [ABSTRACT FROM AUTHOR]

In this paper, a DL-type conjugate gradient method is presented. The given method is a modification of the Dai–Liao conjugate gradient method. It can also be considered as a modified LS conjugate gradient method. For general objective functions, the proposed method possesses the sufficient descent condition under the Wolfe line search and is globally convergent. Numerical comparisons show that the proposed algorithm slightly outperforms the PRP+ and CG-descent gradient algorithms as well as the Barzilai–Borwein gradient algorithm. [ABSTRACT FROM AUTHOR]

In this paper, a finite filter is used in the structure of the Barzilai–Browein (BB) gradient method in order to propose a new modified BB algorithm for solving large-scale unconstrained optimization problems. Our algorithm is equipped with a relaxed nonmonotone line search technique which allows the algorithm to enjoy the nonmonotonicity properties from scratch. Under some suitable conditions, the global convergence property of the new proposed algorithm is established. Numerical results on some test problems in CUTEr library show the efficiency and effectiveness of the new algorithm in practice too. [ABSTRACT FROM PUBLISHER]

In this paper, we combine the conjugate gradient method with the Barzilai and Borwein gradient method, and propose a Barzilai and Borwein scaling conjugate gradient method for nonlinear unconstrained optimization problems. The new method does not require to compute and store matrices associated with Hessian of the objective functions, and has an advantage of less computational efforts. Moreover, the descent direction property and the global convergence are established when the line search fulfills the Wolfe conditions. The limited numerical experiments and comparisons show that the proposed algorithm is potentially efficient. [ABSTRACT FROM AUTHOR]

This paper presents a new hybrid algorithm for unconstrained optimization problems, which combines the idea of the IMPBOT algorithm with the nonmonotone line search technique. A feature of the proposed method is that at each iteration, a system of linear equations is solved only once to obtain a trial step, via a modified limited-memory BFGS two loop recursion that requires only matrix–vector products, thus reducing the computations and storage. Furthermore, when the trial step is not accepted, the proposed method performs a line search along it using a modified nonmonotone scheme, thus a larger stepsize can be yielded in each line search procedure. Under some reasonable assumptions, the convergence properties of the proposed algorithm are analysed. Numerical results are also reported to show the efficiency of this proposed method. [ABSTRACT FROM PUBLISHER]

This paper extends the technique used in the damped BFGS method of Powell [Algorithms for nonlinear constraints that use Lagrange functions, Math. Program. 14 (1978), 224–248] to the Broyden family of quasi-Newton methods with applications to unconstrained optimization problems. Appropriate conditions on the damped technique are proposed to enforce safely the positive definiteness property for all Broyden's updates. It is shown that this technique maintains theq-superlinear convergence property of the restricted Broyden family of methods for uniformly convex functions. It also extends the global convergence property to all members of the family. Preliminary numerical results are described which show that appropriate ways for employing the proposed technique improve the performance of all members of the Broyden family of methods substantially and significantly in certain cases. They also enforce convergence of divergent quasi-Newton methods. [ABSTRACT FROM AUTHOR]

Abstract: This study devotes to incorporating a nonmonotone strategy with an automatically adjusted trust-region radius to propose a more efficient hybrid of trust-region approaches for unconstrained optimization. The primary objective of the paper is to introduce a more relaxed trust-region approach based on a novel extension in trust-region ratio and radius. The next aim is to employ stronger nonmonotone strategies, i.e. bigger trust-region ratios, far from the optimizer and weaker nonmonotone strategies, i.e. smaller trust-region ratios, close to the optimizer. The global convergence to first-order stationary points as well as the local superlinear and quadratic convergence rates are also proved under some reasonable conditions. Some preliminary numerical results and comparisons are also reported. [Copyright &y& Elsevier]

In this paper, an improved spectral conjugate gradient algorithm is developed for solving nonconvex unconstrained optimization problems. Different from the existent methods, the spectral and conjugate parameters are chosen such that the obtained search direction is always sufficiently descent as well as being close to the quasi-Newton direction. With these suitable choices, the additional assumption in the method proposed by Andrei on the boundedness of the spectral parameter is removed. Under some mild conditions, global convergence is established. Numerical experiments are employed to demonstrate the efficiency of the algorithm for solving large-scale benchmark test problems, particularly in comparison with the existent state-of-the-art algorithms available in the literature. [ABSTRACT FROM AUTHOR]

In this paper, by using a forcing function and the reverse modulus of continuity of gradient, we propose a generalization and development of the nonmonotone line search method for unconstrained optimization problems. The global convergence property of the new method is established under some reasonable conditions that are weaker than those of the existing nonmonotone line search methods. Furthermore, the proof of convergence is simpler than other methods. [ABSTRACT FROM PUBLISHER]

In this paper, we suggest a new conjugate gradient algorithm that for all k≥0 both the descent and the conjugacy conditions are guaranteed. The search direction is selected as a linear combination of−g k+1 and s k , where g k+1=∇ f(x k+1), s k =x k+1−x k and the coefficients in this linear combination are selected in such a way that both the descent and the conjugacy conditions are satisfied at every iteration. In order to define the algorithm and to prove its convergence, the modified Wolfe line search is introduced, in which the parameter in the standard second Wolfe condition is changed at every iteration. It is shown that for general nonlinear functions, the algorithm with modified Wolfe line search generates directions bounded away from infinity. The algorithm uses an acceleration scheme modifying the step length α k in such a manner as to improve the reduction of the function values along the iterations. Numerical comparisons with some conjugate gradient algorithms using a set of 750 unconstrained optimization problems, some of them from the CUTE library, show that the computational scheme outperforms the known conjugate gradient algorithms like Hestenes and Stiefel, Polak et al., Dai and Yuan or hybrid Dai and Yuan, as well as the CG_DESCENT method by Hager and Zhang with the Wolfe line search conditions. [ABSTRACT FROM PUBLISHER]

In this paper, a new combined extended Conjugate-Gradient (CG) and Variable-Metric (VM) methods is proposed for solving unconstrained large-scale numerical optimization problems. The basic idea is to choose a combination of the current gradient and some pervious search directions as a new search direction updated by Al-Bayati's SCVM-method to fit a new step-size parameter using Armijo Inexact Line Searches (ILS). This method is based on the ILS and its numerical properties are discussed using different non-linear test functions with various dimensions. The global convergence property of the new algorithm is investigated under few weak conditions. Numerical experiments show that the new algorithm seems to converge faster and is superior to some other similar methods in many situations. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, we propose a trust region method for unconstrained optimization that can be regarded as a combination of conic model, nonmonotone and line search techniques. Unlike in traditional trust region methods, the subproblem of our algorithm is the conic minimization subproblem; moreover, our algorithm performs a nonmonotone line search to find the next iteration point when a trial step is not accepted, instead of resolving the subproblem. The global and superlinear convergence results for the algorithm are established under reasonable assumptions. Numerical results show that the new method is efficient for unconstrained optimization problems. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, a nonmonotone trust region algorithm for unconstrained optimization problems is presented. In the algorithm, a kind of nonmonotone technique, which is evidently different from Grippo, Lampariello and Lucidi’s approach, is used. Under mild conditions, global and local convergence results of the algorithm are established. Preliminary numerical results show that the new algorithm is efficient. [ABSTRACT FROM AUTHOR]

Abstract: The aim of this paper is to incorporate the preconditioned gradient path in a nonmonotone stabilization algorithm for unconstrained optimization. The global convergence and locally superlinear convergence are established for this class of algorithms. Finally, we report in details the numerical results which show the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, a new descent algorithm for solving unconstrained optimization problem is presented. Its search direction is descent and line search procedure can be avoided except for the first iteration. It is globally convergent under mild conditions. The search direction of the new algorithm is generalized and convergence of corresponding algorithm is also proved. Numerical results show that the algorithm is efficient for given test problems. [Copyright &y& Elsevier]

In this paper a new hybrid conjugate gradient algorithm is proposed and analyzed. The parameter β k is computed as a convex combination of the Polak-Ribière-Polyak and the Dai-Yuan conjugate gradient algorithms, i.e. β=(1− θ k) β+ θ k β. The parameter θ k in the convex combination is computed in such a way that the conjugacy condition is satisfied, independently of the line search. The line search uses the standard Wolfe conditions. The algorithm generates descent directions and when the iterates jam the directions satisfy the sufficient descent condition. Numerical comparisons with conjugate gradient algorithms using a set of 750 unconstrained optimization problems, some of them from the CUTE library, show that this hybrid computational scheme outperforms the known hybrid conjugate gradient algorithms. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, we present a nonmonotone conic trust region method based on line search technique for unconstrained optimization. The new algorithm can be regarded as a combination of nonmonotone technique, line search technique and conic trust region method. When a trial step is not accepted, the method does not resolve the trust region subproblem but generates an iterative point whose steplength satisfies some line search condition. The function value can only be allowed to increase when trial steps are not accepted in close succession of iterations. The local and global convergence properties are proved under reasonable assumptions. Numerical experiments are conducted to compare this method with the existing methods. [Copyright &y& Elsevier]

Abstract: In this paper we consider a conic trust-region method for unconstrained optimization problems and analyze its convergence properties. We propose a convenient curvilinear search method to approximately solve the arising conic trust-region subproblem. Note that this approximate method preserves the strong convergence properties of the exact solution methods and is easy to implement. Both the linear and superlinear convergence of the method are established under commonly used conditions. Numerical experiments are conducted to show the efficiency of the new method. [Copyright &y& Elsevier]

In this paper we present a new class of memory gradient methods for unconstrained optimization problems and develop some useful global convergence properties under some mild conditions. In the new algorithms, trust region approach is used to guarantee the global convergence. Numerical results show that some memory gradient methods are stable and efficient in practical computation. In particular, some memory gradient methods can be reduced to the BB method in some special cases. [ABSTRACT FROM AUTHOR]

Abstract: This paper concerns a nonmonotone line search technique and its application to the trust region method for unconstrained optimization problems. In our line search technique, the current nonmonotone term is a convex combination of the previous nonmonotone term and the current objective function value, instead of an average of the successive objective function values that was introduced by Zhang and Hager [H. Zhang, W.W. Hager, A nonmonotone line search technique and its application to unconstrained optimization, SIAM J. Optim. 14 (4) (2004) 1043–1056]. We incorporate this nonmonotone scheme into the traditional trust region method such that the new algorithm possesses nonmonotonicity. Unlike the traditional trust region method, our algorithm performs a nonmonotone line search to find a new iteration point if a trial step is not accepted, instead of resolving the subproblem. Under mild conditions, we prove that the algorithm is global and superlinear convergence holds. Primary numerical results are reported. [Copyright &y& Elsevier]

Abstract: This paper studies a successive partitioning group correction algorithm and its some modified algorithms for solving large scale sparse unconstrained optimization problems. The methods depend on a symmetric consistent partition of the columns of the Hessian matrix. A q-superlinear convergence result and an r-convergence rate estimate show that the methods have good local convergence properties. The numerical results show that the methods, especially the modified algorithms, may be competitive with some current used algorithms. [Copyright &y& Elsevier]

This paper proposes a line search technique to satisfy a relaxed form of the strong Wolfe conditions in order to guarantee the descent condition at each iteration of the Polak-Ribière-Polyak conjugate gradient algorithm. It is proved that this line search algorithm preserves the usual convergence properties of any descent algorithm. In particular, it is shown that the Zoutendijk condition holds under mild assumptions. It is also proved that the resulting conjugate gradient algorithm is convergent under a strong convexity assumption. For the nonconvex case, a globally convergent modification is proposed. Numerical tests are presented. [ABSTRACT FROM AUTHOR]

Abstract: In this paper we propose a new class of supermemory gradient methods for unconstrained optimization problems. Trust region approach is used in the new algorithms to guarantee the global convergence. In each iteration, the new algorithms generate a suitable trust region radius automatically and obtain the next iterate by solving a simple subproblem. These algorithms converge stably and averagely due to using more iterative information at each iteration, and can be reduced to quasi-Newton methods when the iterate is close to the optimal solution. Numerical results show that this new class of supermemory gradient methods is effective in practical computation. [Copyright &y& Elsevier]

Abstract: This paper studies a successive partitioning group correction Cholesky algorithm and its modified algorithm for solving large scale sparse unconstrained optimization problems. These methods employ an initial Cholesky factorization of their approximation Hessians and then correct the partitioning group(s) of their diagonal factor and lower triangular factor directly and successively at each step. Iterations are generated using forward and backward substitution employing the update factorizations. A self-correcting property, a q-superlinear convergence result and an r-convergence rate estimate show that the two methods both have good local convergence properties. The numerical results show that the two methods, especially the modified algorithm may be competitive with some current used algorithms. [Copyright &y& Elsevier]

In this paper, an algorithm for unconstrained optimization that employs both trust region techniques and curvilinear searches is proposed. At every iteration, we solve the trust region subproblem whose radius is generated adaptively only once. Nonmonotonic backtracking curvilinear searches are performed when the solution of the subproblem is unacceptable. The global convergence and fast local convergence rate of the proposed algorithms axe established under some reasonable conditions. The results of numerical experiments are reported to show the effectiveness of the proposed algorithms. [ABSTRACT FROM AUTHOR]

Abstract: Many methods for solving minimization problems are variants of Newton method, which requires the specification of the Hessian matrix of second derivatives. Quasi-Newton methods are intended for the situation where the Hessian is expensive or difficult to calculate. Quasi-Newton methods use only first derivatives to build an approximate Hessian over a number of iterations. This approximation is updated each iteration by a matrix of low rank. In unconstrained minimization, the original quasi-Newton equation is B k+1 s k = y k , where y k is the difference of the gradients at the last two iterates. In this paper, we first propose a new quasi-Newton equation in which is decided by the sum of y k and A k s k where A k is some matrix. Then we give two choices of A k which carry some second order information from the Hessian of the objective function. The three corresponding BFGS-TYPE algorithms are proved to possess global convergence property. The superlinear convergence of the one algorithm is proved. Extensive numerical experiments have been conducted which show that the proposed algorithms are very encouraging. [Copyright &y& Elsevier]

The steepest descent method is the simplest gradient method for optimization. It is well known that exact line searches along each steepest descent direction may converge very slowly. An important result was given by Barzilar and Borwein, which is proved to be superlinearly convergent for convex quadratic in two dimensional space, and performs quite well for high dimensional problems. The BB method is not monotone, thus it is not easy to be generalized for general nonlinear functions unless certain non-monotone techniques being applied. Therefore, it is very desirable to find stepsize formulae which enable fast convergence and possess the monotone property. Such a stepsize αk for the steepest descent method is suggested in this paper. An algorithm with this new stepsize in even iterations and exact line search in odd iterations is proposed. Numerical results are presented, which confirm that the new method can find the exact solution within 3 iteration for two dimensional problems. The new method is very efficient for small scale problems. A modified version of the new method is also presented, where the new technique for selecting the stepsize is used after every two exact line searches. The modified algorithm is comparable to the Barzilar-Borwein method for large scale problems and better for small scale problems. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, we propose a nonmonotone trust region method for unconstrained optimization. Our method can be regarded as a combination of nonmonotone technique, fixed steplength and trust region method. When a trial step is not accepted, the method does not resolve the subproblem but generates a iterative point whose steplength is defined by a formula. We only allow increase in function value when trial steps are not accepted in close succession of iterations. Under mild conditions, we prove that the algorithm is global convergence and superlinear convergence. Primary numerical results are reported. [Copyright &y& Elsevier]

Abstract: Line search method is sometimes a useful and efficient technique for solving unconstrained optimization problems, especially for solving small and middle scale problems. However, A line search procedure is necessary at each iteration, leading to a significant amount of computation. In order to reduce the evaluations of objective functions and gradients, the line search procedure should be avoided in algorithm design. In this paper we propose a new descent method without line search for unconstrained optimization problems. The algorithm is required to estimate some parameters at each iteration. We analyze theoretically the global convergence of the new algorithm under mild conditions. These theoretical conclusions can help us in designing new efficient methods for optimization problems. [Copyright &y& Elsevier]

In this paper, an adaptive trust region method with nonmonotone technique for unconstrained optimization problems is presented and analyzed. The global and local convergence results of the algorithm are established. Numerical results show that the new method is efficient. [Copyright &y& Elsevier]

The paper presents a new class of memory gradient methods with inexact line searches for unconstrained minimization problems. The methods use more previous iterative information than other methods to generate a search direction and use inexact line searches to select a step-size at each iteration. It is proved that the new methods have global convergence under weak mild conditions. The convergence rate of these methods is also investigated under some special cases. Some numerical experiments show that these new algorithms converge more stably than other line search methods and are effective in solving large scale unconstrained minimization problems. [ABSTRACT FROM AUTHOR]

In this paper, a new gradient-related algorithm for solving large-scale unconstrained optimization problems is proposed. The new algorithm is a kind of line search method. The basic idea is to choose a combination of the current gradient and some previous search directions as a new search direction and to find a step-size by using various inexact line searches. Using more information at the current iterative step may improve the performance of the algorithm. This motivates us to find some new gradient algorithms which may be more effective than standard conjugate gradient methods. Uniformly gradient-related conception is useful and it can be used to analyze global convergence of the new algorithm. The global convergence and linear convergence rate of the new algorithm are investigated under diverse weak conditions. Numerical experiments show that the new algorithm seems to converge more stably and is superior to other similar methods in many situations. [Copyright &y& Elsevier]