Search

The Peaceman–Rachford splitting method (PRSM) has been widely studied in recent years because of its excellent numerical performance, especially with inertial acceleration. In this paper, we propose a multi-step inertial strictly contractive PRSM (shortly, MISCPRSM) for solving convex optimization problems, which has not been studied before. This paper aims to give an exhaustive analysis of the value relationship between multiple inertial parameters. The second subproblem utilizes an additional customized indefinite proximal term in order to obtain a closed-form solution. The global convergence and complexity result of the introduced method are analyzed by using variational inequality and basic inequalities. Finally, through simulation and computed tomography experiments, the effectiveness and robustness of the MISCPRSM are proved, and the detailed values of inertial parameters are given. [ABSTRACT FROM AUTHOR]

In this paper, we focus on the two-block nonconvex and nonsmooth optimization with linear constraints, where the objective function is the sum of a convex but nonsmooth function and a smooth but nonconvex function. This problem encompasses many important applications in engineering and machine learning. Based on the splitting algorithm, making use of the quadratic approximation of the smooth part, and with the help of Armijo line search technique, we first propose a splitting augmented Lagrangian method with partial quadratic approximation. Under some mild conditions, the global convergence of the proposed method is proved. Moreover, when the gradient of the smooth part in the objective function is Lipschitz continuous, we then propose and analyze a variant of the aforementioned method without performing line search. We report some preliminary numerical results on solving nonconvex quadratic regularization problems to show the feasibility and effectiveness of the two proposed methods. Finally, we show applicability and encouraging efficiency of our methods by applying them to solve sparse signal recovery problems. [ABSTRACT FROM AUTHOR]

In this paper, we propose an efficient trust region algorithm for unconstrained optimization problem. Different from the traditional trust region algorithm, our algorithm obtains the next iteration point by projection after getting the trust region trial step. The main feature of this algorithm is that it generates a bounded iteration sequence automatically, and it retains the excellent convergence properties. The numerical tests for support vector machine(SVM) and some unconstrained optimization test sets problems show the efficiency of the proposed algorithm. [ABSTRACT FROM AUTHOR]

In this paper, an accelerated spectral conjugate gradient projection algorithm is proposed for solving the constrained nonlinear pseudo-monotone equations. We set a restart procedure related to the conjugate parameter in the search direction of the spectral conjugate gradient method. We use an accelerated gradient descent scheme to generate the spectral parameter. Finally, we adopt the relaxed-inertial strategy to accelerated algorithm iteration process. Global convergence is proved under mild conditions without the Lipschitz continuity assumption. Numerical comparisons with other methods show the performances of our algorithm for solving large-scale equations. Our applications are on regularized decentralized logistic regression and image de-blurring problems. [ABSTRACT FROM AUTHOR]

In this paper, a hybrid three-term conjugate gradient (CG) method is proposed to solve constrained nonlinear monotone operator equations. The search direction is computed such that it is close to the direction obtained by the memoryless Broyden–Fletcher–Goldferb–Shanno (BFGS) method. Without any condition, the search direction is sufficiently descent and bounded. Moreover, based on some conditions, the search direction satisfy the conjugacy condition without using any line search. The global convergence of the method is established under mild assumptions. Comparison with existing methods is done to test the efficiency of the proposed method through some numerical experiments. Lastly, the applicability of the proposed method is shown. [ABSTRACT FROM AUTHOR]

This paper proposes two innovative vector transport operators, leveraging the Cayley transform, for the generalized Stiefel manifold embedded with a non-standard metric. Specifically, it introduces the differentiated retraction and an approximation of the Cayley transform to the differentiated matrix exponential. These vector transports are demonstrated to satisfy the Ring–Wirth non-expansive condition under non-standard metrics, and one of them is also isometric. Building upon the novel vector transport operators, we extend the modified Polak–Ribière–Polyak (PRP) conjugate gradient method to the generalized Stiefel manifold. Under a non-monotone line search condition, we prove our algorithm globally converges to a stationary point. The efficiency of the proposed vector transport operators is empirically validated through numerical experiments involving generalized eigenvalue problems and canonical correlation analysis. [ABSTRACT FROM AUTHOR]

We study a multi-block separable convex optimization problem with the linear constraints, where the objective function is the sum of m individual convex functions without overlapping variables. The linearized version of the generalized alternating direction method of multipliers (L-GADMM) is particularly efficient for the two-block separable convex programming problem and its convergence was proved when two blocks of variables are alternatively updated. However, the convergence analysis and practical applications of the extended L-GADMM (m ≥ 3) are in their early stages. In this paper, we extend this algorithm to the general case where the objective function consists of the sum of m -block convex functions. Theoretically, we prove global convergence of the new method and establish the worst-case convergence rate in the ergodic and nonergodic senses for the proposed algorithm. The efficiency of the new method is further demonstrated through numerical results on the calibration of the correlation matrices. [ABSTRACT FROM AUTHOR]

In this paper, a new descent approximate modified residual algorithm is developed to solve a large scale system of nonlinear symmetric equations, where the basic strategy to improve its numerical performance is to approximately compute the gradients and the difference of gradients. The error bounds of this approximation are presented, and in virtue of this approximation, a conjugate gradient algorithm for solving large-scale optimization problems in the literature is extended to solve the system of nonlinear symmetric equations without needs of computing and storing the Jacobian matrices or their approximate matrices. It is proved that the obtained search directions in our developed algorithm are sufficiently decent with respect to the so-called approximate modified residues. Under mild assumptions, global and local convergence results of the developed algorithm are proved. Numerical tests indicate that the developed algorithm outperforms the other similar ones available in the literature. [ABSTRACT FROM AUTHOR]

Powell (1984) and Dai (2003) constructed respectively a counterexample to show that the Polak–Ribière–Polyak (PRP) conjugate gradient algorithm fails to globally converge for nonconvex functions even when the exact line search technique is used, which implies similar failure of the weak Wolfe–Powell (WWP) inexact line search technique. Does another inexact line search technique exist that can ensure global convergence for nonconvex functions? This paper gives a positive answer to this question and proposes a modified WWP (MWWP) inexact line search technique. An algorithm is presented that uses the MWWP inexact line search technique; the next point x k + 1 generated by the PRP formula is accepted if a positive condition holds, and otherwise, x k + 1 is defined by a technique for projection onto a parabolic surface. The proposed PRP conjugate gradient algorithm is shown to possess global convergence under inexact line search for nonconvex functions. Numerical performance of the proposed algorithm is shown to be competitive with those of similar algorithms. • A convex parabolic surface for projection is defined and a projection PRP algorithm is proposed. • Any unsatisfactory iteration points are projected onto this convex parabolic surface to get good points. • The PRP conjugate gradient algorithm for general functions under inexact line search is obtained. • Competitive numerical results are reported to show the good performance of the new algorithm. [ABSTRACT FROM AUTHOR]

In this paper, an inexact augmented Lagrangian multiplier method (ALM) is designed for solving the quadratic complementarity problem (QCP). The primary goal is proposing an algorithm exploring the structures of QCPs and incorporating them into the implementation. In the implementation, an inexact proximal alternating minimization method is employed to solve the augmented Lagrangian subproblems. As a result, all the subproblems are either strongly convex quadratic programs or linear equations with positive semidefinite symmetric coefficient matrices, which in turn can either be solved with explicitly formula solutions, or by an accelerated proximal gradient method, or a conjugate gradient method. Convergence rates of these subproblems are given, and global convergence of the ALM algorithm is established under mild assumptions. Preliminary numerical experiments show that the algorithm is feasible and efficient. [ABSTRACT FROM AUTHOR]

Abstract: Recently, Fan [J. Fan, The modified Levenberg–Marquardt method for nonlinear equations with cubic convergence, Math. Comput. 81 (2012) 447–466] proposed a modified Levenberg–Marquardt (MLM) method for nonlinear equations. Using a trust region technique, global and cubic convergence of the MLM method is proved by Fan (2012) [12] under the local error bound condition, which is weaker than nonsingularity. The purpose of the paper is to investigate the convergence properties of the MLM method with a line search technique. Since the search direction of the MLM method may be not a descent direction, standard line searches can not be used directly. In this paper, we propose a nonmonotone second order Armijo line search which guarantees the global convergence of the MLM method. Moreover, we prove that the unit step will be always accepted finally. Then cubic convergence of the MLM method is preserved under the local error bound condition. Some preliminary numerical results are also reported. [Copyright &y& Elsevier]

Abstract: The linear complementarity problem is to find a vector in satisfying , ,, where and are given. In this paper, we use the fact that solving is equivalent to solving the nonlinear equation where is a function from into itself defined by . We build a sequence of smooth functions which is uniformly convergent to the function . We show that, an approximation of the solution of the (when it exists) is obtained by solving for a parameter large enough. Then we give a globally convergent hybrid algorithm which is based on vector divisions and the secant method for solving . We close our paper with some numerical simulations to illustrate our theoretical results, and to show that this method can solve efficiently large-scale linear complementarity problems. [Copyright &y& Elsevier]

Abstract The spectral conjugate gradient method is an effective method for large-scale unconstrained optimization problems. In this paper, based on Quasi-Newton direction and Quasi-Newton equation, a new spectral conjugate gradient method is proposed. This method is motivated by the three-term modified Polak-Ribiyre-Polyak (PRP) method and spectral parameters. The global convergence of algorithm is proved for general functions under a strong Wolfe line search. Numerical results show that the new algorithm is superior to the three-term modified PRP method. [ABSTRACT FROM AUTHOR]

In this paper, we propose a norm descent derivative-free algorithm for solving large-scale nonlinear symmetric equations without involving any information of the gradient or Jacobian matrix by using some approximate substitutions. The proposed algorithm is extended from an efficient three-term conjugate gradient method for solving unconstrained optimization problems, and inherits some nice properties such as simple structure, low storage requirements and symmetric property. Under some appropriate conditions, the global convergence is proved. Finally, the numerical experiments and comparisons show that the proposed algorithm is very effective for large-scale problems. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, we deal with conjugate gradient methods for solving nonlinear least squares problems. Several Newton-like methods have been studied for solving nonlinear least squares problems, which include the Gauss–Newton method, the Levenberg–Marquardt method and the structured quasi-Newton methods. On the other hand, conjugate gradient methods are appealing for general large-scale nonlinear optimization problems. By combining the structured secant condition and the idea of Dai and Liao (2001) , the present paper proposes conjugate gradient methods that make use of the structure of the Hessian of the objective function of nonlinear least squares problems. The proposed methods are shown to be globally convergent under some assumptions. Finally, some numerical results are given. [Copyright &y& Elsevier]

In this paper, a modified conjugate gradient method is proposed for nonconvex optimization. This method possesses the sufficient descent property independent of any line search. The global convergence property of the algorithm is established under the Wolfe line search strategy or the Armijo line search condition, respectively. Additionally, the complexity analysis of the proposed algorithm is investigated. To reach the point with the norm of the gradient below ɛ , the worst-case complexity bound matches that of the gradient method. Moreover, the obtained numerical results demonstrate that the modified method is effective for large-scale optimization problems and image restoration problems. [ABSTRACT FROM AUTHOR]

We present a penalty method with trust-region technique for nonlinear bilevel optimization problem in this paper. This method follows Dennis, El-Alem, and Williamson active set idea and penalty method to transform the nonlinear bilevel optimization problem to unconstrained optimization problem. This method maybe simpler than similar ideas and it does not need to compute a base of the null space. A trust-region technique is used to globalize the algorithm. Global convergence theorem is presented and applications to mathematical programs with equilibrium constraints are given. [ABSTRACT FROM AUTHOR]

We present an interior point method for nonlinear programming in this paper. This method follows Byrd and Omojokun’s idea of step decomposition, which splits the trial step into a normal step and a tangential step. The method employs a new idea of quasi-tangential subproblem, which is used to generate a tangential step that does not lie strictly on the tangent space of the constraints. Quasi-tangential subproblem is finally formulated into an unconstrained quadratic problem by penalizing the constraints. This method is different and maybe simpler than similar ideas, for example, the relaxed tangential step in trust funnel methods (Gould and Toint, 2010; Curtis, et al., 2017). Also, our method does not need to compute a base of the null space. A line search trust-funnel-like strategy is used to globalize the algorithm. Global convergence theorem is presented and applications to mathematical programs with equilibrium constraints are given. [ABSTRACT FROM AUTHOR]

Due to its simplicity and low memory requirement, conjugate gradient methods are widely used for solving large-scale unconstrained optimization problems. In this paper, we propose a three-term conjugate gradient method. The search direction is given by a symmetrical Perry matrix, which contains a positive parameter. The value of this parameter is determined by minimizing the distance of this matrix and the self-scaling memoryless BFGS matrix in the Frobenius norm. The sufficient descent property of the generated directions holds independent of line searches. The global convergence of the given method is established under Wolfe line search for general non-convex functions. Numerical experiments show that the proposed method is promising. [ABSTRACT FROM AUTHOR]

In this paper, an extended nonmonotone line search is proposed to improve the efficiency of the existing line searches. This line search is first proved to be an extension of the classical line search rules. On the one hand, under mild assumptions, global convergence and R-linear convergence are established for the new line search rule. On the other hand, by numerical experiments, it is shown that the line search can integrate the advantages of the existing methods in searching for a suitable step-size. Combined with the spectral step-size, a class of spectral gradient algorithms are developed and employed to solve a large number of benchmark test problems from CUTEst. Numerical results show that the new line search is promising in solving large-scale optimization problems, and outperforms the other similar ones as it is combined with a spectral gradient method. [ABSTRACT FROM AUTHOR]

In this paper, we study the generalized phase retrieval problem: to recover a signal x ∈ C n from the measurements y r = | 〈 a r , x 〉 | 2 , r = 1 , 2 , … , m . The problem can be reformulated as a least-squares minimization problem. Although the cost function is nonconvex, the global convergence of gradient descent algorithms from a random initialization is studied, when m is large enough. We improve the known result of the local convergence from a spectral initialization. When the signal x is real-valued, we prove that the cost function is local convex near the solution { ± x } . To accelerate the gradient descent, we review and apply several efficient line search methods with exact line search stepsize. We also perform a comparative numerical study of the line search methods and the alternative projection method. Numerical simulations demonstrate the superior ability of LBFGS algorithm than other algorithms. [ABSTRACT FROM AUTHOR]

The standard BFGS method plays an important role among the quasi-Newton algorithms for constrained/un-constrained optimization problems. However, Dai (2003) constructed a counterexample to demonstrate that this method may fail for non-convex functions with inexact Wolfe line searches, and Mascarenhas (2004) showed the nonconvergence of this method and other methods in the Broyden family, even with exact line search techniques. These works motivate us to try to find another way to obtain another quasi-Newton method with better convergence based on the standard BFGS formula. In this paper, four approaches are used in the designed algorithm: (1) a modified weak Wolfe–Powell line search technique is introduced; (2) if one defined condition is satisfied, then the search direction and its associated step length are accepted, and the next point is designed; (3) otherwise, a parabolic will be presented, and it is regarded as the projection surface to avoid using the failed direction, and the next point x k + 1 is generated by a projection technique; and (4) to easily obtain the global convergence of the proposed algorithm, the projection point is not used in the current BFGS update but rather for all the next iterations. The new algorithm possesses global convergence for general functions under the inexact modified weak Wolfe–Powell line search technique, and it is shown that other methods in the Broyden class also have this property. Numerical results are reported to show the performance of the given algorithm and other similar methods. [ABSTRACT FROM AUTHOR]

In this paper, we propose a fast inertial self-adaptive projection based algorithm for solving large-scale monotone equations with the relaxation factor. The modified hyperplane projection technique accelerates the computational performance of the proposed algorithm, and the self-adaptive parameter enhances its stability. Under some appropriate assumptions, the global convergence of the proposed algorithm is established. Moreover, the proposed algorithm is suitable to solve large-scale problems because it does not need gradient information or Jacobian matrix. The numerical results indicate that the new algorithm is robust and efficient by comparing with other popular algorithms. [ABSTRACT FROM AUTHOR]

In this paper, a method combining several recent advances is proposed for solving the tensor completion problem. In the proposed method, the ɛ -sparsity index of a matrix, the balanced matricization schemes, the ket augmentation technique, and the discrete cosine transformation are carefully integrated. An easily implementable algorithm is presented to solve the proposed method, and each subproblem can be solved either exactly or approximately by closed formula. The global convergence of the algorithm is established without any assumptions. Experiments with real data show the ability of the proposed method over several state-of-the-art methods, especially with low sample ratios. [ABSTRACT FROM AUTHOR]

In this paper, a hybrid version of the Rivaie–Mustafa–Ismail–Leong, RMIL for short, conjugate gradient method is proposed for unconstrained optimization problem. The search direction fulfills the sufficient descent condition and trust region property at each iteration, independent of any line search. We establish the global convergence of the proposed algorithm under normal assumptions and Wolfe line search. To move forward, incorporating with derivative-free projection technique, the proposed conjugate coefficient is extended to deal with nonlinear monotone equations. Without Lipschitz continuity assumption, the global convergence is proved with mild conditions. The numerical results for both unconstrained optimization instances and large-scale nonlinear equations prove the efficiency of the proposed methods with less computation cost. In particular, we also explore the practical applications on signal recovery and image denoising problems. [ABSTRACT FROM AUTHOR]

The alternating direction method of multipliers (ADMM) is an efficient splitting method for solving separable optimization with linear constraints. In this paper, an inertial proximal partially symmetric ADMM is proposed for solving linearly constrained multi-block nonconvex separable optimization, which can improve the computational efficiency by considering the ideas of inertial proximal point and linearization technique. Meanwhile, the proposed method updates the Lagrange multiplier twice and considers different relaxation factors in each iteration. Under the assumption that the generated sequence is bounded and the auxiliary function satisfies the Kurdyka-Łojasiewicz property, the global convergence of the proposed method with a more relaxed parameter range is analyzed. Moreover, some numerical results on SCAD (for smoothly clipped absolute deviation), image processing and robust PCA nonconvex problems are reported to demonstrate the efficiency and superiority of the proposed method. [ABSTRACT FROM AUTHOR]

This paper is concerned with a sequentially semidefinite programming (SSDP) algorithm for solving nonlinear semidefinite programming problems (NLSDP), which does not use a penalty function or a filter. This method, inspired by the classic SQP method, calculates a trial step by a quadratic semidefinite programming subproblem at each iteration. The trial step is determined such that either the value of the objective function or the measure of constraint violation is sufficiently reduced. In order to guarantee global convergence, the measure of constraint violation in each iteration is required not to exceed a progressively decreasing limit. We prove the global convergence properties of the algorithm under mild assumptions. We also analyze the local behaviour of the proposed method while using a second order correction strategy to avoid Maratos effect. We prove that, under the strict complementarity and the strong second order sufficient conditions with the sigma term, the rate of local convergence is superlinear. Finally, some numerical results with nonlinear semidefinite programming formulation of control design problem with the data contained in C O M P l e i b are given. [ABSTRACT FROM AUTHOR]

Spectral conjugate gradient methods are an efficient family for solving unconstrained optimization problems that have been widely studied in recent decades. In this regard, Li et al. (2019) proposed a spectral three-term conjugate gradient method and proved the global convergence of this algorithm for uniformly convex functions. Our main motivation in this paper is to develop the convergence properties of this method such that the new method possesses suitable convergence for general nonlinear functions. To do end, we introduce a modified spectral conjugate gradient method based on the CG method by Li et al. (2019). We show that the new method fulfills the sufficient descent property without any line search. The new algorithm is globally convergent for general nonlinear functions without the convexity assumption on the objective function. The numerical results indicate that the behavior of the new algorithm is not only effective, but also promising versus other conjugate gradient methods dealing with unconstrained optimization problems of the CUTEst library. [ABSTRACT FROM AUTHOR]

The exact penalty methods are very popular because of their ability to handle degenerate problems and inconsistent constraint linearizations. This paper presents a line search exact penalty method with bi-object strategy (LSBO) for nonlinear constrained optimization. In the algorithm LSBO, the penalty parameter is selected at every iteration such that the sufficient progress toward feasibility and optimality is guaranteed along the search direction. In contrast with classical exact penalization approaches, LSBO method has two goals to determine whether the current iteration is successful or not. One is improving the feasibility and the other is reducing the value of the objective function. Moreover, the penalty parameter is only related to the information at the current iterate point. The sequence of the penalty parameter is non-monotone, which does not affect the global convergence in theory and is seen to be advantageous in practice. It is shown that the algorithm enjoys favorable global convergence properties under the weaker assumptions. Numerical experiments illustrate the behavior of the algorithm on various difficult situations. [ABSTRACT FROM AUTHOR]

This paper discusses a kind of mathematical programs with equilibrium constraints (MPEC for short). By using a complementarity function and a kind of disturbed technique, the original (MPEC) problem is transformed into a nonlinear equality and inequality constrained optimization problem. Then, we combine a generalized gradient projection matrix with penalty function technique to given a generalized project metric algorithm with arbitrary initial point for the (MPEC) problems. In order to avoid Mataros effect, a high-order revised direction is obtained by an explicit formula. Under some relative weaker conditions, the proposed method is proved to possess global convergence and superlinear convergence. [ABSTRACT FROM AUTHOR]

In this paper, a conjugate gradient algorithm for systems of large-scale nonlinear equations is designed by the following steps: (i) A three-terms conjugate gradient direction d k is presented where the direction possesses the sufficient descent property and the trust region property independent of line search technique; (ii) A backtracking line search technique along the direction is proposed to get the step length α k and construct a point; (iii) If the point satisfies the given condition then it is the next point, otherwise the projection-proximal technique is used and get the next point. Both the direction and the line search technique are the derivative-free approaches, then the large-scale nonlinear equations are successfully solved (100,000 variables). The global convergence of the given algorithm is established under suitable conditions. Numerical results show that the proposed method is efficient for large-scale problems. [ABSTRACT FROM AUTHOR]

In this paper we present a new trust-region SQP algorithm for nonlinear programming. This method avoids using a penalty function, nor a filter, and instead establishes a new step acceptance mechanism. Under some reasonable assumptions, the method can be proved to be globally convergent to a KT point. Preliminary numerical experiments are presented that show the potential efficiency of the new approach. [ABSTRACT FROM AUTHOR]

In this paper, by taking a little modification to the Hestenes–Stiefel method, we propose a new way to construct descent directions satisfying the sufficient descent condition. Also, an adaptive conjugacy condition and a intrinsic self-restarting mechanism are revealed, a dynamical adjustment can be regarded as the inheritance and development of properties of standard Hestenes–Stiefel method. Furthermore, we establish global convergence for general nonconvex objective function under mild condition. Numerical results show that our presented methods can be efficient for solving large-scale test problems and therefore is promising. [ABSTRACT FROM AUTHOR]

In this paper, we consider an interior point method for nonlinear semidefinite programming. Yamashita, Yabe and Harada presented a primal–dual interior point method in which a nondifferentiable merit function was used. By using shifted barrier KKT conditions, we propose a differentiable primal–dual merit function within the framework of the line search strategy, and prove its global convergence property under weaker assumptions than the method of Yamashita, Yabe and Harada. [ABSTRACT FROM AUTHOR]

In this paper, a class of optimization problems with nonlinear inequality constraints is discussed. Based on the ideas of sequential quadratic programming algorithm and the method of strongly sub-feasible directions, a new superlinearly convergent algorithm is proposed. The initial iteration point can be chosen arbitrarily for the algorithm. At each iteration, the new algorithm solves one quadratic programming subproblem which is always feasible, and one or two systems of linear equations with a common coefficient matrix. Moreover, the coefficient matrix is uniformly nonsingular. After finite iterations, the iteration points can always enter the feasible set of the problem, and the search direction is obtained by solving one quadratic programming subproblem and only one system of linear equations. The new algorithm possesses global and superlinear convergence under some suitable assumptions without the strict complementarity. Finally, some numerical results are reported to show that the algorithm is promising. [ABSTRACT FROM AUTHOR]

In this paper, a new nonmonotone trust region algorithm with simple quadratic models is proposed. Unlike traditional nonmonotone trust region method, our trust region subproblem is very simple by using a new scale approximation of the minimizing function's Hessian. The global convergence of the proposed algorithm is established under some reasonable conditions. Numerical tests on a set of large scale standard test problems are presented and show that the new algorithm is efficient and robust. [ABSTRACT FROM AUTHOR]

Abstract: This paper is aimed to present a new sequential quadratic programming (SQP) algorithm for finding a solution to nonlinear constrained programming problems with weak conditions, where the improved direction can be yielded by solving one quadratic programming (QP), and the correction direction can be obtained by solving another QP. The main characters of the proposed algorithm are as follows. First, by limiting infeasibility of SQP iterates, the boundedness of the iteration sequence can be obtained in the case of the feasible set being nonempty and bounded as well as the constraint functions being convex. Second, global convergence can be proved under Slater constraint qualification (CQ). Furthermore, superlinear convergence can be ensured under suitable conditions. Third, the proposed algorithm is further improved with a bidirectional line search technique. Finally, some numerical experiments are operated to test the proposed algorithms, and the results demonstrate that they are promising. [Copyright &y& Elsevier]

Abstract: In this paper, we propose a proximal parallel decomposition algorithm for solving the optimization problems where the objective function is the sum of separable functions (i.e., they have no crossed variables), and the constraint set is the intersection of Cartesian products of some simple sets and a linear manifold. The subproblems are solved simultaneously per iterations, which are sum of the decomposed subproblems of the augmented Lagrange function and a quadratic term. Hence our algorithm is named as the ‘proximal parallel splitting method’. We prove the global convergence of the proposed algorithm under some mild conditions that the underlying functions are convex and the solution set is nonempty. To make the subproblems easier, some linearized versions of the proposed algorithm are also presented, together with their global convergence analysis. Finally, some preliminary numerical results are reported to support the efficiency of the new algorithms. [Copyright &y& Elsevier]

Abstract: The conjugate gradient (CG) method is one of the most popular methods for solving smooth unconstrained optimization problems due to its simplicity and low memory requirement. However, the usage of CG methods is mainly restricted to solving smooth optimization problems so far. The purpose of this paper is to present efficient conjugate gradient-type methods to solve nonsmooth optimization problems. By using the Moreau–Yosida regulation (smoothing) approach and a nonmonotone line search technique, we propose a modified Polak–Ribière–Polyak (PRP) CG algorithm for solving a nonsmooth unconstrained convex minimization problem. Our algorithm possesses the following three desired properties. (i) The search direction satisfies the sufficient descent property and belongs to a trust region automatically; (ii) the search direction makes use of not only the gradient information but also the function value information; and (iii) the algorithm inherits an important property of the well-known PRP method: the tendency to turn towards the steepest descent direction if a small step is generated away from the solution, preventing a sequence of tiny steps from happening. Under standard conditions, we show that the algorithm converges globally to an optimal solution. Numerical experiment shows that our algorithm is effective and suitable for solving large-scale nonsmooth unconstrained convex optimization problems. [Copyright &y& Elsevier]

Abstract: In this paper, based on the spectral-scaling secant condition [W.Y. Cheng, D.H. Li, Spectral-scaling BFGS method, Journal of Optimization Theory and Applications, 146 (2010) 305–319], we propose spectral-scaling one parameter Broyden family methods which allow for negative values of the parameter. We show that the proposed methods possess some good properties such as quadratic termination property and single-step convergence rate not inferior to that of the steepest descent method when minimizing an -dimensional quadratic function. Under appropriate conditions, we establish the global convergence of the proposed methods for uniformly convex functions. Numerical results from problems in the CUTE test set show that the proposed methods are promising. [Copyright &y& Elsevier]

Abstract: In this paper we propose a modified regularized Newton method for convex minimization problems whose Hessian matrices may be singular. The proposed method is proved to converge globally if the gradient and Hessian of the objective function are Lipschitz continuous. Under the local error bound condition, we first show that the method converges quadratically, which implies that is equivalent to , where is the solution set and . Then we in turn prove the cubic convergence of the proposed method under the same local error bound condition, which is weaker than nonsingularity. [Copyright &y& Elsevier]

Abstract: Conjugate gradient methods have been paid attention to, because they can be directly applied to large-scale unconstrained optimization problems. In order to incorporate second order information of the objective function into conjugate gradient methods, Dai and Liao (2001) proposed a conjugate gradient method based on the secant condition. However, their method does not necessarily generate a descent search direction. On the other hand, Hager and Zhang (2005) proposed another conjugate gradient method which always generates a descent search direction. In this paper, combining Dai–Liao’s idea and Hager–Zhang’s idea, we propose conjugate gradient methods based on secant conditions that generate descent search directions. In addition, we prove global convergence properties of the proposed methods. Finally, preliminary numerical results are given. [Copyright &y& Elsevier]

Abstract: In this paper, we investigate a class of nonlinear complementarity problems arising from the discretization of the free boundary problem, which was recently studied by Sun and Zeng [Z. Sun, J. Zeng, A monotone semismooth Newton type method for a class of complementarity problems, J. Comput. Appl. Math. 235 (5) (2011) 1261–1274]. We propose a new non-interior continuation algorithm for solving this class of problems, where the full-Newton step is used in each iteration. We show that the algorithm is globally convergent, where the iteration sequence of the variable converges monotonically. We also prove that the algorithm is globally linearly and locally superlinearly convergent without any additional assumption, and locally quadratically convergent under suitable assumptions. The preliminary numerical results demonstrate the effectiveness of the proposed algorithm. [Copyright &y& Elsevier]

Abstract: In this paper, a constraint shifting combined homotopy method for solving multi-objective programming problems with both equality and inequality constraints is presented. It does not need the starting point to be an interior point or a feasible point and hence is convenient to use. Under some assumptions, the existence and convergence of a smooth path to an efficient solution are proven. Simple numerical results are given. [Copyright &y& Elsevier]

Abstract: This paper concerns a filter technique and its application to the trust region method for nonlinear programming (NLP) problems. We used our filter trust region algorithm to solve NLP problems with equality and inequality constraints, instead of solving NLP problems with just inequality constraints, as was introduced by Fletcher et al. [R. Fletcher, S. Leyffer, Ph.L. Toint, On the global converge of an SLP-filter algorithm, Report NA/183, Department of Mathematics, Dundee University, Dundee, Scotland, 1999]. We incorporate this filter technique into the traditional trust region method such that the new algorithm possesses nonmonotonicity. Unlike the tradition trust region method, our algorithm performs a nonmonotone filter technique to find a new iteration point if a trial step is not accepted. Under mild conditions, we prove that the algorithm is globally convergent. [Copyright &y& Elsevier]

Abstract: Recently, Hager and Zhang (2005) proposed a new conjugate gradient method which generates sufficient descent direction , this property is independent of the line search used. In this paper, we take a modification of this method, such that the sufficient descent direction satisfies , this property is also independent of the line search used. Under appropriate conditions, we prove that the proposed method is globally convergent. Moreover, we give a sufficient condition for the global convergence of the proposed general method. The numerical results show that the proposed method is efficient. [Copyright &y& Elsevier]

Abstract: In this paper, we consider a multivariate spectral projected gradient (MSPG) method for bound constrained optimization. Combined with a quasi-Newton property, the multivariate spectral projected gradient method allows an individual adaptive step size along each coordinate direction. On the basis of nonmonotone line search, global convergence is established. A numerical comparison with the traditional SPG method shows that the method is promising. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, we propose a trust region method for unconstrained optimization that can be regarded as a combination of conic model, nonmonotone and line search techniques. Unlike in traditional trust region methods, the subproblem of our algorithm is the conic minimization subproblem; moreover, our algorithm performs a nonmonotone line search to find the next iteration point when a trial step is not accepted, instead of resolving the subproblem. The global and superlinear convergence results for the algorithm are established under reasonable assumptions. Numerical results show that the new method is efficient for unconstrained optimization problems. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, we give some notes on the two modified spectral gradient methods which were developed in . These notes present the relationship between their stepsize formulae and some new secant equations in the quasi-Newton method. In particular, we also introduce another two new choices of stepsize. By using an efficient nonmonotone line search technique, we propose some new spectral gradient methods. Under some mild conditions, we show that these proposed methods are globally convergent. Numerical experiments on a large number of test problems from the CUTEr library are also reported, which show that the efficiency of these proposed methods. [Copyright &y& Elsevier]

Abstract: In this paper, we propose two derivative-free iterative methods for solving nonlinear monotone equations, which combines two modified HS methods with the projection method in Solodov and Svaiter (1998) . The proposed methods can be applied to solve nonsmooth equations. They are suitable to large-scale equations due to their lower storage requirement. Under mild conditions, we show that the proposed methods are globally convergent. The reported numerical results show that the methods are efficient. [Copyright &y& Elsevier]