Search

A modified Polak Ribiere Polyak(PRP) conjugate gradient(CG) method is proposed for solving unconstrained optimization problems. The search direction generated by this method satisfies sufficient descent condition at each iteration and this method inherits one remarkable property of the standard PRP method. Under the standard Armijo line search, the global convergence and the linearly convergent rate of the presented method is established. Some numerical results are given to show the effectiveness of the proposed method by comparing with some existing CG methods. [ABSTRACT FROM AUTHOR]

The conjugate gradient method plays a very important role in several fields, to solve problems of large sizes. To improve the efficiency of this method, a lot of works have been done; in this paper we propose a new modification of PRP method to solve a large scale unconstrained optimization problems in relation with strong Wolf Powell Line Search property, when the latter was used under some conditions, a global convergence result was proved. In comparison with other known methods the efficiency of this method proved that it is better in the number of iterations and in time on 90 proposed problems by use of Matlab. [ABSTRACT FROM AUTHOR]

This paper is devoted to the analysis of worst case complexity bounds for linesearch-type derivative-free algorithms for the minimization of general non-convex smooth functions. We consider a derivative-free algorithm based on a linesearch extrapolation technique. First we prove that it enjoys the same complexity properties which have been proved for pattern and direct search algorithms, namely that the number of iterations and of function evaluations required to drive the norm of the gradient of the objective function below a given threshold ϵ for the first time is O (ϵ - 2) in the worst case. This is the first contribution proving worst-case complexity properties for derivative-free linesearch-type algorithms. Then we show that the lineasearch approach used by the described algorithm allows us to guarantee the further property that the number of iterations such that the norm of the gradient is bigger than ϵ is O (ϵ - 2) in the worst case. [ABSTRACT FROM AUTHOR]

In this study, we introduce a novel hybrid conjugate gradient method with an adaptive strategy called asHCG method. The asHCG method exhibits the following characteristics. (i) Its search direction guarantees sufficient descent property without dependence on any line search. (ii) It possesses strong convergence for the uniformly convex function using a weak Wolfe line search, and under the same line search, it achieves global convergence for the general function. (iii) Employing the Armijo line search, it provides an approximate guarantee for worst-case complexity for the uniformly convex function. The numerical results demonstrate promising and encouraging performances in both unconstrained optimization problems and image restoration problems. [ABSTRACT FROM AUTHOR]

Copyright of Wuhan University Journal of Natural Sciences is the property of Wuhan University and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

We investigated the challenge of unconstrained distributed optimization with a time-varying objective function, employing a prediction-correction approach. Our method introduced a backward Euler prediction step that used the differential information from consecutive moments to forecast the trajectory's future direction. This predicted value was then refined through an iterative correction process. Our analysis and experimental results demonstrated that this approach effectively addresses the optimization problem without requiring the computation of the Hessian matrix's inverse. [ABSTRACT FROM AUTHOR]

Copyright of Baghdad Science Journal is the property of Republic of Iraq Ministry of Higher Education & Scientific Research (MOHESR) and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Conjugate gradient (CG) methods are a popular class of iterative methods for solving linear systems of equations and nonlinear optimization problems. Motivated by the construction of some modern CG methods, we propose two modified CG methods, named DHS* and DPRP*. Under the strong Wolfe line search (SWLS), the two presented methods are proven to be sufficient descent and globally convergent. Preliminary numerical results show that the DHS* and DPRP* methods are effective for the given test problems. [ABSTRACT FROM AUTHOR]

The conjugate gradient (CG) directions are among the important components of the CG algorithms. These directions have proven their effectiveness in many applications—more specifically, in image processing due to their low memory requirements. In this study, we derived a new conjugate gradient coefficient based on the famous quadratic model. The derived algorithm is distinguished by its global convergence and essential descent properties, ensuring robust performance across diverse scenarios. Extensive numerical testing on image restoration and unconstrained optimization problems have demonstrated that the new formulas significantly outperform existing methods. Specifically, the proposed conjugate gradient scheme has shown superior performance compared to the traditional Fletcher–Reeves (FR) conjugate gradient method. This advancement not only enhances computational efficiency on unconstrained optimization problems, but also improves the accuracy and quality of image restoration, making it a highly valuable tool in the field of computational imaging and optimization. [ABSTRACT FROM AUTHOR]

As we know, the stepsize is extremely crucial to gradient method. A new type of stepsize is introduced for the gradient method in the paper, which is generated by minimizing the norm of the approximate model of the gradient along the line of negative gradient direction. Based on the retard technique, we present a new gradient method by minimizing adaptively the approximate model of the objective function and the norm of the approximate model of the gradient along the line of negative gradient direction for strictly convex quadratic minimization. The convergence of the proposed method is established. The numerical experiments on four groups of strictly convex quadratic minimization problems illustrate that the proposed method is very promising. We also extend the new gradient method for convex quadratic minimization to general unconstrained optimization by incorporating a nonmonotone line search. The convergence of the resulting method is established. The numerical experiment on the 147 test functions from the CUTEst library indicates that the resulting method is superior to some efficient gradient methods including the BBQ method (SIAM J Optim. 2021;31(4):3068–3096) and is competitive to two famous conjugate gradient software packages CGOPT (1.0) and CG $ \_ $ _ DESCENT (5.0). [ABSTRACT FROM AUTHOR]

Over the years, a considerable number of conjugate gradient methods have been proposed based on modifications on the well-known classical conjugate gradient methods. These methods were shown to have satisfied descent condition taking into consideration the strong Wolfe line search and other line search schemes. Convergence of objective functions were also guarantied. In this study, a decent conjugate gradient method for solving unconstrained non-linear optimization problems is developed. Algorithm of the proposed method was well developed by constructing its update parameter. Descent properties of the method based on some assumptions on the objective function were established. The convergence analysis of the method showed that it converges globally taking into consideration the strong Wolfe conditions . Dolan and More performance profile was used to compare the numerical strength of this method with other methods, showing clear evidence of better performance of the new method in the profiles tested. [ABSTRACT FROM AUTHOR]

Adaptive cubic regularization (ARC) methods for unconstrained optimization compute steps from linear systems involving a shifted Hessian in the spirit of the Levenberg-Marquardt and trust-region methods. The standard approach consists in performing an iterative search for the shift akin to solving the secular equation in trust-region methods. Such search requires computing the Cholesky factorization of a tentative shifted Hessian at each iteration, which limits the size of problems that can be reasonably considered. We propose a scalable implementation of ARC named ARC q K in which we solve a set of shifted systems concurrently by way of an appropriate modification of the Lanczos formulation of the conjugate gradient (CG) method. At each iteration of ARC q K to solve a problem with n variables, a range of m ≪ n shift parameters is selected. The computational overhead in CG beyond the Lanczos process is thirteen scalar operations to update five vectors of length m , and two n -vector updates for each value of the shift. The CG variant only requires one Hessian-vector product and one dot product per iteration, independently of m . Solves corresponding to inadequate shift parameters are interrupted early. All shifted systems are solved inexactly. Such modest cost makes our implementation scalable and appropriate for large-scale problems. We provide a new analysis of the inexact ARC method including its worst case evaluation complexity, global and asymptotic convergence. We describe our implementation and report numerical experience that confirms that our implementation of ARC q K outperforms a classic Steihaug-Toint trust-region method, and the ARC method of the GALAHAD library. The latter solves the subproblem in nested Krylov subspaces by a Lanczos-based method, which requires the storage of a dense matrix that can be comparable to or larger than the two dense arrays required by our approach if the problem is large or requires many Lanczos iterations. Finally, we generalize our convergence results to inexact Hessians and nonlinear least-squares problems. [ABSTRACT FROM AUTHOR]

In this paper, an augmented memoryless BFGS quasi-Newton method was proposed for solving unconstrained optimization problems. Based on a new modified secant equation, an augmented memoryless BFGS update formula and an efficient optimization algorithm were established. To improve the stability of the numerical experiment, we obtained the scaling parameter by minimizing the upper bound of the condition number. The global convergence of the algorithm was proved, and numerical experiments showed that the algorithm was efficient. [ABSTRACT FROM AUTHOR]

We investigated the challenge of unconstrained distributed optimization with a time-varying objective function, employing a prediction-correction approach. Our method introduced a backward Euler prediction step that used the differential information from consecutive moments to forecast the trajectory's future direction. This predicted value was then refined through an iterative correction process. Our analysis and experimental results demonstrated that this approach effectively addresses the optimization problem without requiring the computation of the Hessian matrix's inverse.

Introduction. Methods of unconstrained optimization play a significant role in machine learning [1–6]. When solving practical problems in machine learning, such as tuning nonlinear regression models, the extremum point of the chosen optimality criterion is often degenerate, which significantly complicates its search. Therefore, degenerate problems are the most challenging in optimization. Known numerical methods for solving the general unconstrained optimization problem, up to the second order, have very low convergence rates when solving degenerate problems [7, 8]. This is explained by the fact that for a significant improvement in convergence rate in this case, it is necessary to use higher-order derivatives in the method than the second order [10]. The purpose of the paper is to develop an efficient quasi-Newton method for solving degenerate unconstrained optimization problems, the idea of which (unlike regularization) involves dividing the entire space into the sum of two orthogonal subspaces. This idea was introduced in [23]. The space division at each iteration of the method is based on the spectral decomposition of the matrix approximating the Hessian of the objective function using the BFGS formula [3]. Each subspace exhibits its own behavior of the objective function, and therefore, an appropriate minimization method is applied on it. Results. A combined quasi-Newton method is presented for solving degenerate unconstrained optimization problems, based on orthogonal decomposition of the Hessian approximation matrix and division of the entire space into the sum of two orthogonal subspaces. On one subspace (the kernel of the Hessian approximation matrix), a method is applied where derivatives in the direction of the 4th order are computed, while on the orthogonal complement to it, a quasi-Newtonian method is applied. A separate one-dimensional search is applied on each of these subspaces to determine the step multiplier in the respective direction. The effectiveness of the presented combined method is confirmed by numerical experiments conducted on widely accepted test functions for unconstrained optimization problems. The proposed method allows obtaining fairly accurate solutions to test tasks in case of degeneracy of the minimum point with significantly lower computational costs for gradient calculations compared to optimization procedures of well-known mathematical packages. Conclusions. The idea of dividing the entire space into the sum of two (or possibly more) orthogonal subspaces when solving complex optimization problems is quite promising in terms of applying a combination of different numerical methods on separate subspaces. In the future, it is planned to conduct theoretical research on the convergence rate of the presented combined method in solving degenerate unconstrained optimization problems.

The Barzilai–Borwein method offers efficient step sizes for large-scale un-constrained optimization problems. However, it may not guarantee global convergence for nonquadratic objective functions. Simulated annealing-based on Barzilai–Borwein (SABB) method addresses this issue by in-corporating a simulated annealing rule. This work proposes a novel step-size strategy for the SABB method, referred to as the SABBm method. Furthermore, we introduce two stabilized variants: SABBstab and SABBmstab. SABBstab combines a simulated annealing rule with a sta-bilization step to ensure convergence. SABBmstab builds upon SABBstab, incorporating the modified step size derived from the SABBm method. The effectiveness and competitiveness of the proposed methods are demon-strated through numerical experiments on CUTEr benchmark problems.

Conjugate gradient minimization methods (CGM) and their accelerated variants are widely used. We focus on the use of cubic regularization to improve the CGM direction independent of the step length computation. In this paper, we propose the Hybrid Cubic Regularization of CGM, where regularized steps are used selectively. Using Shanno's reformulation of CGM as a memoryless BFGS method, we derive new formulas for the regularized step direction. We show that the regularized step direction uses the same order of computational burden per iteration as its non-regularized version. Moreover, the Hybrid Cubic Regularization of CGM exhibits global convergence with fewer assumptions. In numerical experiments, the new step directions are shown to require fewer iteration counts, improve runtime, and reduce the need to reset the step direction. Overall, the Hybrid Cubic Regularization of CGM exhibits the same memoryless and matrix-free properties, while outperforming CGM as a memoryless BFGS method in iterations and runtime. [ABSTRACT FROM AUTHOR]

Abstract In this paper, a new appropriate diagonal matrix estimation of the Hessian is introduced by minimizing the Byrd and Nocedal function subject to the weak secant equation. The Hessian estimate is used to correct the framework of a nonmonotone trust region algorithm with the regularized quasi-Newton method. Moreover, to counteract the adverse effect of monotonicity, we introduce a new nonmonotone strategy. The global and superlinear convergence of the suggested algorithm is established under some standard conditions. The numerical experiments on unconstrained optimization test functions show that the new algorithm is efficient and robust.

Conjugate gradient algorithms come in a wide range of flavors. The conjugate gradient technique primarily concentrates on the spectral parameter. It follows the standard method proposed by Hastens and Stiefel, In this study, we have devised an innovative approach to spectral conjugate gradient methods we get a new direction conjugate gradient method to solve unconstrained optimization problems, which is based on non-linear function using an inexact line searching introduced a novel direction. In specific scenarios, this groundbreaking direction not only guarantees global convergence but also ensures a downward trajectory. Our numerical experiments unequivocally demonstrate that when compared to traditional CG techniques, depending on the number of functions (NOF), the number of iterations (NOI), and time (CPU), and evaluated using the Dolan-More performance profile, our novel method consistently exhibits superior performance across a diverse set of unconstrained function minimization test. and the convergence condition under some Hypotheses by using a strong -Wolfe line search.

In this work, for unconstrained optimization, we introduce an improved Dai‐Liao‐style hybrid conjugate gradient method based on the hybridization‐based self‐adaptive technique, and the search direction generated fulfills the sufficient descent and trust region properties regardless of any line search. The global convergence is established under standard Wolfe line search and common assumptions. Then, combining the hyperplane projection technique and a new self‐adaptive line search, we extend the proposed conjugate gradient method and obtain an improved Dai‐Liao‐style hybrid conjugate gradient projection method to solve constrained nonlinear monotone equations. Under mild conditions, we obtain its global convergence without Lipschitz continuity. In addition, the convergence rates for the two proposed methods are analyzed, respectively. Finally, numerical experiments are conducted to demonstrate the effectiveness of the proposed methods. [ABSTRACT FROM AUTHOR]

Due to their simplicity and global convergence properties, the conjugate gradient (CG) methods are widely used for solving unconstrained optimization problems, especially those of large scale. To establish the global convergence and to obtain better numerical performance in practice, much effort has been devoted to develop new CG methods or even to modify well- known methods. In 2012, Rivaie et al., have proposed a new CG method, called RMIL which has good numerical results and globally convergent under the exact line search. However, in 2016, Dai has pointed out a mistake in the steps of the proof of global convergence of RMIL and hence to guarantee the global convergence he suggested a modified version of RMIL, called RMIL+. In this paper, we present another modified version of RMIL, which is globally convergent via the exact line search. Furthermore, to support the theoretical proof of the global convergence of the modified version in practical computation, a numerical experiment based on comparing it with RMIL, RMIL+, and CG-DESCENT was done. [ABSTRACT FROM AUTHOR]

The Fletcher-Reeves (FR) method is widely recognized for its drawbacks, such as generating unfavorable directions and taking small steps, which can lead to subsequent poor directions and steps. To address this issue, we propose a modification to the FR method, and then we develop it into the three-term conjugate gradient method in this paper. The suggested methods, named "HZF" and "THZF", preserve the descent property of the FR method while mitigating the drawbacks. The algorithms incorporate strong Wolfe line search conditions to ensure effective convergence. Through numerical comparisons with other conjugate gradient algorithms, our modified approach demonstrates superior performance. The results highlight the improved efficacy of the HZF algorithm compared to the FR and three-term FR conjugate gradient methods. The new algorithm was applied to the problem of image restoration and proved to be highly effective in image restoration compared to other algorithms. [ABSTRACT FROM AUTHOR]

We present a new diagonal quasi-Newton method for solving unconstrained optimization problems based on the weak secant equation. To control the diagonal elements, the new method uses new criteria to generate the Hessian approximation. We establish the global convergence of the proposed method with the Armijo line search. Numerical results on a collection of standard test problems demonstrate the superiority of the proposed method over several existing diagonal methods. [ABSTRACT FROM AUTHOR]

Unconstrained optimization problems have a long history in computational mathematics and have been identified as being among the crucial problems in the fields of applied sciences, engineering, and management sciences. In this paper, a new variant of the conjugate descent method for solving unconstrained optimization problems is introduced. The proposed algorithm can be seen as a modification of the popular conjugate descent (CD) algorithm of Fletcher. The algorithm of the proposed method is well-defined, and the sequence of the directions of search is shown to be sufficiently descending. The convergence result of the proposed method is discussed under the common standard conditions. The proposed algorithm together with some existing ones in the literature is implemented to solve a collection of benchmark test problems. Numerical experiments conducted show the performance of the proposed method is very encouraging. Furthermore, an additional efficiency evaluation is carried out on problems arising from signal processing and it works well. [ABSTRACT FROM AUTHOR]

In this paper, a family of hybrid conjugate parameters with restart procedure is proposed. In which, we design a hybrid conjugate parameter by using two hybrid techniques, and set a restart procedure in its search direction according to the conjugate parameter. Under the usual assumptions and the weak Wolfe line search, we prove its sufficient descent property and global convergence. Finally, we choose a specific algorithm from this family and use an acceleration scheme to solve the large-scale unconstrained optimization problems and image restorations. These numerical results show that our algorithm is effective. [ABSTRACT FROM AUTHOR]

In this paper, a new appropriate diagonal matrix estimation of the Hessian is introduced by minimizing the Byrd and Nocedal function subject to the weak secant equation. The Hessian estimate is used to correct the framework of a nonmonotone trust region algorithm with the regularized quasi-Newton method. Moreover, to counteract the adverse effect of monotonicity, we introduce a new nonmonotone strategy. The global and superlinear convergence of the suggested algorithm is established under some standard conditions. The numerical experiments on unconstrained optimization test functions show that the new algorithm is efficient and robust. [ABSTRACT FROM AUTHOR]

In this article, we design an optimal neural network based on new LM training algorithm. The traditional algorithm of LM required high memory, storage and computational overhead because of it required the updated of Hessian approximations in each iteration. The suggested design implemented to converts the original problem into a minimization problem using feed forward type to solve non-linear 3D - PDEs. Also, optimal design is obtained by computing the parameters of learning with highly precise. Examples are provided to portray the efficiency and applicability of this technique. Comparisons with other designs are also conducted to demonstrate the accuracy of the proposed design. [ABSTRACT FROM AUTHOR]

Conjugate gradient methods are an important class of methods for unconstrained optimization, especially for large-scale problems. Recently, they have been much studied. In this paper, a new hybrid conjugate gradient algorithm is proposed and analyzed. The proposed method inherits the features of the HS, DY and NHS conjugate gradient methods. The method can generate the descent direction at every iteration, moreover, this property doesn't depend on any line search. Under the strong Wolfe line search, the global convergence of the proposed method is established. The numerical results also show the feasibility and effectiveness of our algorithm. Furthermore, the proposed algorithm EHD was extended to solve problem of mode function. [ABSTRACT FROM AUTHOR]

The conjugate gradient method is one of the most effective algorithm for unconstrained nonlinear optimization problems. This is due to the fact that it does not need a lot of storage memory and its simple structure properties, which motivate us to propose a new hybrid conjugate gradient method through a convex combination of βκRMIL and βκHS. We compute the convex parameter θκ using the Newton direction. Global convergence is established through the strong Wolfe conditions. Numerical experiments show the superior efficiency of our algorithm to solve unconstrained optimization problem compared to other considered methods. Applied to image restoration problem, our algorithm is competitive with existing algorithms and performs even better when the level of noise in the image is significant. [ABSTRACT FROM AUTHOR]

This paper introduces CLS, a new line search along an arbitrary smooth search path, that starts at the current iterate tangentially to a descent direction. Like the Goldstein line search and unlike the Wolfe line search, the new line search uses, beyond the gradient at the current iterate, only function values. Using this line search with search directions satisfying the bounded angle condition, global convergence to a stationary point is proved for continuously differentiable objective functions that are bounded below and have Lipschitz continuous gradients. The standard complexity bounds are proved under several natural assumptions. [ABSTRACT FROM AUTHOR]

This paper presents a new hybrid conjugate gradient method for solving nonlinear unconstrained optimization problems; it is based on a combination of RMIL (Rivaie-Mustafa-Ismail-Leong) and hSM (hybrid Sulaiman-Mohammed) methods. The proposed algorithm enjoys the sufficient descent condition without depending on any line search; moreover, it is globally convergent under the usual and strong Wolfe line search assumptions. The performance of the algorithm is demonstrated through numerical experiments on a set of 100 test functions from [1] and four image restoration problems with two noise levels. The numerical comparisons with four existing methods show that the proposed method is promising and effective. [ABSTRACT FROM AUTHOR]

The Barzilai-Borwein method offers efficient step sizes for large-scale unconstrained optimization problems. However, it may not guarantee global convergence for nonquadratic objective functions. Simulated annealingbased on Barzilai-Borwein (SABB) method addresses this issue by incorporating a simulated annealing rule. This work proposes a novel step-size strategy for the SABB method, referred to as the SABBm method. Furthermore, we introduce two stabilized variants: SABBstab and SABBmstab. SABBstab combines a simulated annealing rule with a stabilization step to ensure convergence. SABBmstab builds upon SABBstab, incorporating the modified step size derived from the SABBm method. The effectiveness and competitiveness of the proposed methods are demonstrated through numerical experiments on CUTEr benchmark problems. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Education & Science is the property of Republic of Iraq Ministry of Higher Education & Scientific Research (MOHESR) and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

The Conjugate Gradient (CG) method is a powerful iterative approach for solving large-scale minimization problems, characterized by its simplicity, low computation cost and good convergence. In this paper, a new hybrid conjugate gradient HLB method (HLB: Hadji-Laskri-Bechouat) is proposed and analysed for unconstrained optimization. We compute the parameter βkHLB as a convex combination of the Polak-Ribière-Polyak (βkPRP)[1] and the Mohd Rivaie-Mustafa Mamat and Abdelrhaman Abashar (βkRMIL+) i.e βkHLB=(1−θk)βkPRP+θkβkRMIL+ . By comparing numerically CGHLB with PRP and RMIL+ and by using the Dolan and More CPU performance, we deduce that CGHLB is more efficient. [ABSTRACT FROM AUTHOR]

Image noise removal is one of the important research directions in the field of image processing, and it can be transformed into a large scale unconstrained optimization problem. Based on the famous two-phase method, in this paper we propose an adaptive conjugate gradient method to remove the impulse noise from the corrupted images by solving the transformed unconstrained optimization problem, which is defined in terms of an edge-preserving potential function. The proposed method can generate a sufficient descent search direction at each step independent of line search condition, and it has global convergence under the Armijo line search condition. Experimental results show that the proposed method is efficient and robust for impulse noise removal. [ABSTRACT FROM AUTHOR]

Based on the augmented version of the quasi-Newton method proposed by Aminifard et al. (App. Num. Math. 167:187–201, 2021), a new scaled parameter of the self-scaling memoryless BFGS update formula is proposed. The idea is to cluster the eigenvalues of the search direction matrix, obtained by minimizing the difference between the largest and the smallest eigenvalues of the matrix. The sufficient descent property is proved for uniformly convex functions, and the global convergence of the proposed algorithm is proved both for the uniformly convex and general nonlinear objective functions. Numerical experiments on a set of test functions of the CUTEr collection show that the proposed method is efficient. In addition, the proposed algorithm is effectively applied to salt and pepper noise elimination problem. [ABSTRACT FROM AUTHOR]

Purpose: One of the most effective methods for solving unconstrained optimization problems is the trust region method. The strategy of determining the radius of the trust region has a significant effect on the efficiency of this method. On the other hand, imposing the monotonicity condition will decrease the convergence speed of this method. Therefore, improving and increasing the efficiency of this method is one of the most important issues and the attention of researchers. Methodology: Establishing a new adaptive trust region radius as well as combining the trust region method with a non- monotone strategy to avoid the adverse effects of monotonocity. Findings: A new adaptive trust region radius converged to zero is provided, and then a trust region combination is performed using a non-monotone strategy. Running the algorithm on a set of test functions shows that the new adaptive radius, along with the non-monotone strategy used, significantly improves the efficiency of the trust region method. Originality/Value: The presented non-monotone adaptive algorithm has a second-order convergence rate. In addition, it significantly reduces computational costs compared to traditional algorithms. On the other hand, the new adaptive radius avoids the ineffectiveness of the trust region close to the solution. [ABSTRACT FROM AUTHOR]

Nonmonotonicity has been considered to be essentially influential on the efficiency of the iterative procedures of nonlinear optimization. A review of the literature shows that the objective function values available from recent iterations provide worthier information in the nonmonotone schemes. So, with the aim of enhancing probability of applying more recent available function values, a nonmonotone trust region ratio is suggested using a forgetting factor. Meanwhile, modification of a recent adaptive formula for the trust region radius is devised by a nonmonotone reflection as well. Then, based on the two mentioned modifications, an adaptive nonmonotone trust region algorithm is given. In addition, convergence of the method is analysed under classic assumptions. To provide support for our theoretical arguments, computational merits of the given algorithm on a set of CUTEr test functions are depicted. [ABSTRACT FROM AUTHOR]

In this paper, we discuss two efficient three-term conjugate gradient methods (ECG) for impulse noise removal. The directions of ECG are first the direction of steepest descent and then spanned by the three terms: The steepest descent direction, the previous direction, and the gradient differences at the previous and current points. The second and third terms are scaled by two different step sizes called conjugate gradient parameters. Our goal is to generate and control these parameters such that they do not jointly dominate while preserving the effect of all terms, except near the optimizer where the first term dominates the other two terms. They are independent of the line search method and useful for finite precision arithmetic. The global convergence of ECG is proved. The efficiency (the lowest relative cost of function evaluations) and robustness (highest number of solved problems) of ECG compared to known conjugate gradient methods are shown in terms of PSNR (peak signal noise ratio) and time in seconds. [ABSTRACT FROM AUTHOR]

The Barzilai-Borwein (BB) steplengths play great roles in practical gradient methods for solving unconstrained optimization problems. Motivated by the observation that the two well-known BB steplengths correspond to the ordinary and the data least squares, respectively, we introduce a novel family of BB steplengths from the viewpoint of scaled total least squares. Numerical experiments demonstrate that high performance can be received by a carefully-selected BB steplength in the new family. [ABSTRACT FROM AUTHOR]

This paper proposes an adaptive two-parameter accelerated conjugate gradientmethod, which satisfies the sufficient descent condition in the search direction. The Powell restart strategy is designed in the algorithm to improve its numerical performance. Our proposed method does not add extra computational effort compared to other methods. Furthermore, under general assumptions, we demonstrate the global convergence of our proposed method under theWolfe line search. Finally, we compare with other methods on the unconstrained optimization and image restoration problems. Numerical experiments are presented to show that our proposed method is feasible. [ABSTRACT FROM AUTHOR]

Particle Swarm Optimization algorithm (PSO) has been widely utilized for addressing optimization problems due to its straightforward implementation and efficiency in tackling various test functions and engineering optimization problems. Nevertheless, PSO encounters issues like premature convergence and a lack of diversity, particularly when confronted with complex high-dimensional optimization tasks. In this study, we propose an enhanced version of the Island Model Particle Swarm Optimization (IMPSO), where island models are integrated into the PSO algorithm based on several migration strategies. The first contribution consists in applying a new selection and replacement strategies based on tabu search technique, while the second contribution consists in proposing a dynamic migration rate relying on the Biogeography-Based Optimization technique. To assess and validate the effectiveness of the proposed method, several unconstrained benchmark functions are applied. The obtained results confirm that the approach yield better performance than the old version of IMPSO for solving NP-hard optimization problems. Compared to the performance of other well-known evolutionary algorithms, the proposed approach is more efficient and effective. [ABSTRACT FROM AUTHOR]

The traditional BFGS algorithm has been proved very efficient. It is convergent for convex nonlinear optimization problems. However, for non-convex nonlinear optimization problems, it is known that the BFGS algorithm may not be convergent. This paper proposes a robust BFGS algorithm in the sense that the algorithm superlinearly converges to a local minimum under some mild assumptions for both convex and non-convex nonlinear optimization problems. Numerical test on the CUTEst test set is reported to demonstrate the merit of the proposed robust BFGS algorithm. This result shows that the robust BFGS algorithm is very efficient and effective. [ABSTRACT FROM AUTHOR]

The Conjugate Gradient (CG) method is renowned for its rapid convergence in optimization applications. Over the years, several modifications to CG methods have emerged to improve computational efficiency and tackle practical challenges. This paper presents a new three-term hybrid CG method for solving unconstrained optimization problems. This algorithm utilizes a search direction that combines Liu-Storey (LS) and Conjugate Descent (CD) CG coefficients and standardizes it using a spectral which acts as a scheme for the choices of the conjugate parameters. This resultant direction closely approximates the memoryless Broyden-Fletcher-Goldfarb-Shanno (BFGS) quasi-Newton direction, known for its bounded nature and compliance with the sufficient descent condition. The paper establishes the global convergence under standard Wolfe conditions and some appropriate assumptions. Additionally, the numerical experiments were conducted to emphasize the robustness and superior efficiency of this hybrid algorithm in comparison to existing approaches. [ABSTRACT FROM AUTHOR]