Search

As we know, the stepsize is extremely crucial to gradient method. A new type of stepsize is introduced for the gradient method in the paper, which is generated by minimizing the norm of the approximate model of the gradient along the line of negative gradient direction. Based on the retard technique, we present a new gradient method by minimizing adaptively the approximate model of the objective function and the norm of the approximate model of the gradient along the line of negative gradient direction for strictly convex quadratic minimization. The convergence of the proposed method is established. The numerical experiments on four groups of strictly convex quadratic minimization problems illustrate that the proposed method is very promising. We also extend the new gradient method for convex quadratic minimization to general unconstrained optimization by incorporating a nonmonotone line search. The convergence of the resulting method is established. The numerical experiment on the 147 test functions from the CUTEst library indicates that the resulting method is superior to some efficient gradient methods including the BBQ method (SIAM J Optim. 2021;31(4):3068–3096) and is competitive to two famous conjugate gradient software packages CGOPT (1.0) and CG $ \_ $ _ DESCENT (5.0). [ABSTRACT FROM AUTHOR]

Over the years, a considerable number of conjugate gradient methods have been proposed based on modifications on the well-known classical conjugate gradient methods. These methods were shown to have satisfied descent condition taking into consideration the strong Wolfe line search and other line search schemes. Convergence of objective functions were also guarantied. In this study, a decent conjugate gradient method for solving unconstrained non-linear optimization problems is developed. Algorithm of the proposed method was well developed by constructing its update parameter. Descent properties of the method based on some assumptions on the objective function were established. The convergence analysis of the method showed that it converges globally taking into consideration the strong Wolfe conditions . Dolan and More performance profile was used to compare the numerical strength of this method with other methods, showing clear evidence of better performance of the new method in the profiles tested. [ABSTRACT FROM AUTHOR]

Adaptive cubic regularization (ARC) methods for unconstrained optimization compute steps from linear systems involving a shifted Hessian in the spirit of the Levenberg-Marquardt and trust-region methods. The standard approach consists in performing an iterative search for the shift akin to solving the secular equation in trust-region methods. Such search requires computing the Cholesky factorization of a tentative shifted Hessian at each iteration, which limits the size of problems that can be reasonably considered. We propose a scalable implementation of ARC named ARC q K in which we solve a set of shifted systems concurrently by way of an appropriate modification of the Lanczos formulation of the conjugate gradient (CG) method. At each iteration of ARC q K to solve a problem with n variables, a range of m ≪ n shift parameters is selected. The computational overhead in CG beyond the Lanczos process is thirteen scalar operations to update five vectors of length m , and two n -vector updates for each value of the shift. The CG variant only requires one Hessian-vector product and one dot product per iteration, independently of m . Solves corresponding to inadequate shift parameters are interrupted early. All shifted systems are solved inexactly. Such modest cost makes our implementation scalable and appropriate for large-scale problems. We provide a new analysis of the inexact ARC method including its worst case evaluation complexity, global and asymptotic convergence. We describe our implementation and report numerical experience that confirms that our implementation of ARC q K outperforms a classic Steihaug-Toint trust-region method, and the ARC method of the GALAHAD library. The latter solves the subproblem in nested Krylov subspaces by a Lanczos-based method, which requires the storage of a dense matrix that can be comparable to or larger than the two dense arrays required by our approach if the problem is large or requires many Lanczos iterations. Finally, we generalize our convergence results to inexact Hessians and nonlinear least-squares problems. [ABSTRACT FROM AUTHOR]

In this work, for unconstrained optimization, we introduce an improved Dai‐Liao‐style hybrid conjugate gradient method based on the hybridization‐based self‐adaptive technique, and the search direction generated fulfills the sufficient descent and trust region properties regardless of any line search. The global convergence is established under standard Wolfe line search and common assumptions. Then, combining the hyperplane projection technique and a new self‐adaptive line search, we extend the proposed conjugate gradient method and obtain an improved Dai‐Liao‐style hybrid conjugate gradient projection method to solve constrained nonlinear monotone equations. Under mild conditions, we obtain its global convergence without Lipschitz continuity. In addition, the convergence rates for the two proposed methods are analyzed, respectively. Finally, numerical experiments are conducted to demonstrate the effectiveness of the proposed methods. [ABSTRACT FROM AUTHOR]

Due to their simplicity and global convergence properties, the conjugate gradient (CG) methods are widely used for solving unconstrained optimization problems, especially those of large scale. To establish the global convergence and to obtain better numerical performance in practice, much effort has been devoted to develop new CG methods or even to modify well- known methods. In 2012, Rivaie et al., have proposed a new CG method, called RMIL which has good numerical results and globally convergent under the exact line search. However, in 2016, Dai has pointed out a mistake in the steps of the proof of global convergence of RMIL and hence to guarantee the global convergence he suggested a modified version of RMIL, called RMIL+. In this paper, we present another modified version of RMIL, which is globally convergent via the exact line search. Furthermore, to support the theoretical proof of the global convergence of the modified version in practical computation, a numerical experiment based on comparing it with RMIL, RMIL+, and CG-DESCENT was done. [ABSTRACT FROM AUTHOR]

The Fletcher-Reeves (FR) method is widely recognized for its drawbacks, such as generating unfavorable directions and taking small steps, which can lead to subsequent poor directions and steps. To address this issue, we propose a modification to the FR method, and then we develop it into the three-term conjugate gradient method in this paper. The suggested methods, named "HZF" and "THZF", preserve the descent property of the FR method while mitigating the drawbacks. The algorithms incorporate strong Wolfe line search conditions to ensure effective convergence. Through numerical comparisons with other conjugate gradient algorithms, our modified approach demonstrates superior performance. The results highlight the improved efficacy of the HZF algorithm compared to the FR and three-term FR conjugate gradient methods. The new algorithm was applied to the problem of image restoration and proved to be highly effective in image restoration compared to other algorithms. [ABSTRACT FROM AUTHOR]

We present a new diagonal quasi-Newton method for solving unconstrained optimization problems based on the weak secant equation. To control the diagonal elements, the new method uses new criteria to generate the Hessian approximation. We establish the global convergence of the proposed method with the Armijo line search. Numerical results on a collection of standard test problems demonstrate the superiority of the proposed method over several existing diagonal methods. [ABSTRACT FROM AUTHOR]

Unconstrained optimization problems have a long history in computational mathematics and have been identified as being among the crucial problems in the fields of applied sciences, engineering, and management sciences. In this paper, a new variant of the conjugate descent method for solving unconstrained optimization problems is introduced. The proposed algorithm can be seen as a modification of the popular conjugate descent (CD) algorithm of Fletcher. The algorithm of the proposed method is well-defined, and the sequence of the directions of search is shown to be sufficiently descending. The convergence result of the proposed method is discussed under the common standard conditions. The proposed algorithm together with some existing ones in the literature is implemented to solve a collection of benchmark test problems. Numerical experiments conducted show the performance of the proposed method is very encouraging. Furthermore, an additional efficiency evaluation is carried out on problems arising from signal processing and it works well. [ABSTRACT FROM AUTHOR]

In this paper, a family of hybrid conjugate parameters with restart procedure is proposed. In which, we design a hybrid conjugate parameter by using two hybrid techniques, and set a restart procedure in its search direction according to the conjugate parameter. Under the usual assumptions and the weak Wolfe line search, we prove its sufficient descent property and global convergence. Finally, we choose a specific algorithm from this family and use an acceleration scheme to solve the large-scale unconstrained optimization problems and image restorations. These numerical results show that our algorithm is effective. [ABSTRACT FROM AUTHOR]

In this paper, a new appropriate diagonal matrix estimation of the Hessian is introduced by minimizing the Byrd and Nocedal function subject to the weak secant equation. The Hessian estimate is used to correct the framework of a nonmonotone trust region algorithm with the regularized quasi-Newton method. Moreover, to counteract the adverse effect of monotonicity, we introduce a new nonmonotone strategy. The global and superlinear convergence of the suggested algorithm is established under some standard conditions. The numerical experiments on unconstrained optimization test functions show that the new algorithm is efficient and robust. [ABSTRACT FROM AUTHOR]

In this article, we design an optimal neural network based on new LM training algorithm. The traditional algorithm of LM required high memory, storage and computational overhead because of it required the updated of Hessian approximations in each iteration. The suggested design implemented to converts the original problem into a minimization problem using feed forward type to solve non-linear 3D - PDEs. Also, optimal design is obtained by computing the parameters of learning with highly precise. Examples are provided to portray the efficiency and applicability of this technique. Comparisons with other designs are also conducted to demonstrate the accuracy of the proposed design. [ABSTRACT FROM AUTHOR]

The Barzilai-Borwein method offers efficient step sizes for large-scale unconstrained optimization problems. However, it may not guarantee global convergence for nonquadratic objective functions. Simulated annealingbased on Barzilai-Borwein (SABB) method addresses this issue by incorporating a simulated annealing rule. This work proposes a novel step-size strategy for the SABB method, referred to as the SABBm method. Furthermore, we introduce two stabilized variants: SABBstab and SABBmstab. SABBstab combines a simulated annealing rule with a stabilization step to ensure convergence. SABBmstab builds upon SABBstab, incorporating the modified step size derived from the SABBm method. The effectiveness and competitiveness of the proposed methods are demonstrated through numerical experiments on CUTEr benchmark problems. [ABSTRACT FROM AUTHOR]

This paper introduces CLS, a new line search along an arbitrary smooth search path, that starts at the current iterate tangentially to a descent direction. Like the Goldstein line search and unlike the Wolfe line search, the new line search uses, beyond the gradient at the current iterate, only function values. Using this line search with search directions satisfying the bounded angle condition, global convergence to a stationary point is proved for continuously differentiable objective functions that are bounded below and have Lipschitz continuous gradients. The standard complexity bounds are proved under several natural assumptions. [ABSTRACT FROM AUTHOR]

This paper presents a new hybrid conjugate gradient method for solving nonlinear unconstrained optimization problems; it is based on a combination of RMIL (Rivaie-Mustafa-Ismail-Leong) and hSM (hybrid Sulaiman-Mohammed) methods. The proposed algorithm enjoys the sufficient descent condition without depending on any line search; moreover, it is globally convergent under the usual and strong Wolfe line search assumptions. The performance of the algorithm is demonstrated through numerical experiments on a set of 100 test functions from [1] and four image restoration problems with two noise levels. The numerical comparisons with four existing methods show that the proposed method is promising and effective. [ABSTRACT FROM AUTHOR]

The Barzilai–Borwein method offers efficient step sizes for large-scale un-constrained optimization problems. However, it may not guarantee global convergence for nonquadratic objective functions. Simulated annealing-based on Barzilai–Borwein (SABB) method addresses this issue by in-corporating a simulated annealing rule. This work proposes a novel step-size strategy for the SABB method, referred to as the SABBm method. Furthermore, we introduce two stabilized variants: SABBstab and SABBmstab. SABBstab combines a simulated annealing rule with a sta-bilization step to ensure convergence. SABBmstab builds upon SABBstab, incorporating the modified step size derived from the SABBm method. The effectiveness and competitiveness of the proposed methods are demon-strated through numerical experiments on CUTEr benchmark problems.

In this paper, an augmented memoryless BFGS quasi-Newton method was proposed for solving unconstrained optimization problems. Based on a new modified secant equation, an augmented memoryless BFGS update formula and an efficient optimization algorithm were established. To improve the stability of the numerical experiment, we obtained the scaling parameter by minimizing the upper bound of the condition number. The global convergence of the algorithm was proved, and numerical experiments showed that the algorithm was efficient.

Abstract In this paper, a new appropriate diagonal matrix estimation of the Hessian is introduced by minimizing the Byrd and Nocedal function subject to the weak secant equation. The Hessian estimate is used to correct the framework of a nonmonotone trust region algorithm with the regularized quasi-Newton method. Moreover, to counteract the adverse effect of monotonicity, we introduce a new nonmonotone strategy. The global and superlinear convergence of the suggested algorithm is established under some standard conditions. The numerical experiments on unconstrained optimization test functions show that the new algorithm is efficient and robust.

A modified Polak Ribiere Polyak(PRP) conjugate gradient(CG) method is proposed for solving unconstrained optimization problems. The search direction generated by this method satisfies sufficient descent condition at each iteration and this method inherits one remarkable property of the standard PRP method. Under the standard Armijo line search, the global convergence and the linearly convergent rate of the presented method is established. Some numerical results are given to show the effectiveness of the proposed method by comparing with some existing CG methods. [ABSTRACT FROM AUTHOR]

The conjugate gradient method plays a very important role in several fields, to solve problems of large sizes. To improve the efficiency of this method, a lot of works have been done; in this paper we propose a new modification of PRP method to solve a large scale unconstrained optimization problems in relation with strong Wolf Powell Line Search property, when the latter was used under some conditions, a global convergence result was proved. In comparison with other known methods the efficiency of this method proved that it is better in the number of iterations and in time on 90 proposed problems by use of Matlab. [ABSTRACT FROM AUTHOR]

In this study, we introduce a novel hybrid conjugate gradient method with an adaptive strategy called asHCG method. The asHCG method exhibits the following characteristics. (i) Its search direction guarantees sufficient descent property without dependence on any line search. (ii) It possesses strong convergence for the uniformly convex function using a weak Wolfe line search, and under the same line search, it achieves global convergence for the general function. (iii) Employing the Armijo line search, it provides an approximate guarantee for worst-case complexity for the uniformly convex function. The numerical results demonstrate promising and encouraging performances in both unconstrained optimization problems and image restoration problems. [ABSTRACT FROM AUTHOR]

Copyright of Baghdad Science Journal is the property of Republic of Iraq Ministry of Higher Education & Scientific Research (MOHESR) and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Conjugate gradient algorithms come in a wide range of flavors. The conjugate gradient technique primarily concentrates on the spectral parameter. It follows the standard method proposed by Hastens and Stiefel, In this study, we have devised an innovative approach to spectral conjugate gradient methods we get a new direction conjugate gradient method to solve unconstrained optimization problems, which is based on non-linear function using an inexact line searching introduced a novel direction. In specific scenarios, this groundbreaking direction not only guarantees global convergence but also ensures a downward trajectory. Our numerical experiments unequivocally demonstrate that when compared to traditional CG techniques, depending on the number of functions (NOF), the number of iterations (NOI), and time (CPU), and evaluated using the Dolan-More performance profile, our novel method consistently exhibits superior performance across a diverse set of unconstrained function minimization test. and the convergence condition under some Hypotheses by using a strong -Wolfe line search.

Purpose: One of the most effective methods for solving unconstrained optimization problems is the trust region method. The strategy of determining the radius of the trust region has a significant effect on the efficiency of this method. On the other hand, imposing the monotonicity condition will decrease the convergence speed of this method. Therefore, improving and increasing the efficiency of this method is one of the most important issues and the attention of researchers. Methodology: Establishing a new adaptive trust region radius as well as combining the trust region method with a non- monotone strategy to avoid the adverse effects of monotonocity. Findings: A new adaptive trust region radius converged to zero is provided, and then a trust region combination is performed using a non-monotone strategy. Running the algorithm on a set of test functions shows that the new adaptive radius, along with the non-monotone strategy used, significantly improves the efficiency of the trust region method. Originality/Value: The presented non-monotone adaptive algorithm has a second-order convergence rate. In addition, it significantly reduces computational costs compared to traditional algorithms. On the other hand, the new adaptive radius avoids the ineffectiveness of the trust region close to the solution. [ABSTRACT FROM AUTHOR]

Image noise removal is one of the important research directions in the field of image processing, and it can be transformed into a large scale unconstrained optimization problem. Based on the famous two-phase method, in this paper we propose an adaptive conjugate gradient method to remove the impulse noise from the corrupted images by solving the transformed unconstrained optimization problem, which is defined in terms of an edge-preserving potential function. The proposed method can generate a sufficient descent search direction at each step independent of line search condition, and it has global convergence under the Armijo line search condition. Experimental results show that the proposed method is efficient and robust for impulse noise removal. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Education & Science is the property of Republic of Iraq Ministry of Higher Education & Scientific Research (MOHESR) and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

The Conjugate Gradient (CG) method is a powerful iterative approach for solving large-scale minimization problems, characterized by its simplicity, low computation cost and good convergence. In this paper, a new hybrid conjugate gradient HLB method (HLB: Hadji-Laskri-Bechouat) is proposed and analysed for unconstrained optimization. We compute the parameter βkHLB as a convex combination of the Polak-Ribière-Polyak (βkPRP)[1] and the Mohd Rivaie-Mustafa Mamat and Abdelrhaman Abashar (βkRMIL+) i.e βkHLB=(1−θk)βkPRP+θkβkRMIL+ . By comparing numerically CGHLB with PRP and RMIL+ and by using the Dolan and More CPU performance, we deduce that CGHLB is more efficient. [ABSTRACT FROM AUTHOR]

Based on the augmented version of the quasi-Newton method proposed by Aminifard et al. (App. Num. Math. 167:187–201, 2021), a new scaled parameter of the self-scaling memoryless BFGS update formula is proposed. The idea is to cluster the eigenvalues of the search direction matrix, obtained by minimizing the difference between the largest and the smallest eigenvalues of the matrix. The sufficient descent property is proved for uniformly convex functions, and the global convergence of the proposed algorithm is proved both for the uniformly convex and general nonlinear objective functions. Numerical experiments on a set of test functions of the CUTEr collection show that the proposed method is efficient. In addition, the proposed algorithm is effectively applied to salt and pepper noise elimination problem. [ABSTRACT FROM AUTHOR]

Nonmonotonicity has been considered to be essentially influential on the efficiency of the iterative procedures of nonlinear optimization. A review of the literature shows that the objective function values available from recent iterations provide worthier information in the nonmonotone schemes. So, with the aim of enhancing probability of applying more recent available function values, a nonmonotone trust region ratio is suggested using a forgetting factor. Meanwhile, modification of a recent adaptive formula for the trust region radius is devised by a nonmonotone reflection as well. Then, based on the two mentioned modifications, an adaptive nonmonotone trust region algorithm is given. In addition, convergence of the method is analysed under classic assumptions. To provide support for our theoretical arguments, computational merits of the given algorithm on a set of CUTEr test functions are depicted. [ABSTRACT FROM AUTHOR]

In this paper, we discuss two efficient three-term conjugate gradient methods (ECG) for impulse noise removal. The directions of ECG are first the direction of steepest descent and then spanned by the three terms: The steepest descent direction, the previous direction, and the gradient differences at the previous and current points. The second and third terms are scaled by two different step sizes called conjugate gradient parameters. Our goal is to generate and control these parameters such that they do not jointly dominate while preserving the effect of all terms, except near the optimizer where the first term dominates the other two terms. They are independent of the line search method and useful for finite precision arithmetic. The global convergence of ECG is proved. The efficiency (the lowest relative cost of function evaluations) and robustness (highest number of solved problems) of ECG compared to known conjugate gradient methods are shown in terms of PSNR (peak signal noise ratio) and time in seconds. [ABSTRACT FROM AUTHOR]

The Barzilai-Borwein (BB) steplengths play great roles in practical gradient methods for solving unconstrained optimization problems. Motivated by the observation that the two well-known BB steplengths correspond to the ordinary and the data least squares, respectively, we introduce a novel family of BB steplengths from the viewpoint of scaled total least squares. Numerical experiments demonstrate that high performance can be received by a carefully-selected BB steplength in the new family. [ABSTRACT FROM AUTHOR]

This paper proposes an adaptive two-parameter accelerated conjugate gradientmethod, which satisfies the sufficient descent condition in the search direction. The Powell restart strategy is designed in the algorithm to improve its numerical performance. Our proposed method does not add extra computational effort compared to other methods. Furthermore, under general assumptions, we demonstrate the global convergence of our proposed method under theWolfe line search. Finally, we compare with other methods on the unconstrained optimization and image restoration problems. Numerical experiments are presented to show that our proposed method is feasible. [ABSTRACT FROM AUTHOR]

In this article, we design an optimal neural network based on new LM training algorithm. The traditional algorithm of LM required high memory, storage and computational overhead because of it required the updated of Hessian approximations in each iteration. The suggested design implemented to converts the original problem into a minimization problem using feed forward type to solve non-linear 3D - PDEs. Also, optimal design is obtained by computing the parameters of learning with highly precise. Examples are provided to portray the efficiency and applicability of this technique. Comparisons with other designs are also conducted to demonstrate the accuracy of the proposed design.

In this article, an effective neural network is created using unconstrained optimization the brand-new BFGS training algorithm. The fourth order nonlinear partial differential equation is mathematically modeled with feed-forward artificial neural network with some adaptive parameters. The network is trained by new modification of BFGS method to avoid some troubles occurs when the network trained by current BFGS. The conventional updated Hessian approximations approach needed significant memory, storage, and cost computing for each iteration. One of these update’s novel features is its ability to estimate the 2nd order curvature of the goal function (energy functions) with high order precision while using the provided gradient and function value data. It is shown that the global convergence properties of the suggested modification, there is a parameter ρ in the update formulae which ranges from zero to one. The numerical experiments demonstrate that the improved BFGS update will be more accurate and more effective than the traditional BFGS methods. The proposed algorithm has well properties such: it has global convergence for energy function which is convex functions; also to get optimal step length we used a nonmonotone line search technique to modify the effectiveness of the proposed algorithm. Finally, used suggested training algorithm, to learned an appropriate neural network for accurately solving any non-linear PDEs.

In this research, we propose an optimal choice for the non-negative constant in the Dai-Liao conjugate gradient formula based on the prominent Barzilai-Borwein approach by leveraging the nice features of the Frobenius matrix norm. The global convergence of the new modification is demonstrated using some basic assumptions. Numerical comparisons with similar algorithms show that the new approach is reliable in terms of the number of iterations, computing time, and function evaluations for unconstrained minimization, portfolio selection and image restoration problems.

Particle Swarm Optimization algorithm (PSO) has been widely utilized for addressing optimization problems due to its straightforward implementation and efficiency in tackling various test functions and engineering optimization problems. Nevertheless, PSO encounters issues like premature convergence and a lack of diversity, particularly when confronted with complex high-dimensional optimization tasks. In this study, we propose an enhanced version of the Island Model Particle Swarm Optimization (IMPSO), where island models are integrated into the PSO algorithm based on several migration strategies. The first contribution consists in applying a new selection and replacement strategies based on tabu search technique, while the second contribution consists in proposing a dynamic migration rate relying on the Biogeography-Based Optimization technique. To assess and validate the effectiveness of the proposed method, several unconstrained benchmark functions are applied. The obtained results confirm that the approach yield better performance than the old version of IMPSO for solving NP-hard optimization problems. Compared to the performance of other well-known evolutionary algorithms, the proposed approach is more efficient and effective. [ABSTRACT FROM AUTHOR]

The traditional BFGS algorithm has been proved very efficient. It is convergent for convex nonlinear optimization problems. However, for non-convex nonlinear optimization problems, it is known that the BFGS algorithm may not be convergent. This paper proposes a robust BFGS algorithm in the sense that the algorithm superlinearly converges to a local minimum under some mild assumptions for both convex and non-convex nonlinear optimization problems. Numerical test on the CUTEst test set is reported to demonstrate the merit of the proposed robust BFGS algorithm. This result shows that the robust BFGS algorithm is very efficient and effective. [ABSTRACT FROM AUTHOR]

The Conjugate Gradient (CG) method is renowned for its rapid convergence in optimization applications. Over the years, several modifications to CG methods have emerged to improve computational efficiency and tackle practical challenges. This paper presents a new three-term hybrid CG method for solving unconstrained optimization problems. This algorithm utilizes a search direction that combines Liu-Storey (LS) and Conjugate Descent (CD) CG coefficients and standardizes it using a spectral which acts as a scheme for the choices of the conjugate parameters. This resultant direction closely approximates the memoryless Broyden-Fletcher-Goldfarb-Shanno (BFGS) quasi-Newton direction, known for its bounded nature and compliance with the sufficient descent condition. The paper establishes the global convergence under standard Wolfe conditions and some appropriate assumptions. Additionally, the numerical experiments were conducted to emphasize the robustness and superior efficiency of this hybrid algorithm in comparison to existing approaches. [ABSTRACT FROM AUTHOR]

This paper is dedicated to the development of a novel class of quasi-Newton techniques tailored to address computational challenges posed by memory constraints. Such methodologies are commonly referred to as "limited" memory methods. The method proposed herein showcases adaptability by introducing a customizable memory parameter governing the retention of historical data in constructing the Hessian estimate matrix at each iterative stage. The search directions generated through this novel approach are derived from a modified version closely resembling the full memory multi-step BFGS update, incorporating limited memory computation for a singular term to approximate matrix–vector multiplication. Results from numerical experiments, exploring various parameter configurations, substantiate the enhanced efficiency of the proposed algorithm within the realm of limited memory quasi-Newton methodologies category. [ABSTRACT FROM AUTHOR]

One of the quasi-Newton update formulae, namely the Davidon-Fletcher-Powell method, is crucial for resolving nonlinear programming optimization problems. In order to achieve a Newton-like condition that depends on the function values and gradient vectors at each iteration, we construct an alternative positive-definite Hessian approximation in this study. The essential theorems are established to study algorithm convergence. The proposed approach is then tested on well-known test problems and then compared to the standard DFP method. The numerical outcomes demonstrate the effectiveness of the newly developed method. [ABSTRACT FROM AUTHOR]

The conjugate gradient (CG) techniques are a class of unconstrained optimization algorithms with strong local and global convergence qualities and minimal memory needs. While the quasi-Newton methods are reliable and efficient on a wide range of problems and these methods are converge faster than the conjugate gradient methods and require fewer function evaluations, however, they are request substantially more storage, and if the problem is ill-conditioned, they may require several iterations. There is another class, termed preconditioned conjugate gradient method, it is a technique that combines two methods conjugate gradient with quasi-Newton. In this work, we proposed a new two limited memory preconditioned conjugate gradient methods (New1 and New2), to solve nonlinear unconstrained minimization problems, by using new modified symmetric rank one (NMSR1) and new modified Davidon, Fletcher, Powell (NMDFP), and also using projected vectors. We proved that these modifications fulfill some conditions. Also, the descent condition of the new technique has been proved. The numerical results showed the efficiency of the proposed new algorithms compared with some standard nonlinear, unconstrained problems. [ABSTRACT FROM AUTHOR]

This article deals with the global convergence of 3-Term Conjugate Gradient (3TCG) methods on general functions under a novel Inexact Line Search (ILS) strategy. The proposed algorithm satisfies the sufficient descent condition, and its global convergence can be easily proved. To verify the efficiency of the authors' theoretical approach, some numerical experiments and comparisons are presented. These results show that the suggested method is sufficiently efficient and comparable to new similar strategies for solving unconstrained optimization problems. [ABSTRACT FROM AUTHOR]

Following the standard methods proposed by Polak-Ribiere-Polyak (P-R), in this work we introduce two new non-linear conjugate gradient methods for solving unconstraint optimization problem, our new methods based on P-R. Standard method (P-R) have performance well in numerical result but does not satisfy global convergency condition. In this paper we modified double attractive and powerful parameters that have better performance and good numerical result than P-R method, also each of our robust method can satisfies the descent condition and global convergency condition by using wolf condition. More over the second method modified by logistic mapping form, the main novelty is their numerical results and demonstrate performance well with compare to a standard method. [ABSTRACT FROM AUTHOR]

The subject of this study is the analysis of the spectral secant equation for the well-known Davidon–Fletcher–Powell (DFP) quasi-Newton updating formula. More precisely, we plan to make the memoryless DFP formula well conditioned in several aspects. We first focus our efforts on obtaining eigenvalues of the DFP formula and directly analyzing its spectral condition number. Then, we proceed in a different direction by using the Byrd–Nocedal measure function as well. Lastly, we propose a hybrid adaptive formula for the spectral parameter of the method. [ABSTRACT FROM AUTHOR]

In this paper, we studied a two member family of diagonal conjugate gradient like methods for unconstrained minimization problems. The proposed search directions of the methods are obtained by incorporating a diagonal Hessian approximation approach with the modifications of the Polak–Ribière–Polyak and Liu-Storey parameters. The resulting directions guarantees the sufficient decrease of the objective function free from any line search. The global convergence of the proposed methods using both the Wolfe and Armijo-type line search conditions without the convexity assumption on the function is provided. Furthermore, the R-linear convergence rate of the methods are analyzed under Wolfe line search condition. Numerical experiments demonstrated the performance of the method on general unconstrained minimization problems and image restoration problem. [ABSTRACT FROM AUTHOR]