Search

In this work, multi-variable derivative-free optimization algorithms for unconstrained optimization problems are developed. A novel procedure for approximating the gradient of multi-variable objective functions based on non-commutative maps is introduced. The procedure is based on the construction of an exploration sequence to specify where the objective function is evaluated and the definition of so-called gradient generating functions which are composed with the objective function such that the procedure mimics a gradient descent algorithm. Various theoretical properties of the proposed class of algorithms are investigated and numerical examples are presented.

This paper presents a novel Adaptive Mutation Quantum-inspired Squirrel Search Algorithm (AM-QSSA). Firstly, based on the population mutation, a location-update of quantum state correlative and attractor method is proposed. By introducing a random process to modify the sliding factor of the local attractor, the inevitable lack of diversity in the population renewal method is solved. The premature convergence problem of adaptive mutation rate improvement algorithm based on squirrel position update mode is introduced. Meanwhile, the paper decomposes the location update process of the SSA, and improves it with quantum-behavior. Furthermore, it proposes a novel quantum-inspired squirrels search algorithm. This method finds the complementary effect between quantum behavior and squirrel search algorithm, and solves the problem of premature convergence probability of SSA. In addition, it improves population diversity, and achieves the balance between global and local search. The efficiency of the proposed AM-QSSA is evaluated using exploitation analysis, exploration analysis, success rate analysis, convergence rate analysis on classical benchmark functions as well as Congress on Evolutionary Computation (CEC) 2017 test functions. For further study, AM-QSSA optimizes an image registration problem for an extensive study to check its applicability. The results reveal that AM-QSSA is more efficient and stable than SSA. And it is comparable to the most advanced optimization algorithms.

The impact of neutrosophy has increased rapidly in many areas of science and technology in recent years. Furthermore, numerous applications of the neutrosophic theory have become more usual. We aim to use neutrosophy to enhance Dai–Liao conjugate gradient (CG) iterative method. In particular, we suggest and explore a new neutrosophic logic system intended to compute the essential parameter t required in Dai–Liao CG iterations. Theoretical examination and numerical experiments signify the effectiveness of the introduced method for controlling t. By incorporation of the neutrosophy in the Dai–Liao conjugate gradient principle, we established novel Dai–Liao CG iterations for solving large-scale unconstrained optimization problems. Global convergence is proved under standard assumptions and with the use of the inexact line search. Finally, computational evidence shows the computational effectiveness of the proposed fuzzy neutrosophic Dai–Liao CG method.

In this paper, a three-dimensional subspace method is proposed, in which the search direction is generated by minimizing the approximation model of the objective function in a three-dimensional subspace. The approximation model of the objective function is not unique, and alternatives can be chosen between a symmetric quadratic model and a conic model by specific criteria. Moreover, the idea of a WLY conjugate gradient method is applied to characterize the change of gradient direction between adjacent iteration points. The strategy of initial stepsize and nonmonotone line search are adopted, and the global convergence of the presented algorithm is established under mild assumptions. In numerical experiments, we use a collection of 80 unconstrained optimization test problems to show the competitive performance of the presented method.

We study the use of inverse harmonic Rayleigh quotients with target for the stepsize selection in gradient methods for nonlinear unconstrained optimization problems. This not only provides an elegant and flexible framework to parametrize and reinterpret existing stepsize schemes, but it also gives inspiration for new flexible and tunable families of steplengths. In particular, we analyze and extend the adaptive Barzilai–Borwein method to a new family of stepsizes. While this family exploits negative values for the target, we also consider positive targets. We present a convergence analysis for quadratic problems extending results by Dai and Liao (IMA J Numer Anal 22(1):1–10, 2002), and carry out experiments outlining the potential of the approaches.

In this paper, we investigate two ideas in the context of the interpolation-based optimization paradigm tailored to derivative-free black-box optimization problems. The proposed architecture maintains a radial basis function interpolation model of the actual objective that is managed according to a trust-region globalization scheme. We focus on two distinctive ideas. Firstly, we explore an original sampling strategy to adapt the interpolation set to the new trust region. A better-than-linear interpolation model is guaranteed by maintaining a well-poised supporting subset that pursues a near regular simplex geometry of n + 1 points plus the trust-region center. This strategy improves the geometric distribution of the interpolation points whilst also optimally exploiting the existing interpolation set. On account of the associated minimal interpolation set size, the better-than-linear interpolation model will exhibit curvature, which is a necessary condition for the second idea. Therefore, we explore the generalization of the classic spherical to an ellipsoidal trust-region geometry by matching the contour ellipses with the inverse of the local problem hessian. This strategy is enabled by the certainty of a curved interpolation model and is introduced to accounts for the local output anisotropy of the objective function when generating new interpolation points. Instead of adapting the sampling strategy to an ellipsoid, we carry out the sampling in an affine transformed space. The combination of both methods is validated on a set of multivariate benchmark problems and compared with ORBIT.

In this paper, a new family of Dai-Liao–type conjugate gradient methods are proposed for unconstrained optimization problem. In the new methods, the modified secant equation used in [H. Yabe and M. Takano, Comput. Optim. Appl. 28 (2004) 203–225] is considered in Dai and Liao’s conjugacy condition. Under some certain assumptions, we show that our methods are globally convergent for general functions with strong Wolfe line search. Numerical results illustrate that our proposed methods can outperform some existing ones.

We present a new algorithm for large-scale unconstrained minimization that, at each iteration, minimizes, approximately, a quadratic model of the objective function plus a regularization term, not necessarily based on a norm. We prove convergence assuming only gradient continuity and complexity results assuming Lipschitz conditions. For solving the subproblems in the case of regularizations based on the 3-norm, we introduce a new method that quickly obtains the approximate solutions required by the theory. We present numerical experiments.

Two improved nonlinear conjugate gradient methods are proposed by using the second inequality of the strong Wolfe line search. Under usual assumptions, we proved that the improved methods possess the sufficient descent property and global convergence. By testing the unconstrained optimization problems which taken from the CUTE library and other usual test collections, some large-scale numerical experiments for the presented methods and their comparisons are executed. The detailed results are listed in tables and their corresponding performance profiles are reported in figures, which show that our improved methods are superior to their comparisons.

The Secant equation has long been the foundation of quasi-Newton methods, as updated Hessian approximations satisfy the equation with each iteration. Several publications have lately focused on modified versions of the Secant relation, with promising results. This study builds on that idea by deriving a Secant-like modification that uses non-linear quantities to construct Hessian (or its inverse) approximation updates. The method uses data from the two most recent iterations to provide an alternative to the Secant equation with the goal of producing improved Hessian approximations that induce faster convergence to the objective function optimal solution. The reported results provide strong evidence that the proposed method is promising and warrants further investigation.

Conjugate gradient methods (CG) constitute excellent neural network training methods that are simplicity, flexibility, numerical efficiency, and low memory requirements. In this paper, we introduce a new three-term conjugate gradient method, for solving optimization problems and it has been tested on artificial neural networks (ANN) for training a feed-forward neural network. The new method satisfied the descent condition and sufficient descent condition. Global convergence of the new (NTTCG) method has been tested. The results of numerical experiences on some wellknown test function shown that our new modified method is very effective, by relying on the number of functions evaluation and number of iterations, also included the numerical results for training feed-forward neural networks with other well-known method in this field.

Conjugate gradient (CG) method is well-known for its ability to solve unconstrained optimization (UO.) problems. This article presenting a new CG method with sufficient descent conditions which improves the former method developed by Rvaie, Mustafa, Ismail and Leong (RMIL). The efficacy of the proposed method has been demonstrated through simulations on the Kijang Emas pricing regression problem. The daily data between January 2021 to May 2021 were obtained from Malaysian Ministry of Health and Bank Negara Malaysia. The dependent variable for this study was the Kijang Emas price, and the independent variables were the coronavirus disease (COVID-19) measures (i.e., new cases, R-naught, death cases, new recovered). Data collected were analyzed on its correlation and coefficient determinant, and the influences of COVID-19 on Kijang Emas price was examined through multiple linear regression model. Findings revealed that the suggested technique outperformed the existing CG algorithms in terms of computing efficiency.

In this paper, a new formula of 𝛽𝑘 is suggested for the conjugate gradient method of solving unconstrained optimization problems based on three terms and step size of cubic. Our new proposed CG method has descent condition, sufficient descent condition, conjugacy condition, and global convergence properties. Numerical comparisons with two standard conjugate gradient algorithms show that this algorithm is very effective depending on the number of iterations and the number of functions evaluated.

The beetle antennae search (BAS) algorithm is a memetic meta-heuristic optimization algorithm capable of solving combinatorial optimization problems. In this paper, the binary version of BAS (BBAS) is modified by adding a V-shaped transfer function. In this way, we introduce the V-shaped transfer function-based binary BAS (VSBAS) algorithm, which is a more effective and efficient version of BBAS in the case of large input data. Applications using real-world data sets on a binary Markowitz-based portfolio selection (BMPS) problem validate the excellent performance of VSBAS on large input data and demonstrate that it is a marvelous alternative against other ordinary memetic meta-heuristic optimization algorithms. Note that, because the meta-heuristic algorithms compared in this paper are directly applicable only to unconstrained optimization, the penalty function method was used to keep their solutions in the feasible district. In order to support and promote the findings of this work, we have constructed a complete MATLAB package for the interested user, which is freely available through GitHub.

Chemical Reaction Optimization (CRO) is a simple and efficient evolutionary optimization algorithm by simulating chemical reactions. As far as the current research is concerned, the algorithm has been successfully used for solving a number of real-world optimization tasks. In our paper, a new real encoded chemical reaction optimization algorithm is proposed to boost the efficiency of the optimization operations in standard chemical reactions optimization algorithm. Inspired by the evolutionary operation of the differential evolution algorithm, an improved search operation mechanism is proposed based on the underlying operation. It is modeled to further explore the search space of the algorithm under the best individuals. Afterwards, to control the perturbation frequency of the search strategy, the modification rate is increased to balance between the exploration ability and mining ability of the algorithm. Meanwhile, we also propose a new population initialization method that incorporates several models to produce high-quality initialized populations. To validate the effectiveness of the algorithm, nine unconstrained optimization algorithms are used as benchmark functions. As observed from the experimental results, it is evident that the proposed algorithm is significantly better than the standard chemical reaction algorithm and other evolutionary optimization algorithms. Then, we also apply the proposed model to address the synthesis problem of two antenna array synthesis. The results also reveal that the proposed algorithm is superior to other approaches from different perspectives.

A swarm-based nature-inspired optimization algorithm, namely, the fruit fly optimization algorithm (FOA), hasa simple structure and is easy to implement. However, FOA has a low success rate and a slow convergence, because FOA generates new positions around the best location, using a fixed search radius. Several improved FOAs have been proposed. However, their exploration ability is questionable. To make the search process smooth, transitioning from the exploration phase to the exploitation phase, this paper proposes a new FOA, constructed from a cooperation of the multileader and the probabilistic random walk strategies (CPFOA). This involves two population types working together. CPFOAs performance is evaluated by 18 well-known standard benchmarks. The results showed that CPFOA outperforms both the original FOA and its variants, in terms of convergence speed and performance accuracy. The results show that CPFOA can achieve a very promising accuracy, when compared with the well-known competitive algorithms. CPFOA is applied to optimize twoapplications: classifying the real datasets with multilayer perceptron and extracting the parameters of a very compact T-S fuzzy system to model the Box and Jenkins gas furnace data set. CPFOA successfully find parameters with a very high quality, compared with the best known competitive algorithms.

In this paper, a new conjugacy condition is established to solve unconstrained optimization problems based on a new quasi-Newton equation. We present a modified Dai–Liao conjugate gradient method to solve unconstrained optimization problems with a new value of the parameter t based on the new conjugacy condition. The presented algorithm has the following properties: (i) the modified Dai–Liao conjugate gradient method considers both the gradient and function value information. (ii) The global convergence is achieved for the modified Dai–Liao conjugate gradient method under some suitable assumptions. (iii) Numerical experiments on unconstrained optimization problems and image restoration problems are conducted, and the numerical results show that our method is efficient.

Currently, test problems are used to test the effectiveness of new global optimization methods. In this article, we analyze test global optimization problems to test the numerical efficiency of methods for their solution. At present, about 200 test problems of unconditional optimization and more than 1000 problems of conditional optimization have been developed. We can find these test problems on the Internet. However, most of these test problems are not informative for testing the effectiveness of global optimization methods. The solution of test problems of conditional optimization, as a rule, has trivial solutions. This allows the parameters of the algorithms to be tuned before these solutions are obtained. In test problems of conditional optimization, the accuracy of the fulfillment of constraints is important. Often, small errors in the constraints lead to a significant change in the value of an objective function. Construction of a new package of test problems to test the numerical efficiency of global optimization methods and compare the exact quadratic regularization method with existing methods.The author suggests limiting oneself to test problems of unconstrained optimization with unknown solutions. A package of test problems of unconstrained optimization is pro-posed, which includes known test problems with unknown solutions and modifications of some test problems proposed by the author. We also propose to include in this package J. Nie polynomial functions with unknown solutions. This package of test problems will simplify the verification of the numerical effectiveness of methods. The more effective methods will be those that provide the best solutions. The paper compares existing global optimization methods with the exact quadratic regularization method proposed by the author. This method has shown the best results in solving most of the test problems. This paper presents some of the results of the author's numerical experiments. In particular, the best solutions were obtained for test problems with unknown solutions. This method allows solving multimodal problems of large dimensions and only a local search program is required for its implementation.

In this paper, we propose a modified q-Dai–Yuan (q-DY) conjugate gradient algorithm based on q-gradient for solving unconstrained optimization problems. The q-gradient is calculated based on q-derivative that requires a dilation parameter q for controlling the balance between the local and global search. The strong global convergence of the algorithm is proved under standard Wolfe conditions, and numerical results show the efficiency of the proposed algorithm.

In this paper, we present a new hybrid conjugate gradient (CG) approach for solving unconstrained optimization problem. The search direction is a hybrid form of the Fletcher-Reeves (FR) and the Dai-Yuan (DY) CG parameters and is close to the direction of the memoryless Broyden-Fletcher-Goldfarb-Shanno (BFGS) quasi-Newton approach. Independent of the line search, the search direction of the new approach satisfies the descent condition and possess the trust region. We establish the global convergence of the approach for general functions under the Wolfe-type and Armijo-type line search. Using the CUTEr library, numerical results show that the propose approach is more efficient than some existing approaches. Furthermore, we give a practical application of the new approach in optimizing risk in portfolio selection.

A new hybrid quasi-Newton search direction ( HQNEI ) is proposed. It uses the update formula of Broyden–Fletcher–Goldfarb–Shanno (BFGS) with a certain conjugate gradient (CG) parameter by a nested direction. The global convergence analysis and superlinear rate, addtionaly with sufficient descent are proved using exact line search. Finally, the computation comparisons are made with original hybrid parents; BFGS and CG, through the efficiency in terms of iteration numbers and CPU-running time showing the superior of HQNEI. Therefore, the results marked preference of HQNEI from other two producer algorithms.

In our research, a new conjugate gradient (CG) method for solving nonlinear systems of equations is introduced and studied. The proposed method defined as a convex combination between MMWU and RMIL...

Recently, a one-parameter extension of the Polak–Rebière–Polyak method has been suggested, having acceptable theoretical features and promising numerical behavior. Here, based on an eigenvalue analysis on the method with the aim of avoiding a search direction in the direction of the maximum magnification by a symmetric version of the search direction matrix, an adaptive formula for computing parameter of the method is proposed. Under standard assumptions, the given formula ensures the sufficient descent property and guarantees the global convergence of the method. Numerical experiments are done on a collection of CUTEr test problems. They show practical effectiveness of the suggested formula for the parameter of the method.

We propose a nonmonotone trust region method for unconstrained optimization problems on Riemannian manifolds. Global convergence to the first-order stationary points is proved under some reasonable conditions. We also establish local R-linear, super-linear and quadratic convergence rates. Preliminary experiments show that the algorithm is efficient.

In this paper, we propose a new nonmonotone trust region Barzilai-Borwein (BB for short) method for solving unconstrained optimization problems. The proposed method is given by a novel combination of a modified Metropolis criterion, BB-stepsize and trust region method. The new method uses the reciprocal of BB-stepsize to approximate the Hessian matrix of the objective function in the trust region subproblems, and accepts some bad solutions according to the modified Metropolis criterion based on simulated annealing idea. Under some suitable assumptions, the global convergence of the new method is established. Some preliminary numerical results indicate that, the new method is more efficient compared with the existing trust region BB method.

Motivated by the projection technique, in this paper, we introduce a new method for approximating the solution of nonlinear equations with convex constraints. Under the assumption that the associated mapping is Lipchitz continuous and satisfies a weaker assumption of monotonicity, we establish the global convergence of the sequence generated by the proposed algorithm. Applications and numerical example are presented to illustrate the performance of the proposed method.

Conjugate Gradient (CG) method is the most prominent iterative mathematical technique that can be useful for the optimization of both linear and non-linear systems due to its simplicity, low memory requirement, computational cost, and global convergence properties. However, some of the classical CG methods have some drawbacks which include weak global convergence, poor numerical performance both in terms of number of iterations and the CPU time. To overcome these drawbacks, researchers proposed new variants of the CG parameters with efficient numerical results and nice convergence properties. Some of the variants of the CG method include the scale CG method, hybrid CG method, spectral CG method, three-term CG method, and many more. The hybrid conjugate gradient (CG) algorithm is among the efficient variant in the class of the conjugate gradient methods mentioned above. Some interesting features of the hybrid modifications include inherenting the nice convergence properties and efficient numerical performance of the existing CG methods. In this paper, we proposed a new hybrid CG algorithm that inherits the features of the Rivaie et al. (RMIL*) and Dai (RMIL+) conjugate gradient methods. The proposed algorithm generates a descent direction under the strong Wolfe line search conditions. Preliminary results on some benchmark problems show that the proposed method efficient and promising.

We propose an effective conjugate gradient method belonging to the class of Dai–Liao methods for solving unconstrained optimization problems. We employ a variant of the modified secant condition and introduce a new conjugate gradient parameter by solving an optimization problem. The optimization problem combines the well-known features of the linear conjugate gradient method using some penalty functions. This new parameter takes advantage of function information as well as the gradient information to provide the iterations. Our proposed method is globally convergent under mild assumptions. We examine the ability of the method for solving some real-world problems from image processing field. Numerical results show that the proposed method is efficient in the sense of the PSNR test. We also compare our proposed method with some well-known existing algorithms using a collection of CUTEr problems to show its efficiency.

Recently, there has been a surge of interest in designing variants of the classical Newton-CG in which the Hessian of a (strongly) convex function is replaced by suitable approximations. This is mainly motivated by large-scale finite-sum minimization problems that arise in many machine learning applications. Going beyond convexity, inexact Hessian information has also been recently considered in the context of algorithms such as trust-region or (adaptive) cubic regularization for general non-convex problems. Here, we do that for Newton-MR, which extends the application range of the classical Newton-CG beyond convexity to invex problems. Unlike the convergence analysis of Newton-CG, which relies on spectrum preserving Hessian approximations in the sense of L\"{o}wner partial order, our work here draws from matrix perturbation theory to estimate the distance between the subspaces underlying the exact and approximate Hessian matrices. Numerical experiments demonstrate a great degree of resilience to such Hessian approximations, amounting to a highly efficient algorithm in large-scale problems., Comment: 32 pages, 10 figures

A novel gradient stepsize is derived at the motivation of equipping the Barzilai--Borwein (BB) method with the two dimensional quadratic termination property. A remarkable feature of the novel step...

This brief investigates the distributed unconstrained optimization problem, where its global objective function consists of the sum of $N$ local objective functions. The aim for this brief is to design a discrete-time algorithm to solve the considered distributed optimization problem using only local computations and local information exchanges. To this end, a discrete-time algorithm resorting to the conjugate gradient method is proposed, and by it, the optimal solution to a class of distributed optimization problems over a static undirected graph can be obtained with finite-time computations under some mild conditions. Furthermore, simulations are given to verify the validity of the designed algorithm.

This research is focused on Unconstrained Optimization problems. Among a number of methods that can be used to solve Unconstrained Optimization problems we have worked on Gradient and Coordinate Descent methods. Step size plays an important role for optimization. Here we have performed numerical experiment with Gradient and Coordinate Descent method for several step size choices. Comparison between different variants of Gradient and Coordinate Descent methods and their efficiency are demonstrated by implementing in loss functions minimization problem.

Motivated by gradient methods in optimization theory, we give methods based on $\psi$-fractional derivatives of order $\alpha$ in order to solve unconstrained optimization problems. The convergence of these methods is analyzed in detail. This paper also presents an Adams-Bashforth-Moulton (ABM) method for the estimation of solutions to equations involving $\psi$-fractional derivatives. Numerical examples using the ABM method show that the fractional order $\alpha$ and weight $\psi$ are tunable parameters, which can be helpful for improving the performance of gradient descent methods., Comment: 27 pages

In this paper we present a new BFGS method for solving unconstrained optimization problems, using a modified rational approximation model The idea is to improve the Barzilai and Borwein approximation (BBA) [27] by incorporating a new parameter Under certain conditions the global convergent result of the proposed method is established The numerical results have shown that, the new method is promising and outperforms other classical methods Besides, the new method was used to solve data from Covid-19 and the performance was compared with Least Square Method (LSM) The outcome has shown that the new method has less relative error and can be used in place of LSM in regression analysis © 2020, Research Publication All rights reserved

This article takes on solving the problem of multicriteria conditional optimization. This problem is one of the most key tasks of the current time and has its application in many areas. Reuse of various existing algorithms for solving unconstrained optimization is proposed. Different methods of multicriteria unconditional optimization are reviewed. The advantages and disadvantages of each algorithm are analyzed. The algorithms modified to take into account the constraints. Additional algorithms of transition from solving an unconditional optimization problem to a conditional optimization problem are developed. A genetic algorithm SPEA2 was used to test the developed algorithms. Examples of solving the problem at hand using the aforementioned algorithms are presented. A comparative analysis of the final results was conducted.

In this paper, we propose a new conjugate gradient method for solving nonlinear unconstrained optimization. The new method consists of three parts, the first part of them is the parameter of Hestenes-Stiefel (HS). The proposed method is satisfying the descent condition, sufficient descent condition and conjugacy condition. We give some numerical results to show the efficiency of the suggested method.

Conjugate gradient method is the most used for solving the unconstrained optimization problem. In this paper we propose a new hybrid conjugate gradient method which is obtained from a convex combin...

To solve a large-scale unconstrained optimization problem, in this paper we propose a class of spectral three-term conjugate gradient methods. We indicate that the proposed class, in fact, generate...

The conjugate gradient (CG) scheme is regarded as among the efficient methods for large-scale optimization problems. Several versions of CG methods have been presented recently owing to their rapid convergence, simplicity, and their less memory requirements. In this article, we construct a new CG algorithm via the combination of the classical methods of Fletcher-Reeves (FR), and Hestenes-Stiefel (HS). The new CG method possesses the descent properties and converge globally provided the exact minimization condition is satisfied. The tests of the new CG method using MATLAB are analysed in terms of iteration number and CPU time. Numerical results have been reported which shows that the proposed CG method performs better compare to other CG methods.

Variants of the Newton method are very popular for solving unconstrained optimization problems. The study on global convergence of the BFGS method has also made good progress. The q-gradient reduces to its classical version when q approaches 1. In this paper, we propose a quantum-Broyden–Fletcher–Goldfarb–Shanno algorithm where the Hessian is constructed using the q-gradient and descent direction is found at each iteration. The algorithm presented in this paper is implemented by applying the independent parameter q in the Armijo–Wolfe conditions to compute the step length which guarantees that the objective function value decreases. The global convergence is established without the convexity assumption on the objective function. Further, the proposed method is verified by the numerical test problems and the results are depicted through the performance profiles.

In this paper, we analyse the worst-case complexity of nonlinear stepsize control (NSC) algorithms for solving convex smooth unconstrained optimization problems. We show that, to drive the norm of ...

The conjugate gradient method is a very efficient iterative technique for solving large-scale unconstrainedoptimization problems. Motivated by recent modifications of some variants of the method and construction of hybrid methods, this study proposed four hybrid methods that are globally convergent as well as computationally efficient. The approach adopted for constructing the hybrid methods entails projecting ten recently modified conjugate gradient methods. Each of the hybrid methods is shown to satisfy the descent property independent of any line search technique and globally convergent under the influence of strong Wolfe line search. Results obtained from numerical implementation of these methods and performance profiling show that the methods are very competitive with well-known traditional methods.

A modified method of particle swarm optimization (PSO) has been developed in the article. It is based on a diversity mechanism (DM), which greatly improves optimization method efficiency. In order to show the enhanced PSO modification, a comparative analysis was conducted. It is based on two proposed criteria and time of the algorithms' execution. The modified method was applied to the problem of constrained optimization of PI-controller tuning. A new criterion was developed with pit-in-pit topology, which includes stability requirement, a set of inequalities (constrains), and a set of optimization criteria to minimize. It allows reducing the initial problem to the problem of unconstrained optimization. The efficiency of the approach is confirmed with four problems of PI-controller tuning.

In this study, a new self-scaling variable metric (VM)-updating method for solving nonlinear unconstrained optimization problems is presented. The general strategy of (New VM-updating) is to propose a new quasi-newton condition used for update the usual DFP Hessian to a number of times in a way to be specified in some iteration with PCG method to improve the performance of the Hessian approximation. We show that it produces a positive definite matrix. Experimental results indicate that the new suggested method was more efficient than the standard DFP method, with respect to the number of functions evaluations (NOF) and number of iterations (NOI).