Search

In this paper, we propose a new Newton method for minimizing convex optimization problems with singular Hessian matrices including the special case that the Hessian matrix of the objective function is singular at any iteration point. The new method we proposed has some updates in the regularized parameter and the search direction. The step size of our method can be obtained by using Armijo backtracking line search. We also prove that the new method has global convergence. Some numerical experimental results show that the new method performs well for solving convex optimization problems whose Hessian matrices of the objective functions are singular everywhere. [ABSTRACT FROM AUTHOR]

In this paper, we consider a class of composite multiobjective optimization problems, subject to a closed convex constraint set, defined on Riemannian manifolds. To tackle this problem, we propose the generalized conditional gradient method with two step size strategies, including Armijo step size and the nonmonotone line search step size. Under some reasonable conditions, the global convergence result is established, and the iteration-complexity bound for composite multiobjective optimization problems is presented on Riemannian manifolds. [ABSTRACT FROM AUTHOR]

This paper introduces two three-term trust region conjugate gradient algorithms, TT-TR-WP and TT-TR-CG, which are capable of converging under non-Lipschitz continuous gradient functions without any additional conditions. These algorithms possess sufficient descent and trust region properties, and demonstrate global convergence. In order to assess their numerical performance, we compare them with two classical algorithms in terms of restoring noisy gray-scale and color images as well as solving large-scale unconstrained problems. In restoring noisy gray-scale images, we set the performance of TT-TR-WP as the standard, then TT-TR-CG takes around 2.33 times longer. The other algorithms around 2.46 and 2.41 times longer, respectively. In solving the same color images, the proposed algorithms exhibit relative good performance over other algorithms. Additionally, TT-TR-WP and TT-TR-CG are competitive in unconstrained problems, and the former has wide applicability while the latter has strong robustness. Moreover, the proposed algorithms are both more outstanding than the baseline algorithms in terms of applicability and robustness. [ABSTRACT FROM AUTHOR]

In this paper, we propose a modification that improves efficiency, robustness and reliability of the famous HS conjugate gradient method. In particular, we propose a hybrid of the HS and DHS methods, where DHS is another recent modification of the HS method. Irrespective of the line search, the search direction of the proposed method is sufficiently descent. Moreover, the new approach guarantees global convergence for general functions under the strong Wolfe line search. Numerical results and performance profiles are reported, and indicate that the new approach outperforms three similar methods in the literature. We also give a practical application of the new approach in minimizing risk in portfolio selection. [ABSTRACT FROM AUTHOR]

Large-scale symmetric and asymmetric matrices have emerged in predicting the relationship between genes and diseases. The emergence of large-scale matrices increases the computational complexity of the problem. Therefore, using low-rank matrices instead of original symmetric and asymmetric matrices can greatly reduce computational complexity. In this paper, we propose an approximation conjugate gradient method for solving the low-rank matrix recovery problem, i.e., the low-rank matrix is obtained to replace the original symmetric and asymmetric matrices such that the approximation error is the smallest. The conjugate gradient search direction is given through matrix addition and matrix multiplication. The new conjugate gradient update parameter is given by the F-norm of matrix and the trace inner product of matrices. The conjugate gradient generated by the algorithm avoids SVD decomposition. The backtracking linear search is used so that the approximation conjugate gradient direction is computed only once, which ensures that the objective function decreases monotonically. The global convergence and local superlinear convergence of the algorithm are given. The numerical results are reported and show the effectiveness of the algorithm. [ABSTRACT FROM AUTHOR]

In this paper, we develop some three term nonlinear conjugate gradient methods based on the Hestenes–Stiefel (HS), the Polak–Ribière–Polyak (PRP) and the Liu–Storey (LS) methods. The proposed algorithms always generate sufficient descent directions which satisfy g k T d k = − ∥ g k ∥ 2 . When the Wolfe or the Armijo line search is used, we establish the global convergence of the proposed methods in a concise way. Moreover, the linear convergence rate of the methods is discussed as well. The extensive numerical results show the efficiency of the proposed methods. [ABSTRACT FROM AUTHOR]

The Conjugate Gradient (CG) method is renowned for its rapid convergence in optimization applications. Over the years, several modifications to CG methods have emerged to improve computational efficiency and tackle practical challenges. This paper presents a new three-term hybrid CG method for solving unconstrained optimization problems. This algorithm utilizes a search direction that combines Liu-Storey (LS) and Conjugate Descent (CD) CG coefficients and standardizes it using a spectral which acts as a scheme for the choices of the conjugate parameters. This resultant direction closely approximates the memoryless Broyden-Fletcher-Goldfarb-Shanno (BFGS) quasi-Newton direction, known for its bounded nature and compliance with the sufficient descent condition. The paper establishes the global convergence under standard Wolfe conditions and some appropriate assumptions. Additionally, the numerical experiments were conducted to emphasize the robustness and superior efficiency of this hybrid algorithm in comparison to existing approaches. [ABSTRACT FROM AUTHOR]

The subspace minimization conjugate gradient (SMCG) methods proposed by Yuan and Store are efficient iterative methods for unconstrained optimization, where the search directions are generated by minimizing the quadratic approximate models of the objective function at the current iterative point. Although the SMCG methods have illustrated excellent numerical performance, they are only used to solve unconstrained optimization problems at present. In this paper, we extend the SMCG methods and present an efficient SMCG method for solving nonlinear monotone equations with convex constraints by combining it with the projection technique, where the search direction is sufficiently descent.Under mild conditions, we establish the global convergence and R-linear convergence rate of the proposed method. The numerical experiment indicates that the proposed method is very promising. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new modification of a three-term derivative-free projection algorithm for solving a system of monotone nonlinear equations. The global convergence was proved under some suitable conditions, and the search direction of the proposed algorithm satisfies the sufficient descent property. Numerical computations were presented to display the efficacy of the proposed algorithm compared to some existing algorithms. Moreover, the proposed algorithm is applied to solve problems of image recovery. [ABSTRACT FROM AUTHOR]

This research paper is determining the impact of integrated energy management and optimization control for Novel power systems. Research objectives have been made based on integrating power system for reducing power consumption based on optimization parameters. Optimal control is mainly dealing with control law by finding out a given system and delivering certain criteria for achieving goals. There are mainly three parts for problem optimization purposes those are included decision, constraints and objective functions. Four different objectives has been made those are mainly discussed about the importance of the power systems regarding integrated management system. There are various factors that are affecting optimizations including the role of the multigrain recursion, global convergence, properties of the optimization model and local convergence. Along with this, the optimization control system has been played a crucial role for the power systems development purposes. Research has been based on the effective design of energy management for power systems. The primary application for the optimization techniques is based on the storage system, electromagnetic-based design and mapping design for the microwave structure. The rate of unsustainable energy management is increasing and sustainable energy management system is decreasing. The conclusion has been based on the significance of the optimization control for the power systems. [ABSTRACT FROM AUTHOR]

In this paper, we propose a stabilized sequential quadratic programming (SQP) method for optimization problems in function spaces. A form of the problem considered in this paper can widely formulate many types of applications, such as obstacle problems, optimal control problems, and so on. Moreover, the proposed method is based on the existing stabilized SQP method and can find a point satisfying the Karush-Kuhn-Tucker (KKT) or asymptotic KKT conditions. One of the remarkable points is that we prove its global convergence to such a point under some assumptions without any constraint qualifications. In addition, we guarantee that an arbitrary accumulation point generated by the proposed method satisfies the KKT conditions under several additional assumptions. Finally, we report some numerical experiments to examine the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

The most important advantage of conjugate gradient methods (CGs) is that these methods have low memory requirements and convergence speed. This paper contains two main parts that deal with two application problems, as follows. In the first part, three new parameters of the CG methods are designed and then combined by employing a convex combination. The search direction is a four-term hybrid form for modified classical CG methods with some newly proposed parameters. The result of this hybridization is the acquisition of a newly developed hybrid CGCG method containing four terms. The proposed CGCG has sufficient descent properties. The convergence analysis of the proposed method is considered under some reasonable conditions. A numerical investigation is carried out for an unconstrained optimization problem. The comparison between the newly suggested algorithm (CGCG) and five other classical CG algorithms shows that the new method is competitive with and in all statuses superior to the five methods in terms of efficiency reliability and effectiveness in solving large-scale, unconstrained optimization problems. The second main part of this paper discusses the image restoration problem. By using the adaptive median filter method, the noise in an image is detected, and then the corrupted pixels of the image are restored by using a new family of modified hybrid CG methods. This new family has four terms: the first is the negative gradient; the second one consists of either the HS-CG method or the HZ-CG method; and the third and fourth terms are taken from our proposed CGCG method. Additionally, a change in the size of the filter window plays a key role in improving the performance of this family of CG methods, according to the noise level. Four famous images (test problems) are used to examine the performance of the new family of modified hybrid CG methods. The outstanding clearness of the restored images indicates that the new family of modified hybrid CG methods has reliable efficiency and effectiveness in dealing with image restoration problems. [ABSTRACT FROM AUTHOR]

In this paper, we develop a nonmonotone line search strategy for minimization of the locally Lipschitz functions. First, the descent direction (DD) is defined based on ∂ϵf(⋅) where ϵ>0 . Next, we introduce a minimization algorithm to find a step length along the DD satisfying the nonsmooth nonmonotone Armijo condition. Choosing an adequate step length is the main purpose of the classic nonmonotone line search methods for a given DD, while in this paper both a search direction and step length are simultaneously computed. The global convergence of the minimization algorithm is proved by some assumptions on the DD. Finally, the proposed algorithm is implemented in the MATLAB environment and compared with another existing nonsmooth algorithm on some nonconvex nonsmooth optimization test problems. The efficiency of the proposed algorithm is shown by numerical results in solving some small-scale and large-scale nonsmooth optimization test problems. [ABSTRACT FROM AUTHOR]

In this paper, we propose an effective inertial-relaxed CGPM by considering the Armijo line search technique proposed by Amini and Kamandi (Numer Algorithms 70:559–570, 2015). The search direction designed possesses good properties. Furthermore, the global convergence is proved in the absence of the Lipschitz continuity. The numerical results on large-scale equations verify that our proposed method is efficient with the lower computational cost. In particular, applying it to sparse signal restoration indicates that the proposed algorithm is promising and competitive. [ABSTRACT FROM AUTHOR]

In this paper, we present a new fixed-point result to draw conclusions about the existence and uniqueness of the solution for a nonlinear Fredholm integral equation of the second kind with non-differentiable Nemytskii operator. To do this, we will transform the problem of locating a fixed point for an integral operator into the problem of locating a solution of an integral equation. Thus, assuming conditions on the Nemytskii operator, we will obtain a global convergence domain for the solution of the considered integral equation, taking for this a uniparametric family of derivativefree iterative processes with quadratic convergence. This result provides us a new fixed-point result for the integral operator considered. [ABSTRACT FROM AUTHOR]

In this paper, we propose a three-term derivative-free projection algorithm for handling large-scale nonlinear monotone equations with convex constrained. The search direction generated by the proposed algorithm satisfies sufficient descent condition at every iteration. Under some suitable conditions, the global convergence of the algorithm is established. Numerical experiments are provided to show the algorithm is promising and competitive for solving monotone nonlinear equations. In addition, we applied the algorithm to solve signal processing problem arising from compressive sensing. [ABSTRACT FROM AUTHOR]

Quantum computing is an emerging field that has had a significant impact on optimization. Among the diverse quantum algorithms, quantum gradient descent has become a prominent technique for solving unconstrained optimization (UO) problems. In this paper, we propose a quantum spectral Polak–Ribiére–Polyak (PRP) conjugate gradient (CG) approach. The technique is considered as a generalization of the spectral PRP method which employs a q-gradient that approximates the classical gradient with quadratically better dependence on the quantum variable q. Additionally, the proposed method reduces to the classical variant as the quantum variable q approaches closer to 1. The quantum search direction always satisfies the sufficient descent condition and does not depend on any line search (LS). This approach is globally convergent with the standard Wolfe conditions without any convexity assumption. Numerical experiments are conducted and compared with the existing approach to demonstrate the improvement of the proposed strategy. [ABSTRACT FROM AUTHOR]

For solving nonlinear monotone equations with convex constraints, Liu and Feng (Numer. Algoritm. 82(1):245–262, 2019) suggested a derivative-free iterative technique. Although they assert that the direction d k satisfies inequality (2.1), however, this is not true, as the derivation of the parameter θ k given by equation (2.7) is not correct. This led to Lemma 2.2, Lemma 3.1 and Theorem 3.1 in Liu and Feng (Numer. Algoritm. 82(1):245–262, 2019) not holding. In addition, Theorem 3.1 is still invalid as the bound for ‖ F (x k + α k ′ d k) ‖ was not established by the authors, instead the authors used the bound for ‖ F (x k + α k d k) ‖ as the bound for ‖ F (x k + α k ′ d k) ‖ . In this paper, We first describe the necessary adjustments and establish the bound for ‖ F (x k + α k ′ d k) ‖ , after which the proposed approach by Liu and Feng continues to converge globally. In addition, we provide some numerical results to support the adjustments. [ABSTRACT FROM AUTHOR]

In this paper, a spectral Dai and Yuan conjugate gradient (CG) method is proposed based on the generalized conjugacy condition for large-scale unconstrained optimization, in which the spectral parameter is motivated by some interesting theoretical features of quadratic convergence associated with the Newton method. Accordingly, utilizing the strong Wolfe line search to yield the step-length, the search direction of the proposed spectral method is sufficiently descending and converges globally. By applying some standard Euclidean optimization test functions, numerical results reports show the advantage of the method over some modified Dai and Yuan CG schemes in literature. In addition, the method also shows some reliable results, when applied to solve an image reconstruction model. [ABSTRACT FROM AUTHOR]

In this paper, we present a refined convergence analysis for a simple yet powerful method for computing a symmetric low-rank orthogonal approximation of a symmetric tensor proposed in the literature. The significance is that the assumption guaranteeing the global convergence is vastly relaxed to only on an input parameter of this algorithm. [ABSTRACT FROM AUTHOR]

The hybrid conjugate gradient (CG) method, recently proposed by authors in [11] for solving unconstrained optimization problems, has gained attention. This approach shares similarities with the Broyden-Fletcher-Goldfarb-Shanno memoryless quasi-Newton (BFGS). In this paper, we modified the search direction proposed in [11] and extended it to solve a nonlinear monotone system of equations. Our modified search direction is bounded and satisfies the descent condition regardless of any line search. With the aid of some valid assumptions, the global convergence of the algorithm is presented. To assess the effectiveness and reliability of our approach, we conduct numerical experiments using six test problems with extremely large dimensions ranging from 1000 to 800,000. Comparative numerical analysis with other methods reveals that our proposed approach excels in both theoretical and numerical aspects. In addition, we explore our proposed method to conduct sparse signal recovery in a setting of compressed sensing. [ABSTRACT FROM AUTHOR]

The paper considers the convergence of the complex block Jacobi diagonalization methods under the large set of the generalized serial pivot strategies. The global convergence of the block methods for Hermitian, normal and J -Hermitian matrices is proven. In order to obtain the convergence results for the block methods that solve other eigenvalue problems, such as the generalized eigenvalue problem, we consider the convergence of a general block iterative process which uses the complex block Jacobi annihilators and operators. [ABSTRACT FROM AUTHOR]

In this paper, we consider the optimization problems with 0/1-loss and sparsity constraints (0/1-LSCO) that involve two blocks of variables. First, we define a τ -stationary point of 0/1-LSCO, according to which we analyze the first-order necessary and sufficient optimality conditions. Based on these results, we then develop a gradient descent Newton pursuit algorithm (GDNP), and analyze its global and locally quadratic convergence under standard assumptions. Finally, numerical experiments on 1-bit compressed sensing demonstrate its superior performance in terms of a high degree of accuracy. [ABSTRACT FROM AUTHOR]

In this paper, we address some of the computational challenges associated with the RMIL+ conjugate gradient parameter by proposing an efficient conjugate gradient (CG) parameter along with its generalization to the Riemannian manifold. This parameter ensures the good convergence properties of the CG method in Riemannian optimization and it is formed by combining the structures of two classical CG methods. The extension utilizes the concepts of retraction and vector transport to establish sufficient descent property for the method via strong Wolfe line search conditions. Additionally, the scheme achieves global convergence using the scaled version of the Ring-Wirth nonexpansive condition. Finally, numerical experiments are conducted to validate the scheme's effectiveness. We consider both unconstrained Euclidean optimization test problems and Riemannian optimization problems. The results reveal that the performance of the proposed method is significantly influenced by the choice of line search in both Euclidean and Riemannian optimizations. [ABSTRACT FROM AUTHOR]

In this paper, we have presented a new formula by applying the conjugacy condition for the second-order Taylor expansion. In comparison with classic conjugate gradient methods, the new formula uses both available gradient and function value information. The formula's global convergence findings are described. Numerical results demonstrate that this strategy is effective and its application. [ABSTRACT FROM AUTHOR]

In this paper, with (Σ, g) being a closed Riemann surface, we analyze the possible concentration behavior of a heat flow related to the Trudinger-Moser energy. We obtain a long time existence for the flow. And along some sequence of times tk → + ∞, we can deduce the convergence of the flow in H2(Σ). Furthermore, the limit function is a critical point of the Trudinger-Moser functional under certain constraint. [ABSTRACT FROM AUTHOR]

In this paper, an approximate descent three-term derivative-free algorithm is developed for a large-scale system of nonlinear symmetric equations where the gradients and the difference of the gradients are computed approximately in order to avoid computing and storing the corresponding Jacobian matrices or their approximate matrices. The new method enjoys the sufficient descent property independent of the accuracy of line search strategies and the error bounds of these approximations are established. Under some mild conditions and a nonmonotone line search technique, the global and local convergence properties are established respectively. Numerical results indicate that the proposed algorithm outperforms the other similar ones available in the literature. [ABSTRACT FROM AUTHOR]

In this paper we propose a class of smoothing Newton-type methods for solving the secondorder cone complementarity problem (SOCCP). The proposed method design is based on a special regularized Chen-Harker-Kanzow-Smale (CHKS) smoothing function. When the solution set of the SOCCP is nonempty, our method has the following convergence properties: (i) it generates a bounded iteration sequence; (ii) the value of the merit function converges to zero; (iii) any accumulation point of the generated iteration sequence is a solution of the SOCCP; (iv) it has the local quadratic convergence rate under suitable assumptions. Some numerical results are reported. [ABSTRACT FROM AUTHOR]

In this paper, a nonmonotone Broyden family method is presented for the unconstrained optimization problems. The proposed line search technique is designed based on a nonmonotone line search proposed by Huang, Wan and Zhang (J. Comput. Appl. Math. 330: 586–604, 2018). The new line search technique can overcome the shortcoming that the nonmonotone line search proposed by Huang et. al only can be applied to conjugate gradient (CG) method, not Broyden family (including BFGS-type) method. The proposed method in this paper possesses some good properties: (i) a new nonmonotone line search technique is presented, (ii) the proposed nonmonotone line search technique can be applied not only to CG method but also to Broyden family (including BFGS-type) method, (iii) global convergence of the Broyden family method has been obtained with the proposed nonmonotone line search for nonconvex unconstrained minimization. In addition, numerical performance shows that the nonmonotone Broyden family method is more competitive versus the classical Broyden family method. [ABSTRACT FROM AUTHOR]

Al-Baali et al. (Comput. Optim. Appl. 60:89–110, 2015) have proposed a three-term conjugate gradient method which satisfies a sufficient descent condition and global convergence. In this paper, we extend this method to a family of three-term conjugate gradient projection methods with a restart procedure and their relaxed-inertial versions for constrained nonlinear pseudo-monotone equations. The accelerated gradient-descent method MSM and the relaxed-inertial strategy are incorporated into the proposed methods to obtain better computational performance. The global convergence of the extended methods are established theoretically without the Lipschitz continuity of the underlying mapping. Numerical results on constrained nonlinear equations show that the extended methods with different settings are efficient. The applicability and efficiency of the extended methods in the regularized decentralized logistic regression and sparse signal restoration problems are also presented and verified. [ABSTRACT FROM AUTHOR]

We propose the truncated nonsmooth Newton multigrid method (TNNMG) as a solver for the spatial problems of the small-strain brittle-fracture phase-field equations. TNNMG is a nonsmooth multigrid method that can solve biconvex, block-separably nonsmooth minimization problems with linear time complexity. It exploits the variational structure inherent in the problem, and handles the pointwise irreversibility constraint on the damage variable directly, without regularization or the introduction of a local history field. In the paper we introduce the method and show how it can be applied to several established models of phase-field brittle fracture. We then prove convergence of the solver to a solution of the nonsmooth Euler–Lagrange equations of the spatial problem for any load and initial iterate. On the way, we show several crucial convexity and regularity properties of the models considered here. Numerical comparisons to an operator-splitting algorithm show a considerable speed increase, without loss of robustness. [ABSTRACT FROM AUTHOR]

In this paper, two families of hybrid conjugate gradient methods with restart procedures are proposed. Their hybrid conjugate parameters are yielded by projection or convex combination of the classical parameters. Moreover, their restart procedures are given uniformly, which are determined by the proposed hybrid conjugate parameters. The search directions of the presented families satisfy the sufficient descent condition. Under usual assumption and the weak Wolfe line search, the proposed families are proved to be globally convergent. Finally, choosing a specific parameter for each family to solve large-scale unconstrained optimization problems, convex constrained nonlinear monotone equations and image restoration problems. All the numerical results are reported and analysed, which show that the proposed families of hybrid conjugate gradient methods are promising. [ABSTRACT FROM AUTHOR]

In this research paper, we introduce a novel gradient-free modified three-term conjugate gradient method designed to solve nonlinear equations subject to convex constraints. Our approach incorporates the projection scheme, which enhances the effectiveness of the proposed method. Building upon the modified three-term conjugate gradient method for solving M-tensor systems and ℓ1-norm-based nonsmooth optimization problems, our method can be regarded as an extension of their technique. By making mild assumptions, we establish the theoretical convergence properties of our iterative method. Through extensive numerical experiments, we demonstrate that our proposed approach is not only highly efficient but also outperforms other existing methods in terms of performance and accuracy. [ABSTRACT FROM AUTHOR]

It is well known that under the Wolfe–Powell inexact line search, the global convergence of the nonlinear conjugate gradient method always requires the Lipschitz continuous condition for nonconvex functions. In this paper, we find that the Lipschitz continuous condition is unnecessary for proving the global convergence for particular algorithms if its searching direction has the well-known sufficient descent property and the trust region feature. Thus, the global convergence of the family conjugate gradient algorithms proposed by Yuan et al. (Numer. Algorithms, 84(2020)) is established without the Lipschitz condition for nonconvex functions since they have these two properties. Furthermore, a new algorithm without a line search technique is presented, and its global convergence is also analysed under suitable assumptions. The numerical results show that the performance of the proposed algorithm is competitive with some particular problems. [ABSTRACT FROM AUTHOR]

The nonlinear programming makes use of quasi-Newton methods, a collection of optimization approaches when traditional Newton's method are challenging due to the calculation of the Jacobian matrix and its inverse. Since the Jacobian matrix is computationally difficult to compute and sometimes not available specifically when dealing with non-smooth monotone systems, quasi-Newton methods with superlinear convergence are preferred for solving nonlinear system of equations. This paper provides a new version of the derivative-free David–Fletcher–Powell (DFP) approach for dealing with nonlinear monotone system of equations with convex constraints. The optimal value of the scaling parameter is found by minimizing the condition number of the DFP matrix. Under certain assumptions, the proposed method has global convergence, required minimal storage and is derivative-free. When compared to standard methods, the proposed method requires less iteration, function evaluations, and CPU time. The image restoration test problems demonstrate the method's reliability and efficiency. [ABSTRACT FROM AUTHOR]