Search

In this paper, a three-term conjugate gradient algorithm is developed for solving large-scale unconstrained optimization problems. The search direction at each iteration of the algorithm is determined by rectifying the steepest descent direction with the difference between the current iterative points and that between the gradients. It is proved that such a direction satisfies the approximate secant condition as well as the conjugacy condition. The strategies of acceleration and restart are incorporated into designing the algorithm to improve its numerical performance. Global convergence of the proposed algorithm is established under two mild assumptions. By implementing the algorithm to solve 75 benchmark test problems available in the literature, the obtained results indicate that the algorithm developed in this paper outperforms the existent similar state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

Plant Propagation Algorithms (PPA) are powerful and flexible solvers for optimisation problems. They are nature-inspired heuristics which can be applied to any optimisation/search problem. There is a growing body of research, mainly experimental, on PPA in the literature. Little, however, has been done on the theoretical front. Given the prominence this algorithm is gaining in terms of performance on benchmark problems as well as practical ones, some theoretical insight into its convergence is needed. The current paper is aimed at fulfilling this by providing a sketch for a global convergence analysis. [ABSTRACT FROM AUTHOR]

The problem of finding a vector with the fewest nonzero elements that satisfies an underdetermined system of linear equations is an NP-complete problem that is typically solved numerically via convex heuristics or nicely behaved nonconvex relaxations. In this paper we consider the elementary method of alternating projections (MAP) for solving the sparsity optimization problem without employing convex heuristics. In a parallel paper we recently introduced the restricted normal cone which generalizes the classical Mordukhovich normal cone and reconciles some fundamental gaps in the theory of sufficient conditions for local linear convergence of the MAP algorithm. We use the restricted normal cone together with the notion of superregularity, which is inherently satisfied for the affine sparse optimization problem, to obtain local linear convergence results with estimates for the radius of convergence of the MAP algorithm applied to sparsity optimization with an affine constraint. [ABSTRACT FROM AUTHOR]

In this paper, denoising of smooth ( H10-regular) images is considered. The purpose of the paper is basically twofold. First, to compare the denoising methods based on L1- and L2-fitting. Second, to analyze and realize an active-set method for solving the non-smooth optimization problem arising from the former approach. More precisely, we formulate the algorithm, proof its convergence, and give an efficient numerical realization. Several numerical experiments are presented, where the convergence of the proposed active-set algorithm is studied and the denoising properties of the methods based on L1- and L2-fitting are compared. Also a heuristic method for determining the regularization parameter is presented and tested. [ABSTRACT FROM AUTHOR]

In this paper we study the convex problem of optimizing the sum of a smooth function and a compactly supported nonsmooth term with a specific separable form. We analyze the block version of the generalized conditional gradient method when the blocks are chosen in a cyclic order. A global sublinear rate of convergence is established for two different stepsize strategies commonly used in this class of methods. Numerical comparisons of the proposed method to both the classical conditional gradient algorithm and its random block version demonstrate the effectiveness of the cyclic block update rule. [ABSTRACT FROM AUTHOR]

Coordinate descent algorithms solve optimization problems by successively performing approximate minimization along coordinate directions or coordinate hyperplanes. They have been used in applications for many years, and their popularity continues to grow because of their usefulness in data analysis, machine learning, and other areas of current interest. This paper describes the fundamentals of the coordinate descent approach, together with variants and extensions and their convergence properties, mostly with reference to convex objectives. We pay particular attention to a certain problem structure that arises frequently in machine learning applications, showing that efficient implementations of accelerated coordinate descent algorithms are possible for problems of this type. We also present some parallel variants and discuss their convergence properties under several models of parallel execution. [ABSTRACT FROM AUTHOR]

The stagnation phenomena regularly happen in solving complex optimization problems by a class of computational intelligence algorithms which lack theoretical guidance for their parameters setting. The search characteristics of these algorithms are analyzed in this paper, based on which an adaptive computational intelligence optimization algorithm called cloud drops algorithm is proposed by adopting a cloud model to express the randomness and fuzziness of its search process. The cloud drops algorithm is characterized by representing, mining and recreating the uncertain knowledge about its search process for the optimal solution. None of search parameters are predefined in the proposed algorithm and, whatever the initial solution set is, the whole system can adaptively approach to the global optimum. Based on the theory of stochastic processes, the almost sure convergence of the proposed algorithm is proved under certain conditions by introducing a martingale approach into traditional Markov Chain analysis. Two benchmark problems are tested with the proposed algorithm and the other two existing algorithms as a comparison. The results show that the proposed algorithm has faster convergence speed, better self-adaptability, and stronger ability to deal with stagnation phenomena effectively. [ABSTRACT FROM AUTHOR]

Abstract: The Prediction Error Method (PEM) is related to an optimization problem built on input/output data collected from the system to be identified. It is often hard to find the global solution of this optimization problem because the corresponding objective function presents local minima and/or the search space is constrained to a nonconvex set. The shape of the cost function, and hence the difficulty in solving the optimization problem, depends directly on the experimental conditions, more specifically on the spectrum of the input/output data collected from the system. Therefore, it seems plausible to improve the convergence to the global minimum by properly choosing the spectrum of the input; in this paper, we address this problem. We present a condition for convergence to the global minimum of the cost function and propose its inclusion in the input design. We present the application of the proposed approach to case studies where the algorithms tend to get trapped in nonglobal minima. [Copyright &y& Elsevier]

This paper first proposes a trust region algorithm to obtain a stationary point of unconstrained multiobjective optimisation problem. Under suitable assumptions, the global convergence of the new algorithm is established. We then extend the trust region method to solve the non-smooth multiobjective optimisation problem. [ABSTRACT FROM PUBLISHER]

Abstract: In this paper, a new algorithm is proposed to solve the nonlinear constrained optimization problems. Unlike sequential quadratic programming (SQP) type methods, this algorithm does not involve solutions of quadratic programs. It is merely necessary to solve systems of linear equations with small scale to obtain a direction. The scheme is based on an idea of -effective active set strategies. The theoretical analysis shows that global and superlinear convergence can be induced under some suitable conditions. [Copyright &y& Elsevier]

Abstract: This paper presents an augmented Lagrangian methodology with a stochastic population based algorithm for solving nonlinear constrained global optimization problems. The method approximately solves a sequence of simple bound global optimization subproblems using a fish swarm intelligent algorithm. A stochastic convergence analysis of the fish swarm iterative process is included. Numerical results with a benchmark set of problems are shown, including a comparison with other stochastic-type algorithms. [Copyright &y& Elsevier]

Abstract: Many real life problems can be stated as a minimax optimization problem, such as the problems in economics, finance, management, engineering and other fields. In this paper, we present an algorithm with nonmonotone strategy and second-order correction technique for minimax optimization problems. Using this scheme, the new algorithm can overcome the difficulties of the Maratos effect occurred in the nonsmooth optimization, and the global and superlinear convergence of the algorithm can be achieved accordingly. Numerical experiments indicate some advantages of this scheme. [Copyright &y& Elsevier]