Search

Penalty function is an important tool in solving many constrained optimization problems in areas such as industrial design and management. In this paper, we study exactness and algorithm of an objective penalty function for inequality constrained optimization. In terms of exactness, this objective penalty function is at least as good as traditional exact penalty functions. Especially, in the case of a global solution, the exactness of the proposed objective penalty function shows a significant advantage. The sufficient and necessary stability condition used to determine whether the objective penalty function is exact for a global solution is proved. Based on the objective penalty function, an algorithm is developed for finding a global solution to an inequality constrained optimization problem and its global convergence is also proved under some conditions. Furthermore, the sufficient and necessary calmness condition on the exactness of the objective penalty function is proved for a local solution. An algorithm is presented in the paper in finding a local solution, with its convergence proved under some conditions. Finally, numerical experiments show that a satisfactory approximate optimal solution can be obtained by the proposed algorithm. [ABSTRACT FROM AUTHOR]

Differential Evolution (DE) is a simple and efficient optimizer, especially for continuous optimization. For these reasons DE has often been employed for solving various engineering problems. On the other hand, the DE structure has some limitations in the search logic, since it contains too narrow a set of exploration moves. This fact has inspired many computer scientists to improve upon DE by proposing modifications to the original algorithm. This paper presents a survey on DE and its recent advances. A classification, into two macro-groups, of the DE modifications is proposed here: (1) algorithms which integrate additional components within the DE structure, (2) algorithms which employ a modified DE structure. For each macro-group, four algorithms representative of the state-of-the-art in DE, have been selected for an in depth description of their working principles. In order to compare their performance, these eight algorithm have been tested on a set of benchmark problems. Experiments have been repeated for a (relatively) low dimensional case and a (relatively) high dimensional case. The working principles, differences and similarities of these recently proposed DE-based algorithms have also been highlighted throughout the paper. Although within both macro-groups, it is unclear whether there is a superiority of one algorithm with respect to the others, some conclusions can be drawn. At first, in order to improve upon the DE performance a modification which includes some additional and alternative search moves integrating those contained in a standard DE is necessary. These extra moves should assist the DE framework in detecting new promising search directions to be used by DE. Thus, a limited employment of these alternative moves appears to be the best option in successfully assisting DE. The successful extra moves are obtained in two ways: an increase in the exploitative pressure and the introduction of some randomization. This randomization should not be excessive though, since it would jeopardize the search. A proper increase in the randomization is crucial for obtaining significant improvements in the DE functioning. Numerical results show that, among the algorithms considered in this study, the most efficient additional components in a DE framework appear to be the population size reduction and the scale factor local search. Regarding the modified DE structures, the global and local neighborhood search and self-adaptive control parameter scheme, recently proposed in literature, seem to be the most promising modifications. [ABSTRACT FROM AUTHOR]

Iterative shrinkage/thresholding (1ST) algorithms have been recently proposed to handle a class of convex unconstrained optimization problems arising in image restoration and other linear inverse problems. This class of problems results from combining a linear observation model with a nonquadratic regularizer (e.g., total variation or wavelet-based regularization). It happens that the convergence rate of these 1ST algorithms depends heavily on the linear observation operator, becoming very slow when this operator is ill-conditioned or ill-posed. In this paper, we introduce two-step 1ST (TwIST) algorithms, exhibiting much faster convergence rate than 1ST for ill-conditioned problems. For a vast class of nonquadratic convex regularizers (lP norms, some Besov norms, and total variation), we show that TwIST converges to a minimizer of the objective function, for a given range of values of its parameters. For noninvertible observation operators, we introduce a monotonic version of TwIST (MTwIST); although the convergence proof does not apply to this scenario, we give experimental evidence that MTwIST exhibits similar speed gains over 1ST. The effectiveness of the new methods are experimentally confirmed on problems of image deconvolution and of restoration with missing samples. [ABSTRACT FROM AUTHOR]

Abstract: The problem of Systems Identification starts with a time-series of observed data and tries to determine the simplest model capable of exhibiting the observed behavior. This optimization problem searches the model from a space of possible models. In this paper, we present the theory and algorithms to perform Qualitative and Quantitative Systems Identification for Linear Time-Invariant Dynamic Systems. The methods described here are based on successive elimination of the components of the system''s response. Sinusoidals of high frequencies are eliminated first, then their carrying waves. We continue with the process until we obtain a non-oscillatory carrier. At this point, we determine the order of the carrier. This procedure allows us to determine how many sinusoidal components and exponential components are found in the impulse response of the system under study. The number of components determines the order of the system. The paper is composed of two important parts, the statement of some mathematical properties of the responses of Linear Time Invariant Dynamic Systems, and the proposal of a set of filters that allows us to implement the recognition algorithm. We present the application of the proposed methodology to analyze and model the electrical circuits and electrical power systems. [Copyright &y& Elsevier]

Abstract: We consider a class of global optimization problems, in which the objective function is Lebesgue integrable. In this paper, we present a sufficient and necessary condition for computing the essential infimum. This optimality condition can be used to design algorithms for solving the global optimization problem. [Copyright &y& Elsevier]

Abstract: In this paper, a new adaptive differential evolution algorithm (ADEA) is proposed for multiobjective optimization problems. In ADEA, the variable parameter F based on the number of the current Pareto-front and the diversity of the current solutions is given for adjusting search size in every generation to find Pareto solutions in mutation operator, and the select operator combines the advantages of DE with the mechanisms of Pareto-based ranking and crowding distance sorting. ADEA is implemented on five classical multiobjective problems, the results illustrate that ADEA efficiently achieves two goals of multiobjective optimization problems: find the solutions converge to the true Pareto-front and uniform spread along the front. [Copyright &y& Elsevier]

Abstract: Many practical problems can be formulated as constrained optimization problems. Due to the simple concept and easy implementation, the penalty function method has been one of the most common techniques to handle constraints. However, the performance of this technique greatly relies on the setting of penalty factors, which are usually determined by manual trial and error, and the suitable penalty factors are often problem-dependent and difficult to set. In this paper, a differential evolution approach based on a co-evolution mechanism, named CDE, is proposed to solve the constrained problems. First, a special penalty function is designed to handle the constraints. Second, a co-evolution model is presented and differential evolution (DE) is employed to perform evolutionary search in spaces of both solutions and penalty factors. Thus, the solutions and penalty factors evolve interactively and self-adaptively, and both the satisfactory solutions and suitable penalty factors can be obtained simultaneously. Simulation results based on several benchmark functions and three well-known constrained design problems as well as comparisons with some existed methods demonstrate the effectiveness, efficiency and robustness of the proposed method. [Copyright &y& Elsevier]

Studies of the convergence of algorithms often revolve around the existence of a function with respect to which monotonic descent is required. In this paper, we show that under relatively lenient conditions, "stage-dependent descent" (not necessarily monotonic) is sufficient to guarantee convergence. This development also provides the impetus to examine optimization algorithms. We show that one of the important avenues in the study of convergence, namely, the theory of epi-convergence imposes stronger conditions than are necessary to establish the convergence of an optimization algorithm. Working from a relaxation of epi-convergence, we introduce the notion of ∂-compatibility, and prove several results that permit relaxations of conditions imposed by previous approaches to algorithmic convergence. Finally, to illustrate the usefulness of the concepts, we combine stage-dependent descent with results derivable from ∂-compatibility to provide a basis for the convergence of a general algorithmic statement that might be used for stochastic and nondifferentiable optimization. [ABSTRACT FROM AUTHOR]

We apply nonsmooth optimization techniques to classification problems, with particular reference to the Transductive Support Vector Machine (TSVM) approach, where the considered decision function is nonconvex and nondifferentiable, hence difficult to minimize. We present some numerical results obtained by running the proposed method on some standard test problems drawn from the binary classification literature. [ABSTRACT FROM AUTHOR]

The article presents some approximate optimal solutions and a generalized augmented Lagrangian in nonlinear programming. It also establishes dual function and dual problem based on the generalized augmented Lagrangian. It also obtain approximate KKT necessary optimality condition of the generalized augmented Lagrangian dual problem. The article further proves that the approximate stationary points of generalized augmented Lagrangian problem converge to that of the original problem.

We discuss a multigrid approach to the optimization of systems governed by differential equations. Such optimization problems appear in many applications and are of a different nature than systems of equations. Our approach uses an optimization-based multigrid algorithm in which the multigrid algorithm relies explicitly on nonlinear optimization models as subproblems on coarser grids. Our goal is not to argue for a particular optimization-based multigrid algorithm, but instead to demonstrate how multigrid can be used to accelerate nonlinear programming algorithms. Furthermore, using several model problems we give evidence (both theoretical and numerical) that the optimization setting is well suited to multigrid algorithms. Some of the model problems show that the optimization problem may be more amenable to multigrid than the governing differential equation. In addition, we relate the multigrid approach to more traditional optimization methods as further justification for the use of an optimization-based multigrid algorithm. [ABSTRACT FROM AUTHOR]

Abstract. We consider primal-dual algorithms for certain types of infinite-dimensional optimization problems. Our approach is based on the generalization of the technique of finite dimensional Euclidean Jordan algebras to the case of infinite-dimensional JB-algebras of finite rank. This generalization enables us to develop polynomial-time primal-dual algorithms for "infinite-dimensional second-order cone programs." We consider as an example a long step primal dual algorithm based on the Nesterov-Todd direction. It is shown that this algorithm can be generalized along with complexity estimates to the infinite-dimensional situation under consideration. An application is given to an important problem of control theory: multi-criteria analytic design of the linear regulator. The calculation of the Nesterov Todd direction requires in this case solving one matrix differential Riccati equation plus solving a finite-dimensional system of linear algebraic equations on each iteration. The number of equations and unknown variables of this algebraic system is m + 1, where m is a number of quadratic performance criteria. [ABSTRACT FROM AUTHOR]