Search

In this paper, a three-term conjugate gradient algorithm is developed for solving large-scale unconstrained optimization problems. The search direction at each iteration of the algorithm is determined by rectifying the steepest descent direction with the difference between the current iterative points and that between the gradients. It is proved that such a direction satisfies the approximate secant condition as well as the conjugacy condition. The strategies of acceleration and restart are incorporated into designing the algorithm to improve its numerical performance. Global convergence of the proposed algorithm is established under two mild assumptions. By implementing the algorithm to solve 75 benchmark test problems available in the literature, the obtained results indicate that the algorithm developed in this paper outperforms the existent similar state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

This paper considers localization of a source or a sensor from distance measurements. We argue that linear algorithms proposed for this purpose are susceptible to poor noise performance. Instead given a set of sensors/anchors of known positions and measured distances of the source/sensor to be localized from them we propose a potentially nonconvex weighted cost function whose global minimum estimates the location of the source/sensor one seeks. The contribution of this paper is to provide nontrivial ellipsoidal and polytopic regions surrounding these sensors/anchors of known positions, such that if the object to be localized is in this region, localization occurs by globally exponentially convergent gradient descent in the noise free case. Exponential convergence in the noise free case represents practical convergence as it ensures graceful performance degradation in the presence of noise. These results guide the deployment of sensors/anchors so that small subsets can be made responsible for practical localization in geographical areas determined by our approach. [ABSTRACT FROM AUTHOR]

This paper focuses on a class of discrete-time interconnected pendulumlike systems. Sufficient conditions for the global convergence of systems with both structured and unstructured uncertainties in its linear part are established in terms of linear matrix inequalities (LMIs). In the traditional decentralized control of large scale systems, the effects of interconnections were seldom studied. In this paper, a square matrix specifying the interconnecting structure is introduced in order to discuss the effects of nonlinear interconnections. It is shown that the global convergence of the whole interconnected system can be achieved by designing an appropriate interconnection matrix. In order to solve the nonlinear matrix inequalities (NMIs) arising in the synthesis problem, a global optimization algorithm is presented which can be used to handle a class of NMI problems. An illustrative example is given to demonstrate the applicability and validity of the main results and the presented algorithm. [ABSTRACT FROM AUTHOR]

This paper proposes several globally convergent geometric optimization algorithms on Riemannian manifolds, which extend some existing geometric optimization techniques. Since any set of smooth constraints in the Euclidean space Rn (corresponding to constrained optimization) and the Rn space itself (corresponding to unconstrained optimization) are both special Riemannian manifolds, and since these algorithms are developed on general Riemannian manifolds, the techniques discussed in this paper provide a uniform framework for constrained and unconstrained optimization problems. Unlike some earlier works, the new algorithms have less restrictions in both convergence results and in practice. For example, global minimization in the one-dimensional search is not required. All the algorithms addressed in this paper are globally convergent. For some special Riemannian manifold other than Rn, the newalgorithms are very efficient. Convergence rates are obtained. Applications are discussed. [ABSTRACT FROM AUTHOR]

In this paper, a new sequential penalty algorithm, based on the L[sub ∞] exact penalty function, is proposed for a general nonlinear constrained optimization problem. The algorithm has the following characteristics: it can start from an arbitrary initial point; the feasibility of the subproblem is guaranteed; the penalty parameter is adjusted automatically; global convergence without any regularity assumption is proved. The update formula of the penalty parameter is new. It is proved that the algorithm proposed in this paper behaves equivalently to the standard SQP method after sufficiently many iterations. Hence, the local convergence results of the standard SQP method can be applied to this algorithm. Preliminary numerical experiments show the efficiency and stability of the algorithm. [ABSTRACT FROM AUTHOR]

This paper contains a new convergence analysis for the Lewis and Torczon generalized pattern search (GPS) class of methods for unconstrained and linearly constrained optimization. This analysis is motivated by a desire to understand the successful behavior of the algorithm under hypotheses that are satisfied by many practical problems. Specifically, even if the objective function is discontinuous or extended-valued, the methods find a limit point with some minimizing properties. Simple examples show that the strength of the optimality conditions at a limit point depends not only on the algorithm, but also on the directions it uses and on the smoothness of the objective at the limit point in question. The contribution of this paper is to provide a simple convergence analysis that supplies detail about the relation of optimality conditions to objective smoothness properties and to the defining directions for the algorithm, and it gives previous results as corollaries. [ABSTRACT FROM AUTHOR]

In this paper, a finite filter is used in the structure of the Barzilai–Browein (BB) gradient method in order to propose a new modified BB algorithm for solving large-scale unconstrained optimization problems. Our algorithm is equipped with a relaxed nonmonotone line search technique which allows the algorithm to enjoy the nonmonotonicity properties from scratch. Under some suitable conditions, the global convergence property of the new proposed algorithm is established. Numerical results on some test problems in CUTEr library show the efficiency and effectiveness of the new algorithm in practice too. [ABSTRACT FROM PUBLISHER]

Abstract: Recently, Gandomi and Alavi proposed a meta-heuristic optimization algorithm, called Krill Herd (KH), for global optimization [Gandomi AH, Alavi AH. Krill Herd: A New Bio-Inspired Optimization Algorithm. Communications in Nonlinear Science and Numerical Simulation, 17(12), 4831–4845, 2012.]. This paper represents an optimization method to global optimization using a novel variant of KH. This method is called the Stud Krill Herd (SKH). Similar to genetic reproduction mechanisms added to KH method, an updated genetic reproduction schemes, called stud selection and crossover (SSC) operator, is introduced into the KH during the krill updating process dealing with numerical optimization problems. The introduced SSC operator is originated from original Stud genetic algorithm. In SSC operator, the best krill, the Stud, provides its optimal information for all the other individuals in the population using general genetic operators instead of stochastic selection. This approach appears to be well capable of solving various functions. Several problems are used to test the SKH method. In addition, the influence of the different crossover types on convergence and performance is carefully studied. Experimental results indicate an instructive addition to the portfolio of swarm intelligence techniques. [Copyright &y& Elsevier]

This paper first proposes a trust region algorithm to obtain a stationary point of unconstrained multiobjective optimisation problem. Under suitable assumptions, the global convergence of the new algorithm is established. We then extend the trust region method to solve the non-smooth multiobjective optimisation problem. [ABSTRACT FROM PUBLISHER]

Abstract: Accurate kinetic models are of great significance for the simulation and analysis for hydrogen fuel cells. The proton exchange membrane (PEM) fuel cell is a complex nonlinear, multi-variable system. The mathematical modeling of PEM fuel cell usually leads to nonlinear parameter estimation problems which often contain more than one minimum. In this paper, a novel bio-inspired P systems based optimization algorithm, named BIPOA, is proposed to solve PEM fuel cell model parameter estimation problems. In BIPOA, the nested membrane structure and new rules such as adaptive mutation rule, partial migration rule and autophagy rule are combined to improve the algorithm''s global search capacities and convergence accuracy. Studies on some benchmark test functions indicate that the BIPOA outperforms the other two methods (PSOPS and GAs) in both convergence speed and accuracy. In addition, experimental results reveal that the model predictive outputs are in better agreement with the actual experimental data. Therefore, the BIPOA is a helpful and reliable technique for estimating the PEM fuel cell model parameters and is available to other complex parameter estimation problems of fuel cell models. [Copyright &y& Elsevier]

Abstract: A directed searching optimization algorithm (DSO) is proposed to solve constrained optimization problems in this paper. The proposed algorithm includes two important operations — position updating and genetic mutation. Position updating enables the non-best solution vectors to mimic the best one, which is beneficial to the convergence of the DSO; genetic mutation can increase the diversity of individuals, which is beneficial to preventing the premature convergence of the DSO. In addition, we adopt the penalty function method to balance objective and constraint violations. We can obtain satisfactory solutions for constrained optimization problems by combining the DSO and the penalty function method. Experimental results indicate that the proposed algorithm can be an efficient alternative on solving constrained optimization problems. [Copyright &y& Elsevier]

Abstract: Last few years have witnessed the development of various multi-objective evolutionary algorithms since they allow the generation of the overall Pareto front for multi-objective optimizations. With the problems in the real-world becoming more and more complex, however, no reported work in the literature focuses on the high-dimensional objective optimizations (HOPs). In this paper, we propose an evolutionary algorithm named HOEA (high-dimensional objective evolutionary algorithm) for HOPs. By adopting the concept of nonlinear definition for optimizing object, HOPs can be solved by HOEA, while the well-known multi-objective evolutionary algorithms work poorly on HOPs. We further analyze the nonlinear dynamic properties of HOEA on the basis of martingale theoretical framework. The theoretical results indicate that this new algorithm is indeed capable of achieving convergence. We also conduct experiments on HOEA with two representative test instances. The experimental results either confirm our theoretical results or show that the proposed algorithm is efficient and effective for HOPs. [Copyright &y& Elsevier]

Noise is present in many real-world continuous optimization problems. Stochastic search algorithms such as Evolution Strategies (ESs) have been proposed as effective search methods in such contexts. In this paper, we provide a mathematical analysis of the convergence of a (1+1)-ES on unimodal spherical objective functions in the presence of noise. We prove for a multiplicative noise model that for a positive expected value of the noisy objective function, convergence or divergence happens depending on the infimum of the support of the noise. Moreover, we investigate convergence rates and show that log-linear convergence is preserved in presence of noise. This result is a strong theoretical foundation of the robustness of ESs with respect to noise. [ABSTRACT FROM AUTHOR]

Abstract: The power method is an algorithm for computing the largest eigenvalue of matrix A in absolute value. To find the other eigenvalues one can apply the power method to the matrix for some shift σ. This scheme is called the inverse iteration method. Both of these two methods produce a convergence sequence and the limit is approximated by one of the iterates. In the chosen iterate, it may be difficult to estimate the global error, consisting of the truncation error and the round-off error. In this paper, by using the CESTAC method and the CADNA library, we propose a method for computing the optimal iterate, the iterate for which the global error is minimal. In the proposed method the accuracy of the computed eigenvalue may also be estimated. Some numerical examples are given to show the efficiency of the method. [Copyright &y& Elsevier]

In this paper a new hybrid conjugate gradient algorithm is proposed and analyzed. The parameter β k is computed as a convex combination of the Polak-Ribière-Polyak and the Dai-Yuan conjugate gradient algorithms, i.e. β=(1− θ k) β+ θ k β. The parameter θ k in the convex combination is computed in such a way that the conjugacy condition is satisfied, independently of the line search. The line search uses the standard Wolfe conditions. The algorithm generates descent directions and when the iterates jam the directions satisfy the sufficient descent condition. Numerical comparisons with conjugate gradient algorithms using a set of 750 unconstrained optimization problems, some of them from the CUTE library, show that this hybrid computational scheme outperforms the known hybrid conjugate gradient algorithms. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, we introduce a vector-valued Tikhonov-type regularization algorithm for an extended-valued multiobjective optimization problem. Under some mild conditions, we prove that any sequence generated by this algorithm converges to a weak Pareto optimal solution of the multiobjective optimization problem. Our results improve and generalize some known results. [Copyright &y& Elsevier]

Abstract: In this paper, by means of a new efficient identification technique of active constraints and the method of strongly sub-feasible direction, we propose a new sequential system of linear equations (SSLE) algorithm for solving inequality constrained optimization problems, in which the initial point is arbitrary. At each iteration, we first yield the working set by a pivoting operation and a generalized projection; then, three or four reduced linear equations with a same coefficient are solved to obtain the search direction. After a finite number of iterations, the algorithm can produced a feasible iteration point, and it becomes the method of feasible directions. Moreover, after finitely many iterations, the working set becomes independent of the iterates and is essentially the same as the active set of the KKT point. Under some mild conditions, the proposed algorithm is proved to be globally, strongly and superlinearly convergent. Finally, some preliminary numerical experiments are reported to show that the algorithm is practicable and effective. [Copyright &y& Elsevier]

Abstract: In this paper, we present a nonmonotone conic trust region method based on line search technique for unconstrained optimization. The new algorithm can be regarded as a combination of nonmonotone technique, line search technique and conic trust region method. When a trial step is not accepted, the method does not resolve the trust region subproblem but generates an iterative point whose steplength satisfies some line search condition. The function value can only be allowed to increase when trial steps are not accepted in close succession of iterations. The local and global convergence properties are proved under reasonable assumptions. Numerical experiments are conducted to compare this method with the existing methods. [Copyright &y& Elsevier]

Abstract: This paper presents an affine scaling optimal path approach in association with nonmonotonic interior backtracking line search technique for nonlinear optimization subject to linear constraints. We shall employ the eigensystem decomposition and affine scaling mapping to form affine scaling optimal curvilinear path very easily. By using interior backtracking line search technique, each iterate switches to trial step of strict interior feasibility. The nonmonotone criterion is used to speed up the convergence progress in the contours of objective function with large curvature. Theoretical analysis is given which prove that the proposed algorithm is globally convergent and has a local superlinear convergence rate under some reasonable conditions. The results of numerical experiments are reported to show the effectiveness of the proposed algorithm. [Copyright &y& Elsevier]

In this paper we present a new class of memory gradient methods for unconstrained optimization problems and develop some useful global convergence properties under some mild conditions. In the new algorithms, trust region approach is used to guarantee the global convergence. Numerical results show that some memory gradient methods are stable and efficient in practical computation. In particular, some memory gradient methods can be reduced to the BB method in some special cases. [ABSTRACT FROM AUTHOR]

Iterative shrinkage/thresholding (1ST) algorithms have been recently proposed to handle a class of convex unconstrained optimization problems arising in image restoration and other linear inverse problems. This class of problems results from combining a linear observation model with a nonquadratic regularizer (e.g., total variation or wavelet-based regularization). It happens that the convergence rate of these 1ST algorithms depends heavily on the linear observation operator, becoming very slow when this operator is ill-conditioned or ill-posed. In this paper, we introduce two-step 1ST (TwIST) algorithms, exhibiting much faster convergence rate than 1ST for ill-conditioned problems. For a vast class of nonquadratic convex regularizers (lP norms, some Besov norms, and total variation), we show that TwIST converges to a minimizer of the objective function, for a given range of values of its parameters. For noninvertible observation operators, we introduce a monotonic version of TwIST (MTwIST); although the convergence proof does not apply to this scenario, we give experimental evidence that MTwIST exhibits similar speed gains over 1ST. The effectiveness of the new methods are experimentally confirmed on problems of image deconvolution and of restoration with missing samples. [ABSTRACT FROM AUTHOR]

In this paper we propose a filter-trust-region algorithm for solving nonlinear optimization problems with simple bounds. It extends the technique of Gould et al. [Gould, N.I.M. Sainvitu, C. and Toint, Ph.L., 2005, A filter-trust-region method for unconstrained optimization. SIAM Journal on Optimization, 16(2), 341-357.] designed for unconstrained optimization problems. The two main ingredients of the method are a filter-trust-region algorithm and a gradient-projection method. The algorithm is shown to be globally convergent to at least one first-order critical point. Numerical experiments on a large set of problems are also reported. [ABSTRACT FROM AUTHOR]

Direct search optimization algorithms are becoming an important alternative to well-established gradient based methods. Due to the fact that a single cost function evaluation may take a substantial amount of time, optimization can be a lengthy process. In order to shorten the run time one often resorts to parallel algorithms. Asynchronous algorithms are particularly efficient since they have no synchronization points. This paper is an attempt to establish a convergence theory for a class of such parallel direct search algorithms. The notion of a search direction generator (SDG) is introduced. An algorithmic framework for parallel distributed optimization methods based on SDGs is presented along with the corresponding convergence theory. The theory almost completely decouples the stepsize control from the sufficient descent requirement, which is necessary for the finite termination of the algorithm's inner loop. The proposed framework has several attributes considered very favourable in loosely coupled parallel systems (e.g. clusters of workstations), such as fault tolerance and scalability. The framework is illustrated by optimizing a set of test problems on a cluster of workstations. In all tested cases, a speedup was obtained that increased with the increasing number of workstations. Fault tolerance and scalability of the framework were also demonstrated by removing and adding workstations to the cluster while an optimization run was in progress. [ABSTRACT FROM AUTHOR]

In this paper we discuss the polynomiality of a feasible version of Mehrotra's predictor-corrector algorithm whose variants have been widely used in several interior point method (IPM)-based optimization packages. A numerical example is given that shows that the adaptive choice of centering parameter and correction terms in this algorithm may lead to small steps being taken in order to keep the iterates in a large neighborhood of the central path, which is important for proving polynomial complexity properties of this method. Motivated by this example, we introduce a safeguard in Mehrotra's algorithm that keeps the iterates in the prescribed neighborhood and allows us to obtain a positive lower bound on the step size. This safeguard strategy is also used when the affine scaling direction performs poorly. We prove that the safeguarded algorithm will terminate after at most O(n2log(x°}T s° /e) iterations. By modestly modifying the corrector direction, we reduce the iteration complexity to O(nlog(x°}T s° /e). To ensure fast asymptotic convergence of the algorithm, we changed Mehrotra's updating scheme of the centering parameter slightly while keeping the safeguard. The new algorithms have the same order of iteration complexity as the safeguarded algorithms but enjoy superlinear convergence as well. Numerical results using the McIPM and LIPSOL software packages are reported. [ABSTRACT FROM AUTHOR]

Abstract: This paper studies a successive partitioning group correction algorithm and its some modified algorithms for solving large scale sparse unconstrained optimization problems. The methods depend on a symmetric consistent partition of the columns of the Hessian matrix. A q-superlinear convergence result and an r-convergence rate estimate show that the methods have good local convergence properties. The numerical results show that the methods, especially the modified algorithms, may be competitive with some current used algorithms. [Copyright &y& Elsevier]

In this paper we present a relaxed version of an extragradient–proximal point algorithm recently proposed by Solodov and Svaiter for finding a zero of a maximal monotone operator defined on a Hilbert space. The aim is to introduce a family of parameters in order to accelerate the rate of convergence of this algorithm. First we study the convergence and rate of convergence of the relaxed algorithms and then we apply them to the generalized variational inequality problem. For this problem, the operator is a sum of two operators: the first one is single-valued, monotone and continuous and the second one is the subdifferential of a nonsmooth lower semicontinuous proper convex function f. To make the subproblems easier to solve, we consider, as in the bundle methods, piecewise linear convex approximations of f. We explain how to construct these approximations and how the subproblems fall within the framework of our relaxed extragradient–proximal point algorithm. We prove the convergence of the resulting algorithm without assuming a Dunn property on the single-valued operator. Finally, we report some numerical experiences to illustrate the behavior of our implementable algorithm for different values of the relaxation factor. A comparison with another algorithm is also given. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, a kind of nonlinear optimization problems with nonlinear inequality constraints are discussed, and a new SQP feasible descent algorithm for solving the problems is presented. At each iteration of the new algorithm, a convex quadratic program (QP) which always has feasible solution is solved and a master direction is obtained, then, an improved (feasible descent) direction is yielded by updating the master direction with an explicit formula, and in order to avoid the Maratos effect, a height-order correction direction is computed by another explicit formula of the master direction and the improved direction. The new algorithm is proved to be globally convergent and superlinearly convergent under mild conditions without the strict complementarity. Furthermore, the quadratic convergence rate of the algorithm is obtained when the twice derivatives of the objective function and constrained functions are adopted. Finally, some numerical tests are reported. [Copyright &y& Elsevier]

This paper develops a theory for the global solution of nonconvex optimization problems with parameter-embedded linear dynamic systems. A quite general problem formulation is introduced and a solution is shown to exists. A convexity theory for integrals is then developed to construct convex relaxations for utilization in a branch-and-bound framework to calculate a global minimum. Interval analysis is employed to generate bounds on the state variables implied by the bounds on the embedded parameters. These bounds, along with basic integration theory, are used to prove convergence of the branch-and-bound algorithm to the global minimum of the optimization problem. The implementation of the algorithm is then considered and several numerical case studies are examined thoroughly. [ABSTRACT FROM AUTHOR]

This paper presents a new trust-region algorithm for n-dimension nonlinear optimization subject to m nonlinear inequality constraints. Equivalent KKT conditions are derived, which is the basis for constructing the new algorithm. Global convergence of the algorithm to a first-order KKT point is established under mild conditions on the trial steps, local quadratic convergence theorem is proved for nondegenerate minimizer point. Numerical experiment is presented to show the effectiveness of our approach. [ABSTRACT FROM AUTHOR]

Trust region (TR) algorithms are a class of recently developed algorithms for nonlinear optimization. A new family of TR algorithms for unconstrained optimization, which is the extension of the usual TR method, is presented in this paper. When the objective function is bounded below and continuously differentiable, and the norm of the Hesse approximations increases at most linearly with the iteration number, we prove the global convergence of the algorithms. Limited numerical results are reported, which indicate that our new TR algorithm is competitive. [ABSTRACT FROM AUTHOR]