Search

Stochastic gradient descent algorithm is a classical and useful method for stochastic optimisation. While stochastic gradient descent has been theoretically investigated for decades and successfully applied in machine learning such as training of deep neural networks, it essentially relies on obtaining the unbiased estimates of gradients/subgradients of the objective functions. In this paper, by constructing the randomised differences of the objective function, a gradient‐free algorithm, named the stochastic randomised‐difference descent algorithm, is proposed for stochastic convex optimisation. Under the strongly convex assumption of the objective function, it is proved that the estimates generated from stochastic randomised‐difference descent converge to the optimal value with probability one, and the convergence rates of both the mean square error of estimates and the regret functions are established. Finally, some numerical examples are prsented. [ABSTRACT FROM AUTHOR]

In this paper, two modified spectral conjugate gradient methods which satisfy sufficient descent property are developed for unconstrained optimization problems. For uniformly convex problems, the first modified spectral type of conjugate gradient algorithm is proposed under the Wolfe line search rule. Moreover, the search direction of the modified spectral conjugate gradient method is sufficiently descent for uniformly convex functions. Furthermore, according to the Dai-Liao's conjugate condition, the second spectral type of conjugate gradient algorithm can generate some sufficient decent direction at each iteration for general functions. Therefore, the second method could be considered as a modification version of the Dai-Liao's algorithm. Under the suitable conditions, the proposed algorithms are globally convergent for uniformly convex functions and general functions. The numerical results show that the approaches presented in this paper are feasible and efficient. [ABSTRACT FROM AUTHOR]

An R2 indicator-based multi-objective particle swarm optimiser (R2-MOPSO) can obtain well-convergence and well-distributed solutions while solving two and three objectives optimisation problems. However, R2-MOPSO faces difficulty to tackle many-objective optimisation problems because balancing convergence and diversity is a key issue in high-dimensional objective space. In order to address this issue, this paper proposes a novel algorithm, named R2-MaPSO, which combines the R2 indicator and decomposition-based archiving pruning strategy into particle swarm optimiser for many-objective optimisation problems. The innovations of the proposed algorithm mainly contains three crucial factors: (1) A bi-level archiving maintenance approach based on the R2 indicator and objective space decomposition strategy is designed to balance convergence and diversity. (2) The global-best leader selection is based on the R2 indicator and the personal-best leader selection is based on the Pareto dominance. Meanwhile, the objective space decomposition leader selection adopts the feedback information from the bi-level archive. (3) A new velocity updated method is modified to enhance the exploration and exploitation ability. In addition, an elitist learning strategy and a smart Gaussian learning strategy are embedded into R2-MaPSO to help the algorithm jump out of the local optimal front. The performance of the proposed algorithm is validated and compared with some algorithms on a number of unconstraint benchmark problems, i.e. DTLZ1-DTLZ4, WFG test suites from 3 to 15 objectives. Experimental results have demonstrated a better performance of the proposed algorithm compared with several multi-objective particle swarm optimisers and multi-objective evolutionary algorithms for many-objective optimisation problems. [ABSTRACT FROM AUTHOR]

In the literature, there are a few researches to design some parameters in the proximal point algorithm (PPA), especially for the multi-objective convex optimizations. Introducing some parameters to PPA can make it more flexible and attractive. Mainly motivated by our recent work [Bai et al. A parameterized proximal point algorithm for separable convex optimization. Optim Lett. (2017) doi:10.1007/s11590-017-1195-9], in this paper we develop a general parameterized PPA with a relaxation step for solving the multi-block separable structured convex programming. By making use of the variational inequality and some mathematical identities, the global convergence and the worst-case convergence rate of the proposed algorithm are established. Preliminary numerical experiments on solving a sparse matrix minimization problem from statistical learning validate that our algorithm is more efficient than several state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

This paper proposes a nonmonotone scaled conjugate gradient algorithm for solving large-scale unconstrained optimization problems, which combines the idea of scaled memoryless Broyden-Fletcher-Goldfarb-Shanno preconditioned conjugate gradient method with the nonmonotone technique. An attractive property of the proposed method is that the search direction always provides sufficient descent step at each iteration. This property is independent of the line search used. Under appropriate assumptions, the method is proven to possess global convergence for nonconvex smooth functions, and R-linear convergence for strongly convex functions. Preliminary numerical results and related comparisons show the efficiency of the proposed method in practical computation. [ABSTRACT FROM AUTHOR]

We propose a derivative-free trust region algorithm with a nonmonotone filter technique for bound constrained optimization. The derivative-free strategy is applied for special minimization functions in which derivatives are not all available. A nonmonotone filter technique ensures not only the trust region feature but also the global convergence under reasonable assumptions. Numerical experiments demonstrate that the new algorithm is effective for bound constrained optimization. Locally, optimal parameters with respect to overall computational time on a set of test problems are identified. The performance of the best choice of parameter values obtained by the algorithm we presented which differs from traditionally used values indicates that the algorithm proposed in this paper has a certain advantage for the nondifferentiable optimization problems. [ABSTRACT FROM AUTHOR]

Considering that extending the concept of bees partitioning into subgroups of foragers to the onlooker phase of the cooperative learning artificial bee colony (CLABC) strategy is not only feasible from algorithmic viewpoint but might reflect real behavioral foraging characteristics of bee swarms, this paper studies whether the performance of CLABC can be enhanced by developing a new model for the proposed cooperative foraging scheme. Relying on this idea, we design a modified cooperative learning artificial bee colony algorithm, referred to as mCLABC. The design procedure is built upon the definition of a partitioning scheme of onlookers allowing the generation of subgroups of foragers that might evolve differently by using specific solution search rules. In order to improve the involving of local and global search at both employed and onlooker levels, the multiple search mechanism is further tuned and scheduled according to a random selection strategy defined on the evolving parameters. Moreover, a detailed performance and robustness study of the proposed algorithm dealing with the analysis of the impact of different structural and parametric settings is conducted on benchmark optimization problems. Superior convergence performance, better solution quality, and strong robustness are the main features of the proposed strategy in comparison with recent ABC variants and advanced methods. [ABSTRACT FROM AUTHOR]

Adaptive regularized framework using cubics has emerged as an alternative to line-search and trust-region algorithms for smooth nonconvex optimization, with an optimal complexity among second-order methods. In this paper, we propose and analyze the use of an iteration dependent scaled norm in the adaptive regularized framework using cubics. Within such a scaled norm, the obtained method behaves as a line-search algorithm along the quasi-Newton direction with a special backtracking strategy. Under appropriate assumptions, the new algorithm enjoys the same convergence and complexity properties as adaptive regularized algorithm using cubics. The complexity for finding an approximate first-order stationary point can be improved to be optimal whenever a second-order version of the proposed algorithm is regarded. In a similar way, using the same scaled norm to define the trust-region neighborhood, we show that the trust-region algorithm behaves as a line-search algorithm. The good potential of the obtained algorithms is shown on a set of large-scale optimization problems. [ABSTRACT FROM AUTHOR]

Many-objective optimization problems refer to problems that hold more than three conflicting objectives, which are more challenging than those with two or three objectives. Membrane computing models, usually termed P systems, are a class of living cell-inspired computing models, which provide a rich framework for solving a variety of challenging problems. In this paper, a membrane computing model-based algorithm is proposed for many-objective optimization. Specifically, the population in the skin membrane is divided into two subpopulations, one used for guiding the convergence of populations in the internal membrane, while the other taking charge of the diversity of populations. Experimental results on benchmark test problems for many-objective optimization indicate the superiority of the developed algorithm over existing evolutionary many-objective optimization algorithms and P systems based multi-objective optimization algorithms. [ABSTRACT FROM AUTHOR]

The purpose of this paper is to introduce and study the general split equality equilibrium problem and the general split equilibrium problem in Hilbert spaces. In order to solve these problems, a new simultaneous iterative algorithm is proposed and several strong convergence theorems for the sequences generated by the algorithm are proved. As applications, we utilize our results to study the general split equality optimization problem and the general split optimization problem. The results presented in the paper extend and improve some recent results. [ABSTRACT FROM AUTHOR]

In this paper, a three-term conjugate gradient algorithm is developed for solving large-scale unconstrained optimization problems. The search direction at each iteration of the algorithm is determined by rectifying the steepest descent direction with the difference between the current iterative points and that between the gradients. It is proved that such a direction satisfies the approximate secant condition as well as the conjugacy condition. The strategies of acceleration and restart are incorporated into designing the algorithm to improve its numerical performance. Global convergence of the proposed algorithm is established under two mild assumptions. By implementing the algorithm to solve 75 benchmark test problems available in the literature, the obtained results indicate that the algorithm developed in this paper outperforms the existent similar state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

In 1937, the 16-years-old Hungarian mathematician Endre Weiszfeld, in a seminal paper, devised a method for solving the Fermat-Weber location problem-a problem whose origins can be traced back to the seventeenth century. Weiszfeld's method stirred up an enormous amount of research in the optimization and location communities, and is also being discussed and used till these days. In this paper, we review both the past and the ongoing research on Weiszfed's method. The existing results are presented in a self-contained and concise manner-some are derived by new and simplified techniques. We also establish two new results using modern tools of optimization. First, we establish a non-asymptotic sublinear rate of convergence of Weiszfeld's method, and second, using an exact smoothing technique, we present a modification of the method with a proven better rate of convergence. [ABSTRACT FROM AUTHOR]

Different mutation strategies and control parameters settings directly affect the performance of differential evolution (DE) algorithm. In this paper, a self-adaptive differential evolution algorithm with improved mutation strategy (IMSaDE) is proposed to improve optimization performance of DE. IMSaDE improves the “DE/rand/2” mutation strategy by incorporating elite archive strategy and control parameters adaptation strategy. Both strategies diversify the population and improve the convergence performance of the algorithm. IMSaDE was compared with eleven DE algorithms and six non-DE algorithms by using a set of 20 benchmark functions taken from the literature. Experimental results show that the overall performance of IMSaDE is better than the other competitors. In addition, the size of elite population has a significant impact on the performance of IMSaDE. [ABSTRACT FROM AUTHOR]

Plant Propagation Algorithms (PPA) are powerful and flexible solvers for optimisation problems. They are nature-inspired heuristics which can be applied to any optimisation/search problem. There is a growing body of research, mainly experimental, on PPA in the literature. Little, however, has been done on the theoretical front. Given the prominence this algorithm is gaining in terms of performance on benchmark problems as well as practical ones, some theoretical insight into its convergence is needed. The current paper is aimed at fulfilling this by providing a sketch for a global convergence analysis. [ABSTRACT FROM AUTHOR]

This paper gives a modified Hestenes and Stiefel (HS) conjugate gradient algorithm under the Yuan-Wei-Lu inexact line search technique for large-scale unconstrained optimization problems, where the proposed algorithm has the following properties: (1) the new search direction possesses not only a sufficient descent property but also a trust region feature; (2) the presented algorithm has global convergence for nonconvex functions; (3) the numerical experiment showed that the new algorithm is more effective than similar algorithms. [ABSTRACT FROM AUTHOR]

We consider the submodular function minimization (SFM) and the quadratic minimization problems regularized by the Lovász extension of the submodular function. These optimization problems are intimately related; for example, min-cut problems and total variation denoising problems, where the cut function is submodular and its Lovász extension is given by the associated total variation. When a quadratic loss is regularized by the total variation of a cut function, it thus becomes a total variation denoising problem and we use the same terminology in this paper for "general" submodular functions. We propose a new active-set algorithm for total variation denoising with the assumption of an oracle that solves the corresponding SFM problem. This can be seen as local descent algorithm over ordered partitions with explicit convergence guarantees. It is more flexible than the existing algorithms with the ability for warm-restarts using the solution of a closely related problem. Further, we also consider the case when a submodular function can be decomposed into the sum of two submodular functions F1 and F2 and assume SFM oracles for these two functions. We propose a new active-set algorithm for total variation denoising (and hence SFM by thresholding the solution at zero). This algorithm also performs local descent over ordered partitions and its ability to warm start considerably improves the performance of the algorithm. In the experiments, we compare the performance of the proposed algorithms with state-of-the-art algorithms, showing that it reduces the calls to SFM oracles. [ABSTRACT FROM AUTHOR]

This paper considers the problem of matrix completion when the observed entries are noisy and contain outliers. It begins with introducing a new optimization criterion for which the recovered matrix is defined as its solution. This criterion uses the celebrated Huber function from the robust statistics literature to downweigh the effects of outliers. A practical algorithm is developed to solve the optimization involved. This algorithm is fast, straightforward to implement, and monotonic convergent. Furthermore, the proposed methodology is theoretically shown to be stable in a well defined sense. Its promising empirical performance is demonstrated via a sequence of simulation experiments, including image inpainting. [ABSTRACT FROM AUTHOR]

This paper presents a primal method for finding the minimum L-infinity solution to under-determined linear systems of equations. The method is a two-phase method. Line search is performed at both phases. We establish a condition for a direction to be descent. The convergence proof of the method is shown. Expedient numerical schemes can be used whenever appropriate. Results are presented, which show the superiority of the method over some well-known methods. [ABSTRACT FROM AUTHOR]

Algorithms for computing local minima of smooth objective functions enjoy a mature theory as well as robust and effcient implementations. By comparison, the theory and practice of saddle search is destitute. In this paper, we present results for idealized versions of the dimer and gentlest ascent (GAD) saddle search algorithms that showcase the limitations of what is theoretically achievable within the current class of saddle search algorithms: (1) we present an improved estimate on the region of attraction of saddles, (2) we give explicit examples of potential energy wells from which GAD-type dynamics are unable to escape, and (3) we present a local analysis of \singular points" around which the dynamics gets trapped and prove the existence of quasi-periodic solutions. These results indicate that it is impossible to obtain globally convergent variants of dimer- and GAD-type algorithms. [ABSTRACT FROM AUTHOR]

In this paper, we propose an iteratively reweightedminimization algorithm (IRL1 algorithm) for solving the elasticminimization problem. We prove that any sequence generated by the IRL1 algorithm is bounded and asymptotically regular. We also prove that the sequence is convergent for any rationaland the limit is a stationary point of the elasticminimization problem. Moreover, under certain conditions, we present an error bound between the limit point of convergent sequence and the sparse solution of underdetermined linear system. Numerical experiments on sparse vector recovery are presented to demonstrate the effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

This paper presents Natural Evolution Strategies (NES), a recent family of black-box optimization algorithms that use the natural gradient to update a parameterized search distribution in the direction of higher expected fitness. We introduce a collection of techniques that address issues of convergence, robustness, sample complexity, computational complexity and sensitivity to hyperparameters. This paper explores a number of implementations of the NES family, such as general-purpose multi-variate normal distributions and separable distributions tailored towards search in high dimensional spaces. Experimental results show best published performance on various standard benchmarks, as well as competitive performance on others. [ABSTRACT FROM AUTHOR]

The problem of finding a vector with the fewest nonzero elements that satisfies an underdetermined system of linear equations is an NP-complete problem that is typically solved numerically via convex heuristics or nicely behaved nonconvex relaxations. In this paper we consider the elementary method of alternating projections (MAP) for solving the sparsity optimization problem without employing convex heuristics. In a parallel paper we recently introduced the restricted normal cone which generalizes the classical Mordukhovich normal cone and reconciles some fundamental gaps in the theory of sufficient conditions for local linear convergence of the MAP algorithm. We use the restricted normal cone together with the notion of superregularity, which is inherently satisfied for the affine sparse optimization problem, to obtain local linear convergence results with estimates for the radius of convergence of the MAP algorithm applied to sparsity optimization with an affine constraint. [ABSTRACT FROM AUTHOR]

Nemirovski and Yudin proposed the mirror descent algorithm at the late 1970s to solve convex optimization problems. This method is suitable to solve huge-scale optimization problems. In the paper, we describe a new version of the mirror descent method to solve variational inequalities with pseudomonotone operators. The method can be interpreted as a modification of Popov's two-step algorithm with the use of Bregman projections on the feasible set. We prove the convergence of the sequences generated by the proposed method. [ABSTRACT FROM AUTHOR]

Many-objective optimization refers to optimizing a multi-objective optimization problem (MOP) where the number of objectives is more than 3. Most classical evolutionary multi-objective optimization (EMO) algorithms use the Pareto dominance relation to guide the search, which usually perform poorly in the many-objective optimization scenario. This paper proposes an EMO algorithm based on information separation and neighbor punishment selection (ISNPS) to deal with many-objective optimization problems. ISNPS separates individual's behavior in the population into convergence information and distribution information by rotating the original coordinates in the objective space. Specifically, the proposed algorithm employs one coordinate to reflect individual's convergence and the remaining $$m-1$$ coordinates to reflect individual's distribution, where m is the number of objectives in a given MOP. In addition, a neighborhood punishment strategy is developed to prevent individuals from being crowded. From a series of experiments on 42 test instances with 3-10 objectives, ISNPS has been found to be very competitive against six representative algorithms in the EMO area. [ABSTRACT FROM AUTHOR]

In this paper, we describe a two-stage method for solving optimization problems with bound constraints. It combines the active-set estimate described in Facchinei and Lucidi (J Optim Theory Appl 85(2):265-289, 1995) with a modification of the non-monotone line search framework recently proposed in De Santis et al. (Comput Optim Appl 53(2):395-423, 2012). In the first stage, the algorithm exploits a property of the active-set estimate that ensures a significant reduction in the objective function when setting to the bounds all those variables estimated active. In the second stage, a truncated-Newton strategy is used in the subspace of the variables estimated non-active. In order to properly combine the two phases, a proximity check is included in the scheme. This new tool, together with the other theoretical features of the two stages, enables us to prove global convergence. Furthermore, under additional standard assumptions, we can show that the algorithm converges at a superlinear rate. Promising experimental results demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Most path-following algorithms for tracing a solution path of a parametric nonlinear optimization problem are only certifiably convergent under strong regularity assumptions about the problem functions. In particular, linear independence of the constraint gradients at the solutions is typically assumed, which implies unique multipliers. In this paper we propose a procedure designed to solve problems satisfying a weaker set of conditions, allowing for nonunique (but bounded) multipliers. Each iteration along the path consists of three parts: (1) a Newton corrector step for the primal and dual variables, which is obtained by solving a linear system ot equations, (2) a predictor step for the primal and dual variables, which is found as the solution of a quadratic programming problem, and (3) a jump step for the dual variables, which is found as t he solution of a linear programming problem. We present a convergence proof and demonstrate th e successful solution tracking of the algorithm numerically on a couple of illustrative examples. [ABSTRACT FROM AUTHOR]

Swarm intelligent algorithms are derived from the simulation of natural evolution or collective behavior of animals to seek solutions of complicated optimization problems by exploring and exploiting the search space efficiently and effectively. Through the analysis on the search characteristics of swarm intelligent algorithms, the concept of solution set diversity is introduced in this paper according to the infrastructural changes of solution sets in search processes. Two categories of fundamental search strategies, i.e. the diversification search and the intensification search, are then defined and their influences on the stagnation of solution sets evolution are investigated on the basis of the solution set diversity. It is proved in this paper that an intensification strategy inevitably leads candidates solution to a single solution, which is one of the main sources of stagnation; while a diversification strategy is able to reach any point of the coding space from each initial candidate solution, i.e., the whole searching space is the reachable region of the diversification strategy, but the convergence of the algorithm cannot be guaranteed. Three popular swarm intelligent algorithms, i.e. the canonical generic algorithm, the ant colony system and the discrete particle swarm optimization, are tested with a benchmark problem, and the results support the theoretical conclusions. [ABSTRACT FROM AUTHOR]

An algorithm is developed for projecting a point onto a polyhedron. The algorithm solves a dual version of the projection problem and then uses the relationship between the primal and dual to recover the projection. The techniques in the paper exploit sparsity. Sparse reconstruction by separable approximation (SpaRSA) is used to approximately identify active constraints in the polyhedron, and the dual active set algorithm (DASA) is used to compute a high precision solution. A linear convergence result is established for SpaRSA that does not require the strong concavity of the dual to the projection problem, and an earlier R-linear convergence rate is strengthened to a Q-linear convergence property. An algorithmic framework is developed for combining SpaRSA with an asymptotically preferred algorithm such as DASA. It is shown that only the preferred algorithm is executed asymptotically. Numerical results are given using the polyhedra associated with the Netlib LP test set. A comparison is made to the interior point method contained in the general purpose open source software package IPOPT for nonlinear optimization, and to the commercial package CPLEX, which contains an implementation of the barrier method that is targeted to problems with the structure of the polyhedral projection problem. [ABSTRACT FROM AUTHOR]

We introduce new optimized first-order methods for smooth unconstrained convex minimization. Drori and Teboulle (Math Program 145(1-2):451-482, 2014. doi:) recently described a numerical method for computing the N-iteration optimal step coefficients in a class of first-order algorithms that includes gradient methods, heavy-ball methods (Polyak in USSR Comput Math Math Phys 4(5):1-17, 1964. doi:), and Nesterov's fast gradient methods (Nesterov in Sov Math Dokl 27(2):372-376, 1983; Math Program 103(1):127-152, 2005. doi:). However, the numerical method in Drori and Teboulle (2014) is computationally expensive for large N, and the corresponding numerically optimized first-order algorithm in Drori and Teboulle (2014) requires impractical memory and computation for large-scale optimization problems. In this paper, we propose optimized first-order algorithms that achieve a convergence bound that is two times smaller than for Nesterov's fast gradient methods; our bound is found analytically and refines the numerical bound in Drori and Teboulle (2014). Furthermore, the proposed optimized first-order methods have efficient forms that are remarkably similar to Nesterov's fast gradient methods. [ABSTRACT FROM AUTHOR]

High number of design variables, large size of the search space, and control of a great number of design constraints are major preventive factors in performing optimum design of real-world structures in a reasonable time. The present study intends to examine the computational performance of cascading in design optimization of truss towers with large numbers of design variables. The cascade optimization procedure utilized in this paper reduces the objective function value over a number of optimization stages by initially operating on a small number of design variables, which is gradually increased stage after stage. In fact, the early stages of optimization in the cascade procedure make use of the coarsest configurations with small numbers of design variables and the later stages exploit finer configurations with larger numbers of design variables. The optimization algorithm utilized in each stage of a cascade process is enhanced colliding bodies optimization. High solution accuracy and convergence speed of the proposed method are shown through three test examples. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, we try to solve the semidefinite program with box constraints. Since the traditional projection method for constrained optimization with box constraints is not suitable to the semidefinite constraints, we present a new algorithm based on the feasible direction method. In the paper, we discuss two cases: the objective function in semidefinite programming is linear and nonlinear, respectively. We establish the convergence of our algorithm, and report the numerical experiments which show the effectiveness of the algorithm. [Copyright &y& Elsevier]

Abstract: This paper presents combination of differential evolution (DE) and biogeography-based optimization (BBO) algorithm to solve complex economic emission load dispatch (EELD) problems of thermal generators of power systems. Emission substances like NO X , SO X , COX, Power demand equality constraint and operating limit constraint are considered here. Differential evolution (DE) is one of the very fast and robust, accurate evolutionary algorithms for global optimization and solution of EELD problems. Biogeography-based optimization (BBO) is another new biogeography inspired algorithm. Biogeography deals with the geographical distribution of different biological species. This algorithm searches for the global optimum mainly through two steps: migration and mutation. In this paper combination of DE and BBO (DE/BBO) is proposed to accelerate the convergence speed of both the algorithm and to improve solution quality. To show the advantages of the proposed algorithm, it has been applied for solving multi-objective EELD problems in a 3-generator system with NO X and SO X emission, in a 6-generators system considering NO X emission, in a 6-generator system addressing both valve-point loading and NO X emission. The current proposal is found better in terms of quality of the compromising and individual solution obtained. [Copyright &y& Elsevier]

Abstract: In this paper, the problem of identifying the active constraints for constrained nonlinear programming and minimax problems at an isolated local solution is discussed. The correct identification of active constraints can improve the local convergence behavior of algorithms and considerably simplify algorithms for inequality constrained problems, so it is a useful adjunct to nonlinear optimization algorithms. Facchinei et al. [F. Facchinei, A. Fischer, C. Kanzow, On the accurate identification of active constraints, SIAM J. Optim. 9 (1998) 14–32] introduced an effective technique which can identify the active set in a neighborhood of a solution for nonlinear programming. In this paper, we first improve this conclusion to be more suitable for infeasible algorithms such as the strongly sub-feasible direction method and the penalty function method. Then, we present the identification technique of active constraints for constrained minimax problems without strict complementarity and linear independence. Some numerical results illustrating the identification technique are reported. [Copyright &y& Elsevier]

Abstract: This paper proposes a novel method for solving the Non-convex Economic Dispatch (NED) problems, by the Fuzzy Adaptive Modified Particle Swarm Optimization (FAMPSO). Practical ED problems have non-smooth cost functions with equality and inequality constraints when generator valve-point loading effects are taken into account. Modern heuristic optimization techniques have been given much attention by many researchers due to their ability to find an almost global optimal solution for ED problems. PSO is one of modern heuristic algorithms, in which particles change place to get close to the best position and find the global minimum point. However, the classic PSO may converge to a local optimum solution and the performance of the PSO highly depends on the internal parameters. To overcome these drawbacks, in this paper, a new mutation is proposed to improve the global searching capability and prevent the convergence to local minima. Also, a fuzzy system is used to tune its parameters such as inertia weight and learning factors. In order to evaluate the performance of the proposed algorithm, it is applied to a system consisting of 13 and 40 thermal units whose fuel cost function is calculated by taking account of the effect of valve-point loading. Simulation results demonstrate the superiority of the proposed algorithm compared to other optimization algorithms presented in literature. [Copyright &y& Elsevier]

This paper analyzes the performance of some global search algorithms on a number of space trajectory design problems. A rigorous testing procedure is introduced to measure the ability of an algorithm to identify the set of &epsilon-optimal solutions. From the analysis of the test results, a novel algorithm is derived. The development of the novel algorithm starts from the redefinition of some evolutionary heuristics in the form of a discrete dynamical system. The convergence properties of this discrete dynamical system are used to derive a hybrid evolutionary algorithm that displays very good performance on the particular class of problems presented in this paper. [ABSTRACT FROM AUTHOR]

This paper studies properties of a multi-swarm system based on a concept of physical quantum particles (mQSO). Quantum particles differ from the classic ones in the way they move. As opposed to the classic view of particle movement, where motion is controlled by linear kinematic laws, quantum particles change their location according to random distributions. The procedure for generating a new location for the quantum particle is similar to mutation operators widely used in evolutionary computation with real-valued representation. In this paper we study a set of new distributions of candidates for quantum particle location, and we show different features of these distributions. The distributions considered in this paper are divided into two classes: those with a limited range of the new location coordinates and those without such limitations. They are tested on different types of dynamic optimization problems. Experimental verification has been based on a number of testing environments and two main versions of the algorithm: with and without mechanisms protecting against stagnation caused by convergence of sub-swarms during the search process. The experimental results show the advantages of the distribution class, in which the candidates are spread out in the entire search space, and indicate the positive and negative aspects of application of anti-convergencemechanisms. [ABSTRACT FROM AUTHOR]

This paper presents a new nonmonotone quasi-Newton trust-region algorithm of the conic model for the solution of unconstrained optimization problems, where the computation of the horizontal vectors is easier and the approximate Hessian matrices can be maintained positive definite. Under reasonable assumptions, the global convergence, the locally linear and superlinear convergence of the proposed algorithm are developed, respectively. It is well known that in applying trust-region algorithms, the basic issue is how to solve the trust-region subproblem efficiently. To deal with the issue, an approximate solution method is developed in this paper. Note that the approximate solution method not only is computationally cheap, but also preserves the strong convergence properties as the exact solution methods. Numerical results are shown for a number of test problems from the literature. [ABSTRACT FROM AUTHOR]

This paper presents a subgradient-based neural network to solve a nonsmooth nonconvex optimization problem with a nonsmooth nonconvex objective function, a class of affine equality constraints, and a class of nonsmooth convex inequality constraints. The proposed neural network is modeled with a differential inclusion. Under a suitable assumption on the constraint set and a proper assumption on the objective function, it is proved that for a sufficiently large penalty parameter, there exists a unique global solution to the neural network and the trajectory of the network can reach the feasible region in finite time and stay there thereafter. It is proved that the trajectory of the neural network converges to the set which consists of the equilibrium points of the neural network, and coincides with the set which consists of the critical points of the objective function in the feasible region. A condition is given to ensure the convergence to the equilibrium point set in finite time. Moreover, under suitable assumptions, the coincidence between the solution to the differential inclusion and the "slow solution" of it is also proved. Furthermore, three typical examples are given to present the effectiveness of the theoretic results obtained in this paper and the good performance of the proposed neural network. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, we introduce and consider a new system of general variational inequalities involving four different operators. Using the projection operator technique, we suggest and analyze some new explicit iterative methods for this system of variational inequalities. We also study the convergence analysis of the new iterative method under certain mild conditions. Since this new system includes the system of variational inequalities involving three operators, variational inequalities and related optimization problems as special cases, results obtained in this paper continue to hold for these problems. Our results can be viewed as a refinement and improvement of the previously known results for variational inequalities. [Copyright &y& Elsevier]

Injection moulding optimisation seeks to achieve the highest possible moulding quality under the specified constraints. To this end, the factors (design variables) affecting the moulding quality should be adjusted, including those of process parameters, mould design, part geometry, etc. Past work in this aspect is primarily focused on tuning the process parameters and mould design (e.g., gate location, runner and cooling channel layout), with less attention on the part geometry, and none on them all. To address this problem, this paper presents a PSO (particle swarm optimisation) algorithm for the optimisation of multi-class design variables, such as the part thickness, process parameters (melt temperature, mould temperature, injection time) and gate location. The optimisation is targeted at different aspects of moulding quality, including part warpage, weld lines, air traps, and so on. In applying the PSO algorithm, the paper proposes a modified elite archiving method, which can expedite the convergence speed, hence improving the efficiency of the algorithm. A computer program was developed that automates the steps such as adjusting the part thickness, the injection moulding process parameters and the gate location, activating the CAE software to simulate the injection moulding process, retrieving the simulation results, and evaluating the objective functions. The whole procedure iterates a number of generations by following the search process of the algorithm. A case study was also presented to illustrate as well as to test the proposed methodology, which was demonstrated as both effective and efficient. [ABSTRACT FROM AUTHOR]

This paper considers localization of a source or a sensor from distance measurements. We argue that linear algorithms proposed for this purpose are susceptible to poor noise performance. Instead given a set of sensors/anchors of known positions and measured distances of the source/sensor to be localized from them we propose a potentially nonconvex weighted cost function whose global minimum estimates the location of the source/sensor one seeks. The contribution of this paper is to provide nontrivial ellipsoidal and polytopic regions surrounding these sensors/anchors of known positions, such that if the object to be localized is in this region, localization occurs by globally exponentially convergent gradient descent in the noise free case. Exponential convergence in the noise free case represents practical convergence as it ensures graceful performance degradation in the presence of noise. These results guide the deployment of sensors/anchors so that small subsets can be made responsible for practical localization in geographical areas determined by our approach. [ABSTRACT FROM AUTHOR]

Nonnegative matrix factorization (NMF) is useful to find basis information of nonnegative data. Currently, multiplicative updates are a simple and popular way to find the factorization. However, for the common NMF approach of minimizing the Euclidean distance between approximate and true values, no proof has shown that multiplicative updates converge to a stationary point of the NMF optimization problem. Stationarity is important as it is a necessary condition of a local minimum. This paper discusses the difficulty of proving the convergence. We propose slight modifications of existing updates and prove their convergence. Techniques invented in this paper may be applied to prove the convergence for other bound-constrained optimization problems. [ABSTRACT FROM AUTHOR]

This paper investigates the convergence theorems that are associated with a Discrete Hopfield Neural Network (DHNN) with delay. We present two updating rules, one for serial mode and the other for parallel mode. The speed of convergence of these proposed updating rules is faster than all of the existing updating rules. It has been proved in this paper that a DHNN with delay will converge to a stable state when operating in a serial mode if the matrix of weights of the no-delay term is symmetric. In addition, it has been proved that they will converge to a stable state when operating in a parallel mode if the matrix of weights of the no-delay term is a symmetric and non-negative definite matrix. The condition for convergence of a DHNN without delay can been relaxed from the need to have a symmetric matrix to an even weaker condition of having a quasi-symmetric matrix. The results in this paper extend both the existing results concerning the convergence of a DHNN without delay and our previous findings. By means of the new network structure and its convergence theorems, we propose a local searching algorithm for combinatorial optimization. We also relate the maximum value of a bivariate energy function to the stable states of a DHNN with delay, which generalizes Hopfield's energy function. Moreover, for the serial model we give the relationship between the convergence of the energy function and the convergence of the corresponding network. One application is presented to demonstrate the higher rate of convergence and the accuracy of the classification using our algorithm. [ABSTRACT FROM AUTHOR]

This paper focuses on a class of discrete-time interconnected pendulumlike systems. Sufficient conditions for the global convergence of systems with both structured and unstructured uncertainties in its linear part are established in terms of linear matrix inequalities (LMIs). In the traditional decentralized control of large scale systems, the effects of interconnections were seldom studied. In this paper, a square matrix specifying the interconnecting structure is introduced in order to discuss the effects of nonlinear interconnections. It is shown that the global convergence of the whole interconnected system can be achieved by designing an appropriate interconnection matrix. In order to solve the nonlinear matrix inequalities (NMIs) arising in the synthesis problem, a global optimization algorithm is presented which can be used to handle a class of NMI problems. An illustrative example is given to demonstrate the applicability and validity of the main results and the presented algorithm. [ABSTRACT FROM AUTHOR]

The paper discusses several versions of the method of shortest residuals, a specific variant of the conjugate gradient algorithm, first introduced by Lemaréchal and Wolfe and discussed by Hestenes in a quadratic case. In the paper we analyze the global convergence of the versions considered. Numerical comparison of these versions of the method of shortest residuals and an implementation of a standard Polak–Ribière conjugate gradient algorithm is also provided. It supports the claim that the method of shortest residuals is a viable technique, competitive to other conjugate gradient algorithms. [ABSTRACT FROM AUTHOR]

This paper proposes several globally convergent geometric optimization algorithms on Riemannian manifolds, which extend some existing geometric optimization techniques. Since any set of smooth constraints in the Euclidean space Rn (corresponding to constrained optimization) and the Rn space itself (corresponding to unconstrained optimization) are both special Riemannian manifolds, and since these algorithms are developed on general Riemannian manifolds, the techniques discussed in this paper provide a uniform framework for constrained and unconstrained optimization problems. Unlike some earlier works, the new algorithms have less restrictions in both convergence results and in practice. For example, global minimization in the one-dimensional search is not required. All the algorithms addressed in this paper are globally convergent. For some special Riemannian manifold other than Rn, the newalgorithms are very efficient. Convergence rates are obtained. Applications are discussed. [ABSTRACT FROM AUTHOR]

The problem under consideration is to obtain a measurement schedule for training neural networks. This task is perceived as an experimental design in a given design space that is obtained in such a way as to minimize the difference between the neural network and the system being considered. This difference can be expressed in many different ways and one of them, namely, the D-optimality criterion is used in this paper. In particular, the paper presents a unified and comprehensive treatment of this problem by discussing the existing and previously unpublished properties of the optimum experimental design (OED) for neural networks. The consequences of the above properties are discussed as well. A hybrid algorithm that can be used for both the training and data development of neural networks is another important contribution of this paper. A careful analysis of the algorithm is presented and its comprehensive convergence analysis with the help of the Lyapunov method are given. The paper contains a number of numerical examples that justify the application of the OED theory for neural networks. Moreover, an industrial application example is given that deals with the valve actuator. [ABSTRACT FROM AUTHOR]

Abstract: In this paper a new parameter-optimal high-order Iterative Learning Control (ILC) algorithms is proposed to extend the work of Owens and Feng [Parameter optimisation in iterative learning control. International Journal of Control 14(11), 1059–1069]. If the original plant is positive, this new algorithm will result in convergent learning where the convergence is monotonic to zero tracking error. If the original plant is not positive, it can be shown that by adding a suitable set of basis functions into the algorithm, the tracking error will again converge monotonically to zero. This provides a considerable improvement to earlier work on parameter-optimal ILC as it opens up the possibility of globally convergent algorithms for any linear plant G. The number of parameters needed to ensure convergence could, however, become large. The paper shows that the use of low-order parameterisations is capable of achieving much of the benefit achieved in the ‘ideal’ case. [Copyright &y& Elsevier]

In this paper, denoising of smooth ( H10-regular) images is considered. The purpose of the paper is basically twofold. First, to compare the denoising methods based on L1- and L2-fitting. Second, to analyze and realize an active-set method for solving the non-smooth optimization problem arising from the former approach. More precisely, we formulate the algorithm, proof its convergence, and give an efficient numerical realization. Several numerical experiments are presented, where the convergence of the proposed active-set algorithm is studied and the denoising properties of the methods based on L1- and L2-fitting are compared. Also a heuristic method for determining the regularization parameter is presented and tested. [ABSTRACT FROM AUTHOR]

This paper proposes a new algorithm for the unconstrained optimization problem, which is derived as an application of the control theory called Direct Gradient Descent Control. The static optimization problem is solved by using a dynamic controller in which the convergence speed can be greatly accelerated. The major idea is to consider the objective function F(x) and its time derivative dF(x(t))/dt as the evaluation criteria, and to apply the gradient descent method. It is verified by simulation that the proposed dynamic mathematical programming has a very great capability to converge to the optimal solution. It is also interesting to note that the method has to a certain extent the ability to find not only local optimal solutions, but also global optimal solutions. This paper improves the basic algorithm of dynamic mathematical programming so that it is suited to global optimization. The new algorithm generates movement toward the global optimal solution by utilizing two different trajectories with interaction (the chaotic trajectory and convergence trajectory). It is verified by simulation that the new algorithm functions as a global optimization procedure with a high success probability. © 2005 Wiley Periodicals, Inc. Electron Comm Jpn Pt 3, 88(6): 20–27, 2005; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ecjc.20094 [ABSTRACT FROM AUTHOR]