Search

This paper considers the constrained consensus problem over multi-cluster networks. It is assumed that the agents’ states are constrained by different sets, where each constraint set is privately known by the corresponding agent. Within this framework, a hierarchical projection-based consensus algorithm is presented to solve the considered problem. Technically, the consensus analysis of the proposed algorithm consists of the following three aspects: First, by using the property of the projection operator, the limiting behaviors of the agents’ states generated by the algorithm are investigated. Then, based on the limiting behaviors, it is proven that the agents’ states in the whole network achieve a constrained consensus. Furthermore, by introducing an important auxiliary variable that relates to the agents’ states, the linear convergence of the proposed algorithm is proved. Compared with the existing results, this paper generalizes the constrained consensus methods under single-cluster networks to the multi-cluster ones. Finally, simulations are given to verify the theoretical results. [ABSTRACT FROM AUTHOR]

Many-objective problems refer to the optimization problems containing more than three conflicting objectives. To obtain a representative set of well-distributed non-dominated solutions close to Pareto front in the objective space remains a challenging problem. Many papers have proposed different Multi-Objective Evolutionary Algorithms to solve the lack of the convergence and diversity in many-objective problems. One of the more promising approaches uses a set of reference points to discriminate the solutions and guide the search process. However, this approach was incorporated mainly in Multi-Objective Evolutionary Algorithms, and there are just some few promising adaptations of Particle Swarm Optimization approaches for effectively tackling many-objective problems regarding convergence and diversity. Thus, this paper proposes a practical and efficient Many-Objective Particle Swarm Optimization algorithm for solving many-objective problems. Our proposal uses a set of reference points dynamically determined according to the search process, allowing the algorithm to converge to the Pareto front, but maintaining the diversity of the Pareto front. Our experimental results demonstrate superior or similar performance when compared to other state-of-art algorithms. [ABSTRACT FROM AUTHOR]

Abstract This paper develops a novel algorithm to solve real-world constrained optimization problems, which hybridizes multiobjective optimization techniques with an ϵ-constrained method. First, a constrained optimization problem at hand is transformed into a bi-objective optimization problem. By the transformation, the advantage of multiobjective optimization techniques can be utilized in the constrained optimization area to balance population diversity and convergence. Meanwhile, the ϵ-constrained method is applied, which keeps the population evolving toward feasible region of the constrained optimization problem. In our proposed algorithm, the differential evolution is employed as a search engine to create offspring at each generation. Further, different combinations of mutation operators have been developed to improve the search ability and the population convergence at different stages. The performance of our approach is evaluated on 64 benchmark test functions from three popular test suits. Experimental results demonstrate that our proposed approach is capable of obtaining high-quality solutions on the majority of benchmark test functions, when compared with some other state-of-the-art constrained optimization algorithms. [ABSTRACT FROM AUTHOR]

“ Fuzzy Theoretic Approach to Signals and Systems ” assumes all system variables and parameters as uncertain (i.e. being characterized by membership functions), develops a mathematical theory for analytically determining the membership functions on system variables and parameters, derives algorithms for estimating the parameters of membership functions, and establishes robustness and convergence properties of the estimation algorithms. The membership functions are analytically determined via solving a variational optimization problem that maximizes the “ over-uncertainties-averaged-log-membership ” of the observed data around an initial guess. This paper develops the analytical fuzzy theory for the particular case of a multi-input single-output static system affected by noises. The theory facilitates designing an adaptive filtering algorithm. The robustness of the adaptive filtering algorithm is proved theoretically via a mathematical analysis. Numerical experiments further demonstrate the robustness of the filtering algorithm. A comparison of the algorithm with the state-of-art methods is made using the practical biomedical applications related to the modeling and analysis of heart rate signals for assessing the physiological state of an individual. [ABSTRACT FROM AUTHOR]

Various real-world applications can be formulated as feature selection problems, which have been known to be NP-hard. In this paper, we propose an effective feature selection method based on firefly algorithm (FFA), called return-cost-based binary FFA (Rc-BBFA). The proposed method has the capability of preventing premature convergence and is particularly efficient attributed to the following three aspects. An indicator based on the return-cost is first defined to measure a firefly’s attractiveness from other fireflies. Then, a Pareto dominance-based strategy is presented to seek the attractive one for each firefly. Finally, a binary movement operator based on the return-cost attractiveness and the adaptive jump is developed to update the position of a firefly. The experimental results on a series of public datasets show that the proposed method is competitive in comparison with other feature selection algorithms, including the traditional algorithms, the GA-based algorithm, the PSO-based algorithm, and the FFA-based algorithms. [ABSTRACT FROM AUTHOR]

Abstract: Most real world multi-objective problems (MOPs) have a complicated solution space. Facing such problems, a direction vectors based co-evolutionary multi-objective optimization algorithm (DVCMOA) that introduces the decomposition idea from MOEA/D to co-evolutionary algorithms is proposed in this paper. It is novel in the sense that DVCMOA applies the concept of direction vectors to co-evolutionary algorithms. DVCMOA first divides the entire population into several subpopulations on the basis of the initial direction vectors in the objective space. Then, it solves MOPs through the co-evolutionary interaction among the subpopulations in which individuals are classified according to their direction vectors. Finally, it explores the less developed regions to maintain the relatively uniform distribution of the solution space. In this way, DVCMOA has advantages in convergence, diversity and uniform distribution of the non-dominated solution set, which are explained through comparison with other state-of-the-art multi-objective optimization evolutionary algorithms (MOEAs) in this paper. DVCMOA is shown to be effective on 6 multi-objective 0–1 knapsack problems. [Copyright &y& Elsevier]

Abstract: The research on optimal design of infinite-impulse response (IIR) filters based on optimization techniques has gained much attention in recent years. However, due to the limited performance of the applied optimization techniques, the orders of the filters, which can be obtained, are very low in the previous research. Memetic algorithms (MAs) are widely recognized to have better convergence capability than their conventional counterparts. However, the universality of the MAs, e.g. the ability of solving diverse kinds of digital IIR filter designs, is still limited. In this paper, we design a Two-Stage ensemble Memetic Algorithm (TSMA) framework to more appropriately synthesize the strengths of the evolutionary global search and local search techniques. In the first optimization stage, a competition is held among the candidate local search techniques. Its major idea is to choose the best local search technique and to obtain good initial state. Inheriting the good information of the first stage, the second optimization stage is to implement effective adaptive MA to pursue high-quality solution. The experimental studies presented in this paper contain three aspects: (1) the benefits of the TSMA framework are experimentally investigated by comparing TSMA with its sub-optimizers and recent effective evolutionary algorithms (EAs) on 26 test functions; then (2) TSMA is compared with 4 MAs on the CEC05 functions to comprehensively show the advantages of TSMA; and (3) the TSMA and 6 state-of-the-art algorithms are applied to design high-order digital infinite-impulse response (IIR) filters. The experimental results definitely demonstrate the excellent effectiveness, efficiency and reliability of TSMA on both function optimization and digital IIR filter design tasks. [Copyright &y& Elsevier]

Dissimilarity measures play a key role in exploring the inherent cluster structure of the data for any partitional clustering algorithm. Commonly used dissimilarity functions for clustering purpose are so far confined to the Euclidean, exponential and Mahalanobish distances. In this article we develop generalized algorithms to solve the partitional clustering problems formulated with a general class of Inner Product Induced Norm (IPIN) based dissimilarity measures. We provide an in-depth mathematical analysis of the underlying optimization framework and analytically address the issue of existence of a solution and its uniqueness. In absence of a closed form solution, we develop a fast stochastic gradient descent algorithm and the Minimization by Incremental Surrogate Optimization (MISO) algorithm (in case of constrained optimization) with exponential convergence rate to obtain the solution. We carry out a convergence analysis of the fuzzy and k -means clustering algorithms with the IPIN based dissimilarity measures and also establish how these algorithms guarantee convergence to a stationary point. In addition, we investigate the nature of the stationary point. Novelty of the paper lies in the introduction of a generalized class of divergence measures, development of fuzzy and k -means clustering algorithms with the general class of divergence measures and a thorough convergence analysis of the developed algorithms. [ABSTRACT FROM AUTHOR]

Abstract: Motivated by the recent success of diverse approaches based on differential evolution (DE) to solve constrained numerical optimization problems, in this paper, the performance of this novel evolutionary algorithm is evaluated. Three experiments are designed to study the behavior of different DE variants on a set of benchmark problems by using different performance measures proposed in the specialized literature. The first experiment analyzes the behavior of four DE variants in 24 test functions considering dimensionality and the type of constraints of the problem. The second experiment presents a more in-depth analysis on two DE variants by varying two parameters (the scale factor F and the population size NP), which control the convergence of the algorithm. From the results obtained, a simple but competitive combination of two DE variants is proposed and compared against state-of-the-art DE-based algorithms for constrained optimization in the third experiment. The study in this paper shows (1) important information about the behavior of DE in constrained search spaces and (2) the role of this knowledge in the correct combination of variants, based on their capabilities, to generate simple but competitive approaches. [ABSTRACT FROM AUTHOR]

Structure learning is a very important problem in the field of Bayesian networks (BNs). It is also an active research area for more than 2 decades; therefore, many approaches have been proposed in order to find an optimal structure based on training samples. In this paper, a Particle Swarm Optimization (PSO)-based algorithm is proposed to solve the BN structure learning problem; named BNC-PSO ( B ayesian N etwork C onstruction algorithm using PSO ). Edge inserting/deleting is employed in the algorithm to make the particles have the ability to achieve the optimal solution, while a cycle removing procedure is used to prevent the generation of invalid solutions. Then, the theorem of Markov chain is used to prove the global convergence of our proposed algorithm. Finally, some experiments are designed to evaluate the performance of the proposed PSO-based algorithm. Experimental results indicate that BNC-PSO is worthy of being studied in the field of BNs construction. Meanwhile, it can significantly increase nearly 15% in the scoring metric values, comparing with other optimization-based algorithms. [ABSTRACT FROM AUTHOR]

In this paper, a dynamic mentoring scheme along with a self-regulation scheme have been incorporated in the standard Particle Swarm Optimization (PSO) algorithm to empower the searching particles with human-like characteristics. The algorithm is referred to as a Dynamic Mentoring and Self-Regulation based Particle Swarm Optimization (DMeSR-PSO) algorithm. Based on their experiences, the particles are divided into three groups, viz., the mentor group, the mentee group and the independent learner group where the number of particles in each group is dynamically changing in every iteration. In human learning psychology, mentoring is regarded as a powerful and effective learning process and independent learners are the ones who do not need mentoring and are capable of performing self-regulation of their own knowledge. Therefore, the particles in each of the above three groups have different learning strategies for their velocity updates where the mentors are equipped with a strong self-belief based search, the mentees are taking guidance from the mentors and the independent learners employ self-perception strategy. The DMeSR-PSO algorithm has been extensively evaluated using the simple unimodal and multimodal benchmark functions from CEC2005, more complex shifted and rotated benchmark functions fromCEC2013 and also based on eight real- world problems from CEC2011. The results have been compared with six state-of-the-art PSO variants and five meta-heuristic algorithms for the CEC2005 problems. Further, a comparative analysis on CEC2013 benchmark functions with different PSO variants has also been presented. Finally, DMeSR-PSO's performance on the real-world problems is compared with the top two algorithms from the CEC2011 competition. The results indicate that the proposed learning strategies help DMeSR-PSO to achieve faster convergence and provide better solutions in most of the problems with a 95% confidence level, yielding an effective optimization algorithm for real-world applications. [ABSTRACT FROM AUTHOR]

Abstract: This paper is concerned with a finite-time particle swarm optimization algorithm for odor source localization. First, a continuous-time finite-time particle swarm optimization (FPSO) algorithm is developed based on the continuous-time model of the particle swarm optimization (PSO) algorithm. Since the introduction of a nonlinear damping item, the proposed continuous-time FPSO algorithm can converge over a finite-time interval. Furthermore, in order to enhance its exploration capability, a tuning parameter is introduced into the proposed continuous-time FPSO algorithm. The algorithm’s finite-time convergence is analyzed by using the Lyapunov approach. Second, the discrete-time FPSO algorithm is obtained by using a given dicretization scheme. The corresponding convergence condition is derived by using a linear matrix inequality (LMI) approach. Finally, the features and performance of the proposed FPSO algorithm are illustrated by using two ill-posed functions and twenty-five benchmark functions, respectively. In numerical simulation results, the problem of odor source localization is presented to validate the effectiveness of the proposed FPSO algorithm. [Copyright &y& Elsevier]

Abstract: Recently, Gandomi and Alavi proposed a meta-heuristic optimization algorithm, called Krill Herd (KH). This paper introduces the chaos theory into the KH optimization process with the aim of accelerating its global convergence speed. Various chaotic maps are considered in the proposed chaotic KH (CKH) method to adjust the three main movements of the krill in the optimization process. Several test problems are utilized to evaluate the performance of CKH. The results show that the performance of CKH, with an appropriate chaotic map, is better than or comparable with the KH and other robust optimization approaches. [Copyright &y& Elsevier]

Abstract: The fast global k-means (FGKM) clustering algorithm is one of the most effective approaches for resolving the local convergence of the k-means clustering algorithm. Numerical experiments show that it can effectively determine a global or near global minimizer of the cost function. However, the FGKM algorithm needs a large amount of computational time or storage space when handling large data sets. To overcome this deficiency, a more efficient FGKM algorithm, namely FGKM+A, is developed in this paper. In the development, we first apply local geometrical information to describe approximately the set of objects represented by a candidate cluster center. On the basis of the approximate description, we then propose an acceleration mechanism for the production of new cluster centers. As a result of the acceleration, the FGKM+A algorithm not only yields the same clustering results as that of the FGKM algorithm but also requires less computational time and fewer distance calculations than the FGKM algorithm and its existing modifications. The efficiency of the FGKM+A algorithm is further confirmed by experimental studies on several UCI data sets. [Copyright &y& Elsevier]

Abstract: Load balancers distribute workload across multiple nodes based on a variation of the round robin algorithm, or a more complex algorithm that optimizes a specified objective or allows for horizontal scalability and higher availability. In this paper, we investigate whether robust load balancing can be achieved using a local co-operative mechanism between the resources (nodes). The local aspect of the mechanism implies that each node interacts with a small subset of the nodes that define its neighborhood. The co-operative aspect of the mechanism implies that a node may offload some of load to its neighbor nodes that have lesser load or accept jobs from neighbor nodes that have higher load. Each node is thus only aware of the state of its neighboring nodes and there is no central entity that has the knowledge of the state of all the nodes. We model the overall mechanism of load balancing based on local interactions as a congestion game and show that convergence to the Nash equilibrium is possible using only local interactions. We derive worst case bounds on the number of transfers (time) required to achieve global load balancing under this setup. We also include simulation results to demonstrate emergent global load balancing based only on local interactions and local information. [Copyright &y& Elsevier]

Abstract: Classification is one of the key issues in the fields of decision sciences and knowledge discovery. In this paper, we present a new classification method based on gravitational potential energy between particles. The basic principle of gravitation based classification (GBC) algorithm is to find the equilibrium condition of the classifier, which is modeled as a classifier line between two groups of fixed particles. In the proposed approach, the input data is a collection of masses, which interact with each other based on Newton’s universal law of gravitation and the laws of motion. We present a convex formulation for this problem that always converges to a global optimum solution. The proposed method has been compared with some well-known classification approaches, and the results confirm the high performance of the proposed method. [Copyright &y& Elsevier]

Abstract: This paper presents a novel Constrained Optimization based on Modified Differential Evolution algorithm (COMDE). In the new algorithm, a new directed mutation rule, based on the weighted difference vector between the best and the worst individuals at a particular generation, is introduced. The new directed mutation rule is combined with the modified basic mutation strategy DE/rand/1/bin, where only one of the two mutation rules is applied with the probability of 0.5. The proposed mutation rule is shown to enhance the local search ability of the basic Differential Evolution (DE) and to get a better trade-off between convergence rate and robustness. Two new scaling factors are introduced as uniform random variables to improve the diversity of the population and to bias the search direction. Additionally, a dynamic non-linear increased crossover probability is utilized to balance the global exploration and local exploitation. COMDE also includes a modified constraint handling technique based on feasibility and the sum of constraints violations. A new dynamic tolerance technique to handle equality constraints is also adopted. The effectiveness and benefits of the new directed mutation strategy and modified basic strategy used in COMDE has been experimentally investigated. The effect of the parameters of the crossover probability function and the parameters of the dynamic tolerance equation on the performance of COMDE have been analyzed and evaluated by different experiments. Numerical experiments on 13 well-known benchmark test functions and five engineering design problems have shown that the new approach is efficient, effective and robust. The comparison results between the COMDE and the other 28 state-of-the-art evolutionary algorithms indicate that the proposed COMDE algorithm is competitive with, and in some cases superior to, other existing algorithms in terms of the quality, efficiency, convergence rate, and robustness of the final solution. [Copyright &y& Elsevier]

Abstract: Motivated by concepts in quantum mechanics and particle swarm optimization (PSO), quantum-behaved particle swarm optimization (QPSO) was proposed as a variant of PSO with better global search ability. Although it has been shown to perform well in finding optimal solutions for many optimization problems, there has so far been little theoretical analysis on its convergence and performance. This paper presents a convergence analysis and performance evaluation of the QPSO algorithm and it also proposes two variants of the QPSO algorithm. First, we investigate in detail the convergence of the QPSO algorithm on a probabilistic metric space and prove that the QPSO algorithm is a form of contraction mapping and can converge to the global optimum. This is the first time that the theory of probabilistic metric spaces has been employed to analyze a stochastic optimization algorithm. We provided a new definition for the convergence rate of a stochastic algorithm as well as definitions for three types of convergence according to the correlations between the convergence rate and the objective function values. With these definitions, the effectiveness of the QPSO is evaluated by computing and analyzing the time complexity and the convergence rate of the algorithm. Then, the QPSO with random mean best position (QPSO-RM) and the QPSO with ranking operator (QPSO-RO) are proposed as two improvements of the QPSO algorithm. Finally, some empirical studies on popular benchmark functions are performed in order to make a full performance evaluation and comparison between QPSO, QPSO-RM, QPSO-RO and other variants of PSO. [Copyright &y& Elsevier]

Abstract: In this paper, a hybrid of algorithms for electromagnetism-like mechanisms (EM) and particle swarm optimisation (PSO), called HEMPSO, is proposed for use in designing a functional-link-based Petri recurrent fuzzy neural system (FLPRFNS) for nonlinear system control. The FLPRFNS has a functional link-based orthogonal basis function fuzzy consequent and a Petri layer to eliminate the redundant fuzzy rule for each input calculation. In addition, the FLPRFNS is trained by the proposed hybrid algorithm. The main innovation is that the random-neighbourhood local search is replaced by a PSO algorithm with an instant-update strategy for particle information. Each particle updates its information instantaneously and in this way receives the best current information. Thus, HEMPSO combines the advantages of multiple-agent-based searching, global optimisation, and rapid convergence. Simulation results confirm that HEMPSO can be used to perform global optimisation and offers the advantage of rapid convergence; they also indicate that the FLPRFNS exhibits high accuracy. [Copyright &y& Elsevier]

Abstract: In this paper, a controllable probabilistic particle swarm optimization (CPPSO) algorithm is introduced based on Bernoulli stochastic variables and a competitive penalized method. The CPPSO algorithm is proposed to solve optimization problems and is then applied to design the memoryless feedback controller, which is used in the synchronization of an array of delayed neural networks (DNNs). The learning strategies occur in a random way governed by Bernoulli stochastic variables. The expectations of Bernoulli stochastic variables are automatically updated by the search environment. The proposed method not only keeps the diversity of the swarm, but also maintains the rapid convergence of the CPPSO algorithm according to the competitive penalized mechanism. In addition, the convergence rate is improved because the inertia weight of each particle is automatically computed according to the feedback of fitness value. The efficiency of the proposed CPPSO algorithm is demonstrated by comparing it with some well-known PSO algorithms on benchmark test functions with and without rotations. In the end, the proposed CPPSO algorithm is used to design the controller for the synchronization of an array of continuous-time delayed neural networks. [Copyright &y& Elsevier]

Abstract: In evolutionary multi-objective optimization (EMO), the convergence to the Pareto set of a multi-objective optimization problem (MOP) and the diversity of the final approximation of the Pareto front are two important issues. In the existing definitions and analyses of convergence in multi-objective evolutionary algorithms (MOEAs), convergence with probability is easily obtained because diversity is not considered. However, diversity cannot be guaranteed. By combining the convergence with diversity, this paper presents a new definition for the finite representation of a Pareto set, the B-Pareto set, and a convergence metric for MOEAs. Based on a new archive-updating strategy, the convergence of one such MOEA to the B-Pareto sets of MOPs is proved. Numerical results show that the obtained B-Pareto front is uniformly distributed along the Pareto front when, according to the new definition of convergence, the algorithm is convergent. [Copyright &y& Elsevier]

Abstract: In this paper, we introduce a mathematical model for analyzing the dynamics of the univariate marginal distribution algorithm (UMDA) for a class of parametric functions with isolated global optima. We prove a number of results that are used to model the evolution of UMDA probability distributions for this class of functions. We show that a theoretical analysis can assess the effect of the function parameters on the convergence and rate of convergence of UMDA. We also introduce for the first time a long string limit analysis of UMDA. Finally, we relate the results to ongoing research on the application of the estimation of distribution algorithms for problems with unitation constraints. [Copyright &y& Elsevier]

Abstract: Several variants of the particle swarm optimization (PSO) algorithm have been proposed in recent past to tackle the multi-objective optimization (MO) problems based on the concept of Pareto optimality. Although a plethora of significant research articles have so far been published on analysis of the stability and convergence properties of PSO as a single-objective optimizer, till date, to the best of our knowledge, no such analysis exists for the multi-objective PSO (MOPSO) algorithms. This paper presents a first, simple analysis of the general Pareto-based MOPSO and finds conditions on its most important control parameters (the inertia factor and acceleration coefficients) that govern the convergence behavior of the algorithm to the optimal Pareto front in the objective function space. Computer simulations over benchmark MO problems have also been provided to substantiate the theoretical derivations. [Copyright &y& Elsevier]

Abstract: In the last 20years, while most research on fuzzy approximation theory has focused on nonadaptive fuzzy systems, little work has been done on adaptive fuzzy systems. This paper introduces an algorithm for adaptive fuzzy systems with Variable Universe of Discourse (VUD). By means of contraction–expansion factors, universe of discourse can be modified online, and fuzzy rules can reproduce automatically to adapt to the modified universe of discourse. Thus, dependence on the size of initial rule base is greatly reduced. Using Stone–Weierstrass theorem, VUD adaptive fuzzy systems are proved to be universal approximators with two-order approximation accuracy. In addition, the convergence properties of approximation error are discussed, and a sufficient condition is presented to partition universe of discourse and to calculate the size of rule base. An example is also given to illustrate the approximation power of VUD adaptive fuzzy systems. [Copyright &y& Elsevier]

Abstract: Neuro-fuzzy approach is known to provide an adaptive method to generate or tune fuzzy rules for fuzzy systems. In this paper, a modified gradient-based neuro-fuzzy learning algorithm is proposed for zero-order Takagi–Sugeno inference systems. This modified algorithm, compared with conventional gradient-based neuro-fuzzy learning algorithm, reduces the cost of calculating the gradient of the error function and improves the learning efficiency. Some weak and strong convergence results for this algorithm are proved, indicating that the gradient of the error function goes to zero and the fuzzy parameter sequence goes to a fixed value, respectively. A constant learning rate is used. Some conditions for the constant learning rate to guarantee the convergence are specified. Numerical examples are provided to support the theoretical findings. [Copyright &y& Elsevier]