Search

Abstract: The Prediction Error Method (PEM) is related to an optimization problem built on input/output data collected from the system to be identified. It is often hard to find the global solution of this optimization problem because the corresponding objective function presents local minima and/or the search space is constrained to a nonconvex set. The shape of the cost function, and hence the difficulty in solving the optimization problem, depends directly on the experimental conditions, more specifically on the spectrum of the input/output data collected from the system. Therefore, it seems plausible to improve the convergence to the global minimum by properly choosing the spectrum of the input; in this paper, we address this problem. We present a condition for convergence to the global minimum of the cost function and propose its inclusion in the input design. We present the application of the proposed approach to case studies where the algorithms tend to get trapped in nonglobal minima. [Copyright &y& Elsevier]

Abstract: In this paper, we proposed a strategy to find new local minima based on mutated damping factors and converged damping factor (conventional damping factor). In this proposed strategy, the converged damping factor is used to make the algorithm converge to a certain local minimum, and the mutated damping factors are used to make the trapped solution jump out of present convergence area. We study the behavior of the two kinds of factors in lens system optimization. The proposed strategy is successfully applied in lens system optimization. The result shows that the proposed strategy is reliable to make a further improvement especially when the solution is approaching target design. [Copyright &y& Elsevier]

Abstract: In this paper, we analyze a novel algorithm for 2-D ARMA model parameter estimation in the presence of noise and then develop a fast and efficient blind image restoration algorithm. We show that the novel algorithm can minimize a quadratic convex optimization problem and has a lower computational complexity than the conventional algorithms. As a result, the novel algorithm involves no convergence and local minimum issue. Moreover, the proposed blind image restoration algorithm can overcome the local minimization problem. Computed results confirm that the novel algorithm can more quickly obtain more accurate estimates than the conventional algorithms in the presence of noise. [Copyright &y& Elsevier]

In this paper, a new nonmonotone inexact line search rule is proposed and applied to the trust region method for unconstrained optimization problems. In our line search rule, the current nonmonotone term is a convex combination of the previous nonmonotone term and the current objective function value, instead of the current objective function value . We can obtain a larger stepsize in each line search procedure and possess nonmonotonicity when incorporating the nonmonotone term into the trust region method. Unlike the traditional trust region method, the algorithm avoids resolving the subproblem if a trial step is not accepted. Under suitable conditions, global convergence is established. Numerical results show that the new method is effective for solving unconstrained optimization problems. [ABSTRACT FROM AUTHOR]

Abstract: Multi-frame blind restoration algorithms often have disadvantages of slow convergence and sensitiveness to complex noises. In this paper, we proposed a new multi-frame method based on PSF estimate and TV regularization. Our algorithm consists of first simplifying the multi-frame MAP model through the identification of blur parameters for each frame image and then adding various penalty terms to speed up the convergence in deconvolution process obviously. Finally, an adaptive L2/L1 norm selection scheme is built to deal with various noise distributions. A series of experiment results show that the proposed method can outperform other previous work robustly and efficiently even under large noise and image blurring. [Copyright &y& Elsevier]

This paper first proposes a trust region algorithm to obtain a stationary point of unconstrained multiobjective optimisation problem. Under suitable assumptions, the global convergence of the new algorithm is established. We then extend the trust region method to solve the non-smooth multiobjective optimisation problem. [ABSTRACT FROM PUBLISHER]

In this paper, we address the problem of convergence to Nash equilibria in games with rewards that are initially unknown and must be estimated over time from noisy observations. These games arise in many real-world applications, whenever rewards for actions cannot be prespecified and must be learned online, but standard results in game theory do not consider such settings. For this problem, we derive a multiagent version of Q-learning to estimate the reward functions using novel forms of the-greedy learning policy. Using these Q-learning schemes to estimate reward functions, we then provide conditions guaranteeing the convergence of adaptive play and the better-reply processes to Nash equilibria in potential games and games with more general forms of acyclicity, and of regret matching to the set of correlated equilibria in generic games. A secondary result is that we prove the strong ergoditicity of stochastic adaptive play and stochastic better-reply processes in the case of vanishing perturbations. Finally, we illustrate the efficacy of the algorithms in a set of randomly generated three-player coordination games and show the practical necessity of our results by demonstrating that violations to the derived learning parameter conditions can cause the algorithms to fail to converge. [ABSTRACT FROM AUTHOR]

Abstract: DIRECT is a sample-based global optimisation method for Lipschitz continuous functions defined over compact multidimensional domains. This paper adapts the DIRECT method with a modified termination criterion for global extremum seeking control of multivariable dynamical plants. Finite-time semi-global practical convergence is established based on a periodic sampled-data control law, whose sampling period is a parameter which determines the region and accuracy of convergence. A crucial part of the development is dedicated to a robustness analysis of the DIRECT method against bounded additive perturbations on the objective function. Extremum seeking involving multiple units is also considered within the same context as a means to increase the speed of convergence. Numerical examples of global extremum seeking based on DIRECT are presented at the end. [Copyright &y& Elsevier]

In this paper, an improved spectral conjugate gradient algorithm is developed for solving nonconvex unconstrained optimization problems. Different from the existent methods, the spectral and conjugate parameters are chosen such that the obtained search direction is always sufficiently descent as well as being close to the quasi-Newton direction. With these suitable choices, the additional assumption in the method proposed by Andrei on the boundedness of the spectral parameter is removed. Under some mild conditions, global convergence is established. Numerical experiments are employed to demonstrate the efficiency of the algorithm for solving large-scale benchmark test problems, particularly in comparison with the existent state-of-the-art algorithms available in the literature. [ABSTRACT FROM AUTHOR]

Linear Discriminant Analysis (LDA) is one of the most efficient statistical approaches for feature extraction and dimension reduction. The generalized Foley-Sammon transform and the trace ratio model are very important in LDA and have received increasing interest. An efficient iterative method has been proposed for the resulting trace ratio optimization problem, which, under a mild assumption, is proved to enjoy both the local quadratic convergence and the global convergence to the global optimal solution (Zhang, L.-H., Liao, L.-Z., Ng, M.K.: SIAM J. Matrix Anal. Appl. 31:1584, ). The present paper further investigates the convergence behavior of this iterative method under no assumption. In particular, we prove that the iteration converges superlinearly when the mild assumption is removed. All possible limit points are characterized as a special subset of the global optimal solutions. An illustrative numerical example is also presented. [ABSTRACT FROM AUTHOR]

The perimeter functional is known to oppose serious difficulties when it has to be handled within a topology optimization procedure. In this paper, a regularized perimeter functional Perε, defined for two- and three-dimensional domains, is introduced. On one hand, the convergence of Perε to the exact perimeter when ε tends to zero is proved. On the other hand, the topological differentiability of Perε for ε > 0 is analyzed. These features lead to the design of a topology optimization algorithm suitable for perimeter-dependent objective functionals. Several numerical results illustrate the method. [ABSTRACT FROM AUTHOR]

Design optimization problems are often formulated as an optimization problem whose objective is a function of the output of a large-scale parametric linear system, obtained from the discretization of a PDE. To reduce the high computational cost of the objective and its gradient, model order reduction techniques can be used. This paper uses interpolatory reduced models as surrogate models in an optimization procedure. We replace the standard first-order condition by the relaxed first-order condition, which is more suitable when algebraic reduced models are used as surrogate models. The relaxed first-order condition imposes that the approximation quality of the surrogate model at the interpolation point can be measured and refined and that the surrogate model is equipped with an error bound on the entire parameter space. We propose two optimization algorithms: one uses the error bound to define a trust region and the other penalizes the objective with the error bound. We prove convergence of both methods under mild conditions in a unified framework. These methods are efficient when surrogate models are cheap to build and evaluate, since they only need these operations to achieve convergence. Numerical experiments from civil engineering show good performance of the proposed methods. [ABSTRACT FROM AUTHOR]

In this paper, by using a forcing function and the reverse modulus of continuity of gradient, we propose a generalization and development of the nonmonotone line search method for unconstrained optimization problems. The global convergence property of the new method is established under some reasonable conditions that are weaker than those of the existing nonmonotone line search methods. Furthermore, the proof of convergence is simpler than other methods. [ABSTRACT FROM PUBLISHER]

A generalized gradient projection filter algorithm for inequality constrained optimization is presented. It has three merits. The first is that the amount of computation is lower, since the gradient matrix only needs to be computed one time at each iterate. The second is that the paper uses the filter technique instead of any penalty function for constrained programming. The third is that the algorithm is of global convergence and locally superlinear convergence under some mild conditions. [ABSTRACT FROM AUTHOR]

Working set selection is a major step in decomposition methods for training least squares support vector machines (LS-SVMs). In this paper, a new technique for the selection of working set in sequential minimal optimization- (SMO-) type decomposition methods is proposed. By the new method, we can select a single direction to achieve the convergence of the optimality condition. A simple asymptotic convergence proof for the new algorithm is given. Experimental comparisons demonstrate that the classification accuracy of the new method is not largely different from the existing methods, but the training speed is faster than existing ones. [ABSTRACT FROM AUTHOR]

In this paper, an improved feasible QP-free method is proposed to solve nonlinear inequality constrained optimization problems. Here, a new modified method is presented to obtain the revised feasible descent direction. In view of the computational cost, the most attractive feature of the new algorithm is that only one system of linear equations is required to obtain the revised feasible descent direction. Thereby, per single iteration, it is only necessary to solve three systems of linear equations with the same coefficient matrix. In particular, without the positive definiteness assumption on the Hessian estimate, the proposed algorithm is still global convergence. Under some suitable conditions, the superlinear convergence rate is obtained. [ABSTRACT FROM AUTHOR]

In this paper an algorithm for the numerical solution of nonlinear equations systems is presented. For this purpose the system of equations becomes a function of real value which will be minimized, in the global sense, in a given initial domain (a box in n) using analysis of intervals. The designed algorithm has the ability to determine the existence or not of solutions to the system of equations in a given box. The solutions of the system of equations, if they exist inside the given box, are expressed by means of enclosures by sub-boxes whose size is smaller than the established accuracy. There is not restriction about the relationship between the number of equations and the number of unknowns of the system. It is also carried out an analysis of the convergence of the algorithm, and the results of their application are shown for some test problems. [ABSTRACT FROM AUTHOR]

This paper describes an implementation of an interior-point algorithm for large-scale nonlinear optimization. It is based on the algorithm proposed by Curtis et al. (SIAM J Sci Comput 32:3447-3475, ), a method that possesses global convergence guarantees to first-order stationary points with the novel feature that inexact search direction calculations are allowed in order to save computational expense. The implementation follows the proposed algorithm, but includes many practical enhancements, such as functionality to avoid the computation of a normal step during every iteration. The implementation is included in the IPOPT software package paired with an iterative linear system solver and preconditioner provided in PARDISO. Numerical results on a large nonlinear optimization test set and two PDE-constrained optimization problems with control and state constraints are presented to illustrate that the implementation is robust and efficient for large-scale applications. [ABSTRACT FROM AUTHOR]

Abstract: Centroidal Voronoi tessellations and optimal Delaunay triangulations can be approximated efficiently by non-linear optimisation algorithms. This paper demonstrates that the point distribution used to initialise the optimisation algorithms is important. Compared to conventional random initialisation, certain low-discrepancy point distributions help convergence towards more spatially regular results and require fewer iterations for planar and volumetric tessellations. [Copyright &y& Elsevier]

Abstract: In this paper, a new algorithm is proposed to solve the nonlinear constrained optimization problems. Unlike sequential quadratic programming (SQP) type methods, this algorithm does not involve solutions of quadratic programs. It is merely necessary to solve systems of linear equations with small scale to obtain a direction. The scheme is based on an idea of -effective active set strategies. The theoretical analysis shows that global and superlinear convergence can be induced under some suitable conditions. [Copyright &y& Elsevier]

Abstract: Integrated layout optimization of multi-component structures under static loads and random excitations is studied in this paper. Dynamic responses are obtained by using the mode superposition method, and the prestress effect introduced by the static loads is taken into account in the analyses of dynamic responses. Locations of embedded components and pseudo-densities related to supporting structure are chosen as design variables. Design sensitivities of the dynamic responses with respect to the pseudo-densities are analytically derived. The optimization algorithm GCMMA (Global Convergent Method of Moving Asymptotes algorithm) is used to minimize simultaneously the static strain energy and dynamic responses. Three numerical examples are presented to show the validity of the optimization procedure and its potential application in practical designs. [Copyright &y& Elsevier]

Abstract: The moving horizon estimation (MHE) problem is investigated in this paper for a class of networked systems with multiple packet dropouts. The packet dropout process is modeled as a Bernoulli random process and the networked system is described as a stochastic parameter system model. By choosing a stochastic cost function, a novel solution strategy is presented for MHE optimization problem with multiple packet dropouts. By considering noise constraints, the LOQO algorithm is used to solve the constrained MHE problem. Moreover, the convergence properties of the estimator are studied. Finally, two examples are given to demonstrate the effectiveness of the proposed method and the practical advantages of the MHE method. [Copyright &y& Elsevier]

Abstract: Variable order affine projection algorithms have been recently presented to be used when not only the convergence speed of the algorithm has to be adjusted but also its computational cost and its final residual error. These kind of affine projection (AP) algorithms improve the standard AP algorithm performance at steady state by reducing the residual mean square error. Furthermore these algorithms optimize computational cost by dynamically adjusting their projection order to convergence speed requirements. The main cost of the standard AP algorithm is due to the matrix inversion that appears in the coefficient update equation. Most efforts to decrease the computational cost of these algorithms have focused on the optimization of this matrix inversion. This paper deals with optimization of the computational cost of variable order AP algorithms by recursive calculation of the inverse signal matrix. Thus, a fast exact variable order AP algorithm is proposed. Exact iterative expressions to calculate the inverse matrix when the algorithm projection order either increases or decreases are incorporated into a variable order AP algorithm leading to a reduced complexity implementation. The simulation results show the proposed algorithm performs similarly to the variable order AP algorithms and it has a lower computational complexity. [Copyright &y& Elsevier]

Quantum-behaved particle swarm optimization (QPSO), motivated by concepts from quantum mechanics and particle swarm optimization (PSO), is a probabilistic optimization algorithm belonging to the bare-bones PSO family. Although it has been shown to perform well in finding the optimal solutions for many optimization problems, there has so far been little analysis on how it works in detail. This paper presents a comprehensive analysis of the QPSO algorithm. In the theoretical analysis, we analyze the behavior of a single particle in QPSO in terms of probability measure. Since the particle's behavior is influenced by the contraction-expansion (CE) coefficient, which is the most important parameter of the algorithm, the goal of the theoretical analysis is to find out the upper bound of the CE coefficient, within which the value of the CE coefficient selected can guarantee the convergence or boundedness of the particle's position. In the experimental analysis, the theoretical results are first validated by stochastic simulations for the particle's behavior. Then, based on the derived upper bound of the CE coefficient, we perform empirical studies on a suite of well-known benchmark functions to show how to control and select the value of the CE coefficient, in order to obtain generally good algorithmic performance in real world applications. Finally, a further performance comparison between QPSO and other variants of PSO on the benchmarks is made to show the efficiency of the QPSO algorithm with the proposed parameter control and selection methods. [ABSTRACT FROM AUTHOR]

In this paper, we suggest a new conjugate gradient algorithm that for all k≥0 both the descent and the conjugacy conditions are guaranteed. The search direction is selected as a linear combination of−g k+1 and s k , where g k+1=∇ f(x k+1), s k =x k+1−x k and the coefficients in this linear combination are selected in such a way that both the descent and the conjugacy conditions are satisfied at every iteration. In order to define the algorithm and to prove its convergence, the modified Wolfe line search is introduced, in which the parameter in the standard second Wolfe condition is changed at every iteration. It is shown that for general nonlinear functions, the algorithm with modified Wolfe line search generates directions bounded away from infinity. The algorithm uses an acceleration scheme modifying the step length α k in such a manner as to improve the reduction of the function values along the iterations. Numerical comparisons with some conjugate gradient algorithms using a set of 750 unconstrained optimization problems, some of them from the CUTE library, show that the computational scheme outperforms the known conjugate gradient algorithms like Hestenes and Stiefel, Polak et al., Dai and Yuan or hybrid Dai and Yuan, as well as the CG_DESCENT method by Hager and Zhang with the Wolfe line search conditions. [ABSTRACT FROM PUBLISHER]

Abstract: This paper investigates the overall convergence analysis and proposes a novel Random C/D Switchings PSO algorithm with random switchings between convergence operator and divergence operator. With respect to the standard PSO algorithm, its convergence analysis provides a fundamental theory of selecting convergence operator and divergence operator. During the process of finding the suboptimal or global solution, the random switchings between two typical operators, namely Operator C and Operator D, which are two different ways to update velocities of all particles, are conducted by a so-called convergence ratio parameter, which can determine the tradeoff between exploration ability and exploitation ability from the quantitative perspective. Numerical results on several benchmark functions demonstrate the following observations: (1) The proper convergence ratio is closely related to the landscape of objective function, the dimension of solution space and the number of local optimums. (2) Small convergence ratio, setting to 0.60 or 0.65, may benefit the optimization problem which has many local optimums in the high dimensional space; while large convergence ratio, setting to 0.85 or 0.9, is probably helpful for the optimization problem with few local optimums or flat landscape. [Copyright &y& Elsevier]

In this paper, a new combined extended Conjugate-Gradient (CG) and Variable-Metric (VM) methods is proposed for solving unconstrained large-scale numerical optimization problems. The basic idea is to choose a combination of the current gradient and some pervious search directions as a new search direction updated by Al-Bayati's SCVM-method to fit a new step-size parameter using Armijo Inexact Line Searches (ILS). This method is based on the ILS and its numerical properties are discussed using different non-linear test functions with various dimensions. The global convergence property of the new algorithm is investigated under few weak conditions. Numerical experiments show that the new algorithm seems to converge faster and is superior to some other similar methods in many situations. [ABSTRACT FROM AUTHOR]

The standard optimized link state routing (OLSR) introduces an interesting concept, the multipoint relays (MPRs), to mitigate message overhead during the flooding process. This paper propose a new algorithm for MPRs selection to enhance the performance of OLSR using particle swarm optimization sigmoid increasing inertia weight (PSOSIIW). The sigmoid increasing inertia weight has significance improve the particle swarm optimization (PSO) in terms of simplicity and quick convergence towards optimum solution. The new fitness function of PSOSIIW, packet delay of each node and degree of willingness are introduced to support MPRs selection in OLSR. The throughput, packet loss and end-to-end delay of the proposed method are examined using network simulator 2 (ns2). Overall results indicate that OLSR-PSOSIIW has shown good performance compared to the standard OLSR and OLSR-PSO, particularly for the throughput and end-to-end delay. Generally the proposed OLSR-PSOSIIW shows advantage of using PSO for optimizing routing paths in the MPRs selection algorithm. [ABSTRACT FROM AUTHOR]

Abstract: Accurate kinetic models are of great significance for the simulation and analysis for hydrogen fuel cells. The proton exchange membrane (PEM) fuel cell is a complex nonlinear, multi-variable system. The mathematical modeling of PEM fuel cell usually leads to nonlinear parameter estimation problems which often contain more than one minimum. In this paper, a novel bio-inspired P systems based optimization algorithm, named BIPOA, is proposed to solve PEM fuel cell model parameter estimation problems. In BIPOA, the nested membrane structure and new rules such as adaptive mutation rule, partial migration rule and autophagy rule are combined to improve the algorithm''s global search capacities and convergence accuracy. Studies on some benchmark test functions indicate that the BIPOA outperforms the other two methods (PSOPS and GAs) in both convergence speed and accuracy. In addition, experimental results reveal that the model predictive outputs are in better agreement with the actual experimental data. Therefore, the BIPOA is a helpful and reliable technique for estimating the PEM fuel cell model parameters and is available to other complex parameter estimation problems of fuel cell models. [Copyright &y& Elsevier]

In this paper, we propose and analyze an algorithm that couples the gradient method with a general exterior penalization scheme for constrained or hierarchical minimization of convex functions in Hilbert spaces. We prove that a proper but simple choice of the step sizes and penalization parameters guarantees the convergence of the algorithm to solutions for the optimization problem. We also establish robustness and stability results that account for numerical approximation errors, discuss implementation issues and provide examples in finite and infinite dimension. [ABSTRACT FROM AUTHOR]

The Nelder-Mead algorithm, a longstanding direct search method for unconstrained optimization published in 1965, is designed to minimize a scalar-valued function f of n real variables using only function values, without any derivative information. Each Nelder-Mead iteration is associated with a nondegenerate simplex defined by n + 1 vertices and their function values; a typical iteration produces a new simplex by replacing the worst vertex by a new point. Despite the method's widespread use, theoretical results have been limited: for strictly convex objective functions of one variable with bounded level sets, the algorithm always converges to the minimizer; for such functions of two variables, the diameter of the simplex converges to zero but examples constructed by McKinnon show that the algorithm may converge to a nonminimizing point. This paper considers the restricted Nelder-Mead algorithm, a variant that does not allow expansion steps. In two dimensions we show that for any nondegenerate starting simplex and any twice-continuously differentiable function with positive definite Hessian and bounded level sets, the algorithm always converges to the minimizer. The proof is based on treating the method as a discrete dynamical system and relies on several techniques that are nonstandard in convergence proofs for unconstrained optimization. [ABSTRACT FROM AUTHOR]

The objective of the dynamic economic dispatch (DED) problem is to schedule power generation for the online units for a given time horizon economically, satisfying various operational constraints. Due to the effect of valve-point effects and prohibited operating zones (POZs) in the generating units cost functions, DED problem is a highly non-linear and non-convex optimization problem. The DED problem even may be more complicated if transmission losses and ramp-rate constraints are taken into account. This paper presents a novel and heuristic algorithm to solve DED problem of generating units, by employing time varying acceleration coefficients iteration particle swarm optimization (TVAC-IPSO) method. The effectiveness of the proposed method is examined and validated by carrying out extensive tests on different test systems, i.e. 5-unit and 10-unit test systems. Valve-point effects, POZs and ramp-rate constraints along with transmission losses are considered. To examine the efficiency of the proposed TVAC-IPSO algorithm, comprehensive studies are carried out, which compare convergence properties of the proposed TVAC-IPSO approach with conventional PSO algorithm, in addition to the other recently reported approaches. Numerical results show that the TVAC-IPSO method has good convergence properties and the generation costs resulted from the proposed method are lower than other algorithms reported in recent literature. [Copyright &y& Elsevier]

Abstract: In this paper we focus on the application of global stochastic optimization methods to extremum estimators. We propose a general stochastic method—the master method—which includes several stochastic optimization algorithms as a particular case. The proposed method is sufficiently general to include the Solis–Wets method, the improving hit-and-run algorithm, and a stochastic version of the zigzag algorithm. A matrix formulation of the master method is presented and some specific results are given for the stochastic zigzag algorithm. Convergence of the proposed method is established under a mild set of conditions, and a simple regression model is used to illustrate the method. [Copyright &y& Elsevier]

Abstract: In this paper, an improved interior-type feasible QP-free algorithm for inequality constrained optimization problems is proposed. At each iteration, by solving three systems of linear equations with the same coefficient matrix, a search direction is generated. The algorithm is proved to be globally and superlinearly convergent under some mild conditions. Preliminary numerical results show that the proposed algorithm may be promising. Advantages of the algorithm include: the uniformly nonsingularity of the coefficient matrices without the strictly complementarity condition is obtained. Moreover, the global convergence is achieved even if the number of the stationary points is infinite. [Copyright &y& Elsevier]

Abstract: Much research on Artificial Intelligence (AI) has been focusing on exploring various potential applications of intelligent systems. In most cases, the researches attempt to model human intelligence by mimicking the brain structure and function, but they ignore an important aspect in human learning and decision making: the artificial emotion. In this paper, we present a new unconstrained global optimization method, hybrid chaos optimization algorithm with artificial emotion (HCOAAE), which avoids trapping to local minima, and improves convergence in large space and high-dimension optimization problems. The main purpose of artificial emotion is to mimic decision making behavior process of humans, to choose most suitable parameters of HCOAAE and decide whether to change current search strategy or not in the next iteration. Numerical simulations of 13 benchmark functions with different dimensions are used to test the performance of HCOAAE. Experimental results show that the proposed method significantly outperforms the existing methods in terms of convergence speed, computational effectiveness, and numerical stability. [Copyright &y& Elsevier]

Abstract: In this paper, we investigate a discriminative visual dictionary learning method for boosting the classification performance. Tied to the K-means clustering philosophy, those popular algorithms for visual dictionary learning cannot guarantee the well-separation of the normalized visual word frequency vectors from distinctive classes or large label distances. The rationale of this work is to harness sample label information for learning visual dictionary in a supervised manner, and this target is then formulated as an objective function, where each sample element, e.g., SIFT descriptor, is expected to be close to its assigned visual word, and at the same time the normalized aggregative visual word frequency vectors are expected to possess the property that kindred samples shall be close to each other while inhomogeneous samples shall be far away. By relaxing the hard binary constraints to soft nonnegative ones, a multiplicative nonnegative update procedure is proposed to optimize the objective function along with theoretic convergence proof. Extensive experiments on classification tasks (i.e., natural scene and sports event classifications) all demonstrate the superiority of this proposed framework over conventional clustering based visual dictionary learning. [Copyright &y& Elsevier]

In this paper, a method for solving constrained convex optimization problems is introduced. The problem is cast equivalently as a parametric unconstrained one, the (single) parameter being the optimal value of the original problem. At each stage of the algorithm the parameter is updated, and the resulting subproblem is only approximately solved. A linear rate of convergence of the parameter sequence is established. Using an optimal gradient method due to Nesterov [Dokl. Akad. Nauk SSSR, 269 (1983), pp. 543 547] to solve the arising subproblems, it is proved that the resulting gradient-based algorithm requires an overall of O(log(1/ε)/√ε) inner iterations to obtain an ε-optimal and feasible solution. An image deblurring problem is solved, demonstrating the capability of the algorithm to solve large-scale problems within reasonable accuracy. [ABSTRACT FROM AUTHOR]

The main purpose of the paper is to optimize state feedback parameters using intelligent method, GA, Hermite-Biehler, and chaos algorithm. GA is implemented for local search but it has some deficiencies such as trapping into a local minimum and slow convergence, so the combination of Hermite-Biehler and chaos algorithm has been added to GA to avoid its deficiencies. Dividing search space is usually done by distributed population genetic algorithm (DPGA).Moreover, using generalized Hermite-Biehler Theorem can find the domain of parameters. In order to speed up the convergence at the first step, Hermite-Biehler method finds some intervals for controller, in the next step the GA will be added, and, finally, chaos disturbance will help the algorithm to reach a global minimum. Therefore, the proposed method can optimize the parameters of the state feedback controller. [ABSTRACT FROM AUTHOR]

Abstract: This paper presents extended artificial physics optimization (EAPO), a population-based, stochastic, evolutionary algorithm (EA) for multidimensional search and optimization. EAPO extends the physicomimetics-based Artificial Physics Optimization (APO) algorithm by including each individual’s best fitness history. Including the history improves EAPO’s search capability compared to APO. EAPO and APO invoke a gravitational metaphor in which the force of gravity may be attractive or repulsive, the aggregate effect of which is to move individuals toward local and global optima. A proof of convergence is presented that reveals the conditions under which EAPO is guaranteed to converge. Discrete-time linear system theory is used to develop a second-order difference equation for an individual’s stochastic position vector as a function of time step. Stable solutions require eigenvalues inside the unit circle, leading to explicit convergence criteria relating the run parameters {m i , w, G}. EAPO is tested against several benchmark functions with excellent results. The algorithm converges more quickly than APO and with better diversity. [Copyright &y& Elsevier]

The alternating direction multipliers method (ADMM) has been recently proposed as a practical and efficient algorithm for distributed computing. We discuss its applicability to the average consensus problem in this paper. By carefully relaxing ADMM augmentation coefficients we are able to analytically investigate its properties, and to propose simple and strict analytical bounds. These provide a clear indication on how to choose system parameters for optimized performance. We prove both analytically and via simulations that the proposed approach exhibits convergence speed between the best in the literature (classical and optimized solutions), while providing the most powerful resilience to noise. [ABSTRACT FROM AUTHOR]

The recently emerged compressive sensing (CS) framework aims to acquire signals at reduced sample rates compared to the classical Shannon-Nyquist rate. To date, the CS theory has assumed primarily real-valued measurements; it has recently been demonstrated that accurate and stable signal acquisition is still possible even when each measurement is quantized to just a single bit. This property enables the design of simplified CS acquisition hardware based around a simple sign comparator rather than a more complex analog-to-digital converter; moreover, it ensures robustness to gross nonlinearities applied to the measurements. In this paper we introduce a new algorithm—restricted-step shrinkage (RSS)—to recover sparse signals from 1-bit CS measurements. In contrast to previous algorithms for 1-bit CS, RSS has provable convergence guarantees, is about an order of magnitude faster, and achieves higher average recovery signal-to-noise ratio. RSS is similar in spirit to trust-region methods for nonconvex optimization on the unit sphere, which are relatively unexplored in signal processing and hence of independent interest. [ABSTRACT FROM AUTHOR]

Abstract: Support vector regression (SVR) is a powerful tool in modeling and prediction tasks with widespread application in many areas. The most representative algorithms to train SVR models are Shevade et al.''s Modification 2 and Lin''s WSS1 and WSS2 methods in the LIBSVM library. Both are variants of standard SMO in which the updating pairs selected are those that most violate the Karush–Kuhn–Tucker optimality conditions, to which LIBSVM adds a heuristic to improve the decrease in the objective function. In this paper, and after presenting a simple derivation of the updating procedure based on a greedy maximization of the gain in the objective function, we show how cycle-breaking techniques that accelerate the convergence of support vector machines (SVM) in classification can also be applied under this framework, resulting in significantly improved training times for SVR. [Copyright &y& Elsevier]

A level set method for the topological optimization of domains is proposed and analysed in this paper. The framework of the analysis is that of elliptic optimal control problems with a binary control acting as a source term of the state equation. The main ingredient of the algorithm is the concept of topological derivative. Convergence results are proven and illustrated by some numerical experiments. [ABSTRACT FROM AUTHOR]

In this paper, we present a non-negative patch alignment framework (NPAF) to unify popular non-negative matrix factorization (NMF) related dimension reduction algorithms. It offers a new viewpoint to better understand the common property of different NMF algorithms. Although multiplicative update rule (MUR) can solve NPAF and is easy to implement, it converges slowly. Thus, we propose a fast gradient descent (FGD) to overcome the aforementioned problem. FGD uses the Newton method to search the optimal step size, and thus converges faster than MUR. Experiments on synthetic and real-world datasets confirm the efficiency of FGD compared with MUR for optimizing NPAF. Based on NPAF, we develop non-negative discriminative locality alignment (NDLA). Experiments on face image and handwritten datasets suggest the effectiveness of NDLA in classification tasks and its robustness to image occlusions, compared with representative NMF-related dimension reduction algorithms. [ABSTRACT FROM AUTHOR]

Abstract: A directed searching optimization algorithm (DSO) is proposed to solve constrained optimization problems in this paper. The proposed algorithm includes two important operations — position updating and genetic mutation. Position updating enables the non-best solution vectors to mimic the best one, which is beneficial to the convergence of the DSO; genetic mutation can increase the diversity of individuals, which is beneficial to preventing the premature convergence of the DSO. In addition, we adopt the penalty function method to balance objective and constraint violations. We can obtain satisfactory solutions for constrained optimization problems by combining the DSO and the penalty function method. Experimental results indicate that the proposed algorithm can be an efficient alternative on solving constrained optimization problems. [Copyright &y& Elsevier]

Abstract: Last few years have witnessed the development of various multi-objective evolutionary algorithms since they allow the generation of the overall Pareto front for multi-objective optimizations. With the problems in the real-world becoming more and more complex, however, no reported work in the literature focuses on the high-dimensional objective optimizations (HOPs). In this paper, we propose an evolutionary algorithm named HOEA (high-dimensional objective evolutionary algorithm) for HOPs. By adopting the concept of nonlinear definition for optimizing object, HOPs can be solved by HOEA, while the well-known multi-objective evolutionary algorithms work poorly on HOPs. We further analyze the nonlinear dynamic properties of HOEA on the basis of martingale theoretical framework. The theoretical results indicate that this new algorithm is indeed capable of achieving convergence. We also conduct experiments on HOEA with two representative test instances. The experimental results either confirm our theoretical results or show that the proposed algorithm is efficient and effective for HOPs. [Copyright &y& Elsevier]

Abstract: In the paper, an incomplete active set algorithm is given for mathematical programs with linear complementarity constraints (MPLCC). At each iteration, a finite number of inner-iterations are contained for approximately solving the relaxed nonlinear optimization problem. If the feasible region of the MPLCC is bounded, under the uniform linear independence constraint qualification (LICQ), any cluster point of the sequence generated from the algorithm is a B-stationary point of the MPLCC. Preliminary numerical tests show that the algorithm is promising. [Copyright &y& Elsevier]

Abstract: This paper presents an augmented Lagrangian methodology with a stochastic population based algorithm for solving nonlinear constrained global optimization problems. The method approximately solves a sequence of simple bound global optimization subproblems using a fish swarm intelligent algorithm. A stochastic convergence analysis of the fish swarm iterative process is included. Numerical results with a benchmark set of problems are shown, including a comparison with other stochastic-type algorithms. [Copyright &y& Elsevier]