Search

Stochastic gradient descent (SGD) has become the method of choice for training highly complex and nonconvex models since it can not only recover good solutions to minimize training errors but also generalize well. Computational and statistical properties are separately studied to understand the behavior of SGD in the literature. However, there is a lacking study to jointly consider the computational and statistical properties in a nonconvex learning setting. In this paper, we develop novel learning rates of SGD for nonconvex learning by presenting high-probability bounds for both computational and statistical errors. We show that the complexity of SGD iterates grows in a controllable manner with respect to the iteration number, which sheds insights on how an implicit regularization can be achieved by tuning the number of passes to balance the computational and statistical errors. As a byproduct, we also slightly refine the existing studies on the uniform convergence of gradients by showing its connection to Rademacher chaos complexities. [ABSTRACT FROM AUTHOR]

When solving constrained optimization problems (COPs) by evolutionary algorithms, the search algorithm plays a crucial role. In general, we expect that the search algorithm has the capability to balance not only diversity and convergence but also constraints and objective function during the evolution. For this purpose, this paper proposes a composite differential evolution (DE) for constrained optimization, which includes three different trial vector generation strategies with distinct advantages. In order to strike a balance between diversity and convergence, one of these three trial vector generation strategies is able to increase diversity, and the other two exhibit the property of convergence. In addition, to accomplish the tradeoff between constraints and objective function, one of the two trial vector generation strategies for convergence is guided by the individual with the least degree of constraint violation in the population, and the other is guided by the individual with the best objective function value in the population. After producing offspring by the proposed composite DE, the feasibility rule and the $\boldsymbol {\varepsilon }$ constrained method are combined elaborately for selection in this paper. Moreover, a restart scheme is proposed to help the population jump out of a local optimum in the infeasible region for some extremely complicated COPs. By assembling the above techniques together, a constrained composite DE is proposed. The experiments on two sets of benchmark test functions with various features, i.e., 24 test functions from IEEE CEC2006 and 18 test functions with 10 dimensions and 30 dimensions from IEEE CEC2010, have demonstrated that the proposed method shows better or at least competitive performance against other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Operational indices optimization is crucial for the global optimization in beneficiation processes. This paper presents a multitasking multiobjective evolutionary method to solve operational indices optimization, which involves a formulated multiobjective multifactorial operational indices optimization (MO-MFO) problem and the proposed multiobjective MFO algorithm for solving the established MO-MFO problem. The MO-MFO problem includes multiple level of accurate models of operational indices optimization, which are generated on the basis of a data set collected from production. Among the formulated models, the most accurate one is considered to be the original functions of the solved problem, while the remained models are the helper tasks to accelerate the optimization of the most accurate model. For the MFO algorithm, the assistant models are alternatively in multitasking environment with the accurate model to transfer their knowledge to the accurate model during optimization in order to enhance the convergence of the accurate model. Meanwhile, the recently proposed two-stage assortative mating strategy for a multiobjective MFO algorithm is applied to transfer knowledge among multitasking tasks. The proposed multitasking framework for operational indices optimization has conducted on 10 different production conditions of beneficiation. Simulation results demonstrate its effectiveness in addressing the operational indices optimization of beneficiation problem. Note to Practitioners—Operational indices optimization is a typical approach to achieve global production optimization by efficiently coordinating all the indices to improve the production indices. In this paper, a multiobjective multitasking framework is developed to address the operational indices optimization, which includes a multitasking multiobjective operational indices optimization problem formulation and a multitasking multiobjective evolutionary optimization to solve the above-formulated optimization problem. The proposed approach can achieve a solution set for the decision-making. The simulation results on a real beneficiation process in China with 10 operational conditions show that the proposed approach is able to obtain a superior solution set, which is associated with a higher grade and yield of the product. [ABSTRACT FROM AUTHOR]

This paper derives consistency results for estimation in the finite direct sum of reproducing kernel Hilbert spaces (RKHS) for dependent data. The link between penalized and constrained estimation is established. We consider the relation between topological equivalent norms for direct sums of RKHS. These norms have different implications for estimation. Estimation in a ball of the RKHS defined by these norms essentially results in estimation with a ridge and Lasso penalty, respectively. A greedy algorithm for the solution of the estimation problem under these two norms is discussed for general loss functions. [ABSTRACT FROM AUTHOR]

Differential evolution is one of the most powerful evolutionary algorithms for single objective optimization problems in the literature. Its application in the multiobjective optimization problems is also very successful, and many kinds of promising multiobjective differential evolution (MODE) algorithms have been proposed in the literature. This paper develops a new variant of MODE with two features. First, a set of personal archives are maintained to evolve the search process instead of a population with a fixed size, and a truncation procedure is used to enhance selection pressure. Second, multiple biased mutation operators incorporating the target solution quality are proposed, and an adaptive selection method is adopted to allocate the mutation operators to solutions. The proposed MODE is referred to as the MODE with personal archive and biased self-adaptive mutation selection (BiasMOSaDE). A set of 31 benchmark multiobjective problems selected from the literature are adopted to evaluate its performance. Computational results illustrate that the proposed BiasMOSaDE is competitive or even superior to several state-of-the-art MODEs in the literature. [ABSTRACT FROM AUTHOR]

Due to the growth of modern dataset size and the desire to harness computing power of multiple machines, there is a recent surge of interest in the design of distributed machine learning algorithms. However, distributed algorithms are sensitive to Byzantine attackers who can send falsified data to prevent the convergence of algorithms or lead the algorithms to converge to value of the attackers’ choice. Some recent work proposed interesting algorithms that can deal with the scenario when up to half of the workers are compromised. In this paper, we propose a novel algorithm that can deal with an arbitrary number of Byzantine attackers. The main idea is to ask the parameter server to randomly select a small clean dataset and compute noisy gradient using this small dataset. This noisy gradient will then be used as a ground truth to filter out information sent by compromised workers. We show that the proposed algorithm converges to the neighborhood of the population minimizer regardless the number of Byzantine attackers. We further provide numerical examples to show that the proposed algorithm can benefit from the presence of good workers and achieve better performance than existing algorithms. [ABSTRACT FROM AUTHOR]

This paper compares three modern and two classical multiobjective optimizers (MOOs) as applied to real-world problems in electromagnetics. The behavior of sophisticated optimizers on simple test functions has been studied exhaustively. In contrast, the algorithms here are tested on practical applications, where the function evaluations are computationally expensive, making the convergence rate a crucial factor. The examples considered include the optimization of a narrowband slot antenna, a mushroom-type electromagnetic bandgap structure, and an ultrawideband Vivaldi antenna. Another popular topic in the literature is in comparing classical MOOs on electromagnetics problems. The modern optimizers chosen in this paper are state of the art and each has a distinct design philosophy. This paper introduces two unique MOOs to the electromagnetics community: BORG, an auto-adaptive genetic algorithm and the Multi-Objective Covariance Matrix Adaptation Evolutionary Strategy (MO-CMA-ES), an extension of the popular single-objective CMA-ES. These algorithms are compared to the Multi-objective Evolutionary Algorithm based on Decomposition (MOEA/D), a Chebysheff scalarization algorithm, and two classical MOOs. This paper will study the behavior of these algorithms on problems in electromagnetics with a limited number of function evaluations using five distinct metrics and will provide useful guidelines and recommended optimizer settings. [ABSTRACT FROM AUTHOR]

This paper proposes a united-based imperialist competitive algorithm (UICA) for compensatory neural fuzzy systems. The original imperialist competitive algorithm (ICA) comprises numerous empires in the population, and each empire comprises one imperialist and some colonies. Each country represents a feasible solution in the empire, and the more favorable solutions become imperialists, taking over less favorable solutions (i.e., colonies). In the ICA, each colony moves toward its relevant imperialist according to an assimilation policy. This paper proposes a UICA that focuses on this assimilation policy to explore the characteristics of the colonies. The assimilation policy consists of three major parts in this paper. In the first part, a colony searches for the best previous position of the colony. In the second part, the colony faces the best-so-far imperialist. In the third part, the colony moves toward the corresponding imperialist of the colony. The proposed UICA was applied to nonlinear system problems, and the experimental results indicated that the proposed UICA is effective. [ABSTRACT FROM PUBLISHER]

In this paper, we introduce a novel method for the discovery of value functions for Markov decision processes (MDPs). This method, which we call value function discovery (VFD), is based on ideas from the evolutionary algorithm field. VFDs key feature is that it discovers descriptions of value functions that are algebraic in nature. This feature is unique, because the descriptions include the model parameters of the MDP. The algebraic expression of the value function discovered by VFD can be used in several scenarios, e.g., conversion to a policy (with one-step policy improvement) or control of systems with time-varying parameters. The work in this paper is a first step toward exploring potential usage scenarios of discovered value functions. We give a detailed description of VFD and illustrate its application on an example MDP. For this MDP, we let VFD discover an algebraic description of a value function that closely resembles the optimal value function. The discovered value function is then used to obtain a policy, which we compare numerically to the optimal policy of the MDP. The resulting policy shows near-optimal performance on a wide range of model parameters. Finally, we identify and discuss future application scenarios of discovered value functions. [ABSTRACT FROM PUBLISHER]

In this paper, we propose a hybrid genetic algorithm (GA) for the optimal shape design of an axially symmetric dual-reflector antenna by combining the GA with the moving least square (MLS), which enhances the convergence rate and the global search performance. The MLS is used to construct local interpolation functions from non-uniform sample data and to estimate new superior positions. By combining these superior positions in the next generation, the MLS-GA shows better search performance for the global optimum and a faster convergence rate than those of the conventional GA. To verify the proposed MLS-GA, we applied it to the optimal shape design of the DRA at the W-band. [ABSTRACT FROM AUTHOR]

In the population protocol model [Angluin et al. 2006], it is impossible to design a self-stabilizing leader election protocol without any knowledge of the exact number of nodes in the system. The notion of loose-stabilization, which relaxes the closure requirement of self -stabilization, was introduced in 2009 to circumvent this impossibility. The notion can be described as follows: a loosely-stabilizing protocol guarantees that, starting from any initial configuration, a system reaches a safe configuration eventually, and after that, the system maintains its specification (e.g., the unique leader) not forever, but for a sufficiently long time. The previous work of the authors presented a loosely-stabilizing protocol that solves the leader election on complete graphs using only a given upper bound $N$N on the number of nodes $n$n in the system, instead of the exact value of $n$n. In this paper, we propose two loosely-stabilizing protocols that solve leader election for arbitrary graphs. One is a deterministic protocol that uses the unique identifiers of nodes while the other is a probabilistic protocol that works on anonymous networks. Given an upper bound $N$N on the number of nodes, both protocols maintain a unique leader for $\Omega (Ne^{2N})$Ω(Ne2N) expected steps (holding time) after entering a safe configuration. The first algorithm enters a safe configuration within $O(mN \, \log n)$O(mNlogn) expected steps (convergence time) while the second one does this within $O(mN^2\,\log N)$O(mN2logN) expected steps, where $m$m is the number of edges in the graph. Both protocols require only $O(\log N)$O(logN) bits for each node's memory. A novel concept, called the same speed timer is introduced, by which all nodes of the system can count down their timers at the same speed. This concept allows to achieve fast convergence time of both algorithms. To design the second protocol, we design a self-stabilizing two-hop coloring protocol, which is interesting in its own right. This protocol uses only $O(\log N)$O(logN) memory space per node. We establish a lower bound. Any loosely-stabilizing leader election protocol with expected exponential holding time requires $\Omega (mN)$Ω(mN) expected convergence time. This lower bound shows a near-optimality of the first algorithm. [ABSTRACT FROM AUTHOR]

In sponsored content and service markets, the content and service providers are able to subsidize their target mobile users through directly paying the mobile network operator to lower the price of the data/service access charged by the network operator to the mobile users. The sponsoring mechanism leads to a surge in mobile data and service demand, which in return compensates for the sponsoring cost and benefits the content/service providers. In this paper, we study the interactions among the three parties in the market, namely, the mobile users, the content/service providers, and the network operator, as a two-level game with multiple Stackelberg (i.e., leader) players. Our study is featured by the consideration of global network effects owning to consumers’ grouping. Since the mobile users may have bounded rationality, we model the service-selection process among them as an evolutionary-population follower sub-game. Meanwhile, we model the pricing-then-sponsoring process between the content/service providers and the network operator as a non-cooperative equilibrium searching problem. By investigating the structure of the proposed game, we reveal a few important properties regarding the equilibrium existence and propose a distributed, projection-based algorithm for iterative equilibrium searching. Simulation results validate the convergence of the proposed algorithm and demonstrate how sponsoring helps improve both the providers’ profits and the users’ experience. [ABSTRACT FROM AUTHOR]

This paper proposes a hybrid algorithm based on the genetic algorithm (GA) and the evolution strategy (ES) for the electromagnetic optimization problem. The GA is not good enough at times in searching the optimal solution from the view point of the convergence speed and the solution quality, while the ES has the risk of being trapped in a local minimum. The hybrid algorithm is composed of GA and ES in order to make up for these defects. First, we reached the vicinity of optimal solution using the GA. Then, the ES is used to find the accurate optimal solution. The switching point can be a main issue, which is also resolved in this paper. First, the performance of the convergence speed and the solution accuracy are comparatively tested using the known functions. In addition, the optimized design of the 2.45 GHz coplanar waveguide-fed circularly polarized antenna is carried out as a practical application. Only the GA and the hybrid algorithm reach the satisfactory value, and the more rapid convergence can be shown by the ES in this hybrid method after 380 iterations. [ABSTRACT FROM AUTHOR]

The application of the cross entropy (CE) method to tone injection (TI) can make the peak-to-average power ratio (PAPR) of the orthogonal frequency division multiplexing (OFDM) signals reduced efficiently with great complexity savings. However, as a population-based search method, the CE method requires large samples to find a high-quality solution, which means higher complexity and more time consuming concerning convergence. To reduce the complexity while achieving promising PAPR performance, an improved CE based TI scheme is proposed in this paper. First, a structured extended constellation for rectangular quadrature amplitude modulation is designed to provide more freedom degrees. With proper parameter selection, the proposed extended constellation can offer more design flexibilities and achieve a better tradeoff between freedom degree and power increase. Then, by analyzing the impact of the subcarrier candidate population on the conventional CE based TI scheme, we derive an explicit expression for the determination of the size of the subcarrier candidate population. And, based on the prior information carried by the clipping signal, proper candidate subcarriers are selected. Finally, an improved CE based TI scheme is proposed, which can efficiently enhance the solution quality and greatly accelerate the convergence speed. Simulation results demonstrate that the improved CE based TI scheme achieves great improvements on the performance of PAPR, bit error rate, power increase and complexity compared with the conventional CE based TI scheme. [ABSTRACT FROM PUBLISHER]

This paper proposes a hybrid, multiobjective optimization algorithm enabling global optimum tracking in permanent-magnet (PM) traction motor design. The methodology developed is based on the Artificial Bee Colony technique, strength Pareto evolutionary algorithm, and differential evolution strategy ensuring fast and reliable convergence to the optimal Pareto front. The effectiveness of the derived methodology is compared with other well-established and powerful algorithms from the literature through both appropriate test functions and an application example concerning an unequal teeth surface-mounted PM wheel motor design. [ABSTRACT FROM AUTHOR]

A finite-element analysis-based optimal design of an electric machine takes considerable time for its objective evaluation and has many local minima. Thus, selecting an appropriate global convergence optimization with fast convergence speed is necessary in the optimal design of an electric machine. In this paper, a novel global search optimization algorithm, mass ionized particle optimization (MIPO), is newly proposed. The MIPO is the population-based algorithm, which reflects the interactive force between the ionized particles. The global convergence and the convergence speed are validated by comparison with the particle swarm optimization, which have already been proved for its global convergence when applied to a well-known Goldstein–Price function as a benchmark function. In addition, the algorithm has been applied to the optimal design of an interior permanent magnet synchronous machine aiming for its torque ripple reduction. [ABSTRACT FROM AUTHOR]

We consider the high-dimensional discriminant analysis problem. For this problem, different methods have been proposed and justified by establishing exact convergence rates for the classification risk, as well as the \ell 2 convergence results to the discriminative rule. However, sharp theoretical analysis for the variable selection performance of these procedures have not been established, even though model interpretation is of fundamental importance in scientific data analysis. This paper bridges the gap by providing sharp sufficient conditions for consistent variable selection using the sparse discriminant analysis. Through careful analysis, we establish rates of convergence that are significantly faster than the best known results and admit an optimal scaling of the sample size $n$ , dimensionality $p$ , and sparsity level $s$ in the high-dimensional setting. Sufficient conditions are complemented by the necessary information theoretic limits on the variable selection problem in the context of high-dimensional discriminant analysis. Exploiting a numerical equivalence result, our method also establish the optimal results for the ROAD estimator and the sparse optimal scoring estimator. Furthermore, we analyze an exhaustive search procedure, whose performance serves as a benchmark, and show that it is variable selection consistent under weaker conditions. Extensive simulations demonstrating the sharpness of the bounds are also provided. [ABSTRACT FROM AUTHOR]

In relay-assisted cellular networks, when and how to cooperate for potential relay nodes is an important issue, particularly from the energy-aware perspective. Moreover, addressing the dynamic cooperative partner-selection problem with incomplete private information is a challenging issue, particularly in the competitive and changing environments. In this paper, we model the energy-aware dynamic competition of cooperative partner selection as an evolutionary game. Specifically, the replicator dynamics are applied to model the adaptation of strategic interactions while considering the dynamic nature in time dependence, and the evolutionarily stable strategy (ESS) is referred to as the robust solution of the proposed game. Moreover, we prove the uniqueness of the resulting ESS, guarantee the convergence to the ESS, and analyze the convergence behavior under the replicator dynamics from different perspectives. In practical terms, we further develop a cooperative partner-selection evolutionary algorithm based on the Q-learning approach, which only requires local knowledge for the potential relay nodes. In addition, we extend our observation by studying the dynamic cooperative partner selection with delay and discuss the properties of the resulting ESS. [ABSTRACT FROM AUTHOR]

A near-optimal base-station antenna array synthesis suitable for broadcasting applications is presented. The array is required to provide a high-gain radiation pattern with a main lobe slightly tilted from the horizontal plane and null filling inside an angular sector under the main lobe. To satisfy the above requirements, a novel invasive weed optimization (IWO) variant called Taguchi initialized IWO (TI-IWO) is proposed in this paper. In TI-IWO, the Taguchi's optimization method is employed to initialize effectively the positions of the weeds used by the IWO method. In this way, the fitness function starts from lower values and, thus, the TI-IWO method finds better near-optimal solutions than the conventional IWO method. The proposed method has been applied to linear arrays. Due to its easy implementation in practice, a uniform-amplitude excitation distribution is considered to be applied on the array elements. Two cases of isotropic source arrays are studied under specific requirements for maximum possible gain, main lobe tilting, and null filling. Also, the TI-IWO method is applied to optimize realistic cases of collinear wire dipole arrays in front of a mast under the same requirements and an additional one concerning the impedance matching of all the dipoles. [ABSTRACT FROM AUTHOR]