Search

Weight elimination offers a simple and efficient improvement of training algorithm of feedforward neural networks. It is a general regularization technique in terms of the flexible scaling parameters. Actually, the weight elimination technique also contains the weight decay regularization for a large scaling parameter. Many applications of this technique and its improvements have been reported. However, there is little research concentrated on its convergence behavior. In this paper, we theoretically analyze the weight elimination for cyclic learning method and determine the conditions for the uniform boundedness of weight sequence, and weak and strong convergence. Based on the assumed network parameters, the optimal choice for the scaling parameter can also be determined. Moreover, two illustrative simulations have been done to support the theoretical explorations as well. [ABSTRACT FROM AUTHOR]

In this paper, the recursive least square algorithm is designed for the big data learning of a feedforward neural network. The proposed method as the combination of the recursive least square and feedforward neural network obtains four advantages over the alone algorithms: it requires less number of regressors, it is fast, it has the learning ability, and it is more compact. Stability, convergence, boundedness of parameters, and local minimum avoidance of the proposed technique are guaranteed. The introduced strategy is applied for the modeling of the crude oil blending process. [ABSTRACT FROM AUTHOR]

Abstract: Selection of the optimal neural architecture to solve a pattern classification problem entails to choose the relevant input units, the number of hidden neurons and its corresponding interconnection weights. This problem has been widely studied in many research works but their solutions usually involve excessive computational cost in most of the problems and they do not provide a unique solution. This paper proposes a new technique to efficiently design the MultiLayer Perceptron (MLP) architecture for classification using the Extreme Learning Machine (ELM) algorithm. The proposed method provides a high generalization capability and a unique solution for the architecture design. Moreover, the selected final network only retains those input connections that are relevant for the classification task. Experimental results show these advantages. [Copyright &y& Elsevier]

Abstract: A comparison of behavior-based and planning approaches of robot control is presented in this paper. We focus on miniature mobile robotic agents with limited sensory abilities. Two reactive control mechanisms for an agent are considered—a radial basis function neural network trained by evolutionary algorithm and a traditional reinforcement learning algorithm over a finite agent state space. The control architecture based on localization and planning is compared to the former method. [Copyright &y& Elsevier]

Abstract: Evaluation of the current board position is critical in computer game engines. In sufficiently complex games, such a task is too difficult for a traditional brute force search to accomplish, even when combined with expert knowledge bases. This motivates the investigation of alternatives. This paper investigates the combination of neural networks, particle swarm optimization (PSO), and evolutionary algorithms (EAs) to train a board evaluator from zero knowledge. By enhancing the survivors of an EA with PSO, the hybrid algorithm successfully trains the high-dimensional neural networks to provide an evaluation of the game board through self-play. Experimental results, on the benchmark game of Capture Go, demonstrate that the hybrid algorithm can be more powerful than its individual parts, with the system playing against EA and PSO trained game engines. Also, the winning results of tournaments against a Hill-Climbing trained game engine confirm that the improvement comes from the hybrid algorithm itself. The hybrid game engine is also demonstrated against a hand-coded defensive player and a web player. [Copyright &y& Elsevier]

In this paper we present in a continuous-time framework an online approach to direct adaptive optimal control with infinite horizon cost for nonlinear systems. The algorithm converges online to the optimal control solution without knowledge of the internal system dynamics. Closed-loop dynamic stability is guaranteed throughout. The algorithm is based on a reinforcement learning scheme, namely Policy Iterations, and makes use of neural networks, in an Actor/Critic structure, to parametrically represent the control policy and the performance of the control system. The two neural networks are trained to express the optimal controller and optimal cost function which describes the infinite horizon control performance. Convergence of the algorithm is proven under the realistic assumption that the two neural networks do not provide perfect representations for the nonlinear control and cost functions. The result is a hybrid control structure which involves a continuous-time controller and a supervisory adaptation structure which operates based on data sampled from the plant and from the continuous-time performance dynamics. Such control structure is unlike any standard form of controllers previously seen in the literature. Simulation results, obtained considering two second-order nonlinear systems, are provided. [Copyright &y& Elsevier]

Abstract: Modeling neural networks with ordinary differential equations systems is a sensible approach, but also very difficult. This paper describes a new algorithm based on linear genetic programming which can be used to reverse engineer neural networks. The RODES algorithm automatically discovers the structure of the network, including neural connections, their signs and strengths, estimates its parameters, and can even be used to identify the biophysical mechanisms involved. The algorithm is tested on simulated time series data, generated using a realistic model of the subthalamopallidal network of basal ganglia. The resulting ODE system is highly accurate, and results are obtained in a matter of minutes. This is because the problem of reverse engineering a system of coupled differential equations is reduced to one of reverse engineering individual algebraic equations. The algorithm allows the incorporation of common domain knowledge to restrict the solution space. To our knowledge, this is the first time a realistic reverse engineering algorithm based on linear genetic programming has been applied to neural networks. [Copyright &y& Elsevier]