Search

Recently models of neural networks that can deal with complex numbers, complex-valued neural networks, have been proposed and several studies on their abilities of information processing have been done. In this paper we investigate existence conditions of energy functions for a class of fully connected complex-valued neural networks and propose an energy function, analogous to those of real-valued Hopfield-type neural networks. It is also shown that, similar to the real-valued ones, the energy function enables us to analyze qualitative behaviors of the complex-valued neural networks. We present dynamic properties of the complex-valued neural networks obtained by qualitative analysis using the energy function. A synthesis method of complex-valued associative memories by utilizing the analysis results is also discussed.

Complex variable based backpropagation algorithm (CVBP) is a new development in neural networks (ANN). The new tool of approximation is designed to train complex-variable based neural networks (CNN) in which the weights, functions of activation are complex in nature. The CVBP also is developed over a quadratic error function (same as the backpropagation algorithm). The present paper explores the possibility of using different Error Functions and compares the performance of each of them (ANN and CNN over Error Functions) by applying to a surface classification problem.

The properties of the critical points caused by the hierarchical structure of complex-valued neural networks are investigated. If the loss function used is not regular as a complex function, the critical points caused by the hierarchical structure are all saddle points.

In this paper, we consider the parameter-estimation problem of time domain models in the form of ARX or NARX. Instead of Least-Square based methods and the Maximum Likelihood method, the parameters of the time domain model will be estimated from test data using feed-forward neural networks. In the applications of neural networks, commonly used activation functions for function approximation are nonlinear transfer functions (e.g., Log-sigmoid function, Hyperbolic tangent function and polynomial function) as well as linear transfer function. In our study, we investigate the equivalence of linear/nonlinear Autoregressive exogenous (ARX/NARX) models with feed-forward neural network models, in which the activation function of hidden neuron is one of these most frequently used functions. The coefficients of ARX/NARX models are given by neural network weight values for different activation functions. We show that for pure linear systems, ARX models can be obtained from feed-forward neural networks with commonly used hidden neuron activation functions such as log-sigmoid function, hyperbolic tangent function, polynomial function, exponential function as well as pure linear function. For nonlinear systems, ARX and NARX models can be obtained from neural networks with polynomial or exponential hidden neuron activation functions to describe the linear contribution and nonlinear contribution respectively.

The approximation capabilities of an adaptive higher-order neural network (AHONN) with a neuron-adaptive activation function (NAF) to any nonlinear continuous functional and any nonlinear continuous operator are studied. Universal approximation theorems of AHONN to continuous functionals and continuous operators are given, and learning algorithms are derived to tune the free parameters in NAF as well as connection weights between neurons. We apply the algorithms to approximate continuous dynamical systems (operators).

This paper presents a digital VLSI implementation of a feedforward neural network classifier based on the saturating linear activation function. The architecture consists of one-hidden layer performing the weighted sum followed by a saturating linear activation function. The hardware implementation of such a network presents a significant advantage in terms of circuit complexity as compared to a network based on a sigmoid activation function, but without compromising the classification performance. Simulation results on two benchmark problems show that feedforward neural networks with the saturating linearity perform as well as networks with the sigmoid activation function. The architecture can also handle variable precision resulting in a higher computational resources at lower precision.

A new support vector regression (SVR) algorithm based on the statistical method is proposed to select an initial structure for the certain neural networks and fuzzy neural networks. There are repeated SVR methods to use in the proposed method. Because the SVR approach is equivalent to solve a linear constrained quadratic programming problem the number of hidden nodes and adjustable parameters in the proposed networks are easily obtained. Based on the better initial structure for the proposed networks, the convergence epoch of the proposed networks are faster than the conventional neural networks and fuzzy neural networks with BP learning algorithm and robust BP learning algorithm. Simulation results are provided to show the validity and applicability of the proposed method.

In this paper, a multi-branch structure of neural networks is studied to make their size compact. The multi-branch structure has shown improved performance against conventional neural networks. As a result, it has been proved that the number of nodes of networks and the computational cost for training networks can be reduced. In addition, it could be said that proposed multi-branch networks are special cases of higher order neural networks, however, they obtain higher order effect easier without suffering the parameter explosion problem.

A novel neural network model, the procedure neural network (PNN) is proposed, in which input term is associated with a procedure. Two forms of procedure neural networks are constructed. One is procedure neural network expanded on certain base functions, the other is procedure neural network based on projective combination, and they are equivalent to each other in structure. For the later procedure neural networks the continuity theorem, continuous functional approximation theorem and computing capability theorem are presented. Selection strategies of base functions and time aggregations are specially discussed. Supervised learning algorithm for training of procedure neural networks is provided. Finally, an application example, which is adaptive to the case of procedure neural networks, is simulated.

Based on the wavelet and multiwavelet theory, some new notions of wavelet neural networks and multiwavelet neural networks have been proposed as an alternative to feedforward neural networks for approximating arbitrary nonlinear functions by some scholars recently. In this paper we give a comparative research on (multi-)wavelet neural networks,especially emphasized on their mathematical basis,the systematic synthesis, and the efficacy analysis,such as the "curse of dimensionality". Some experiments are given to verify the efficiency of wavelet neural networks.