Search

Feedforward neural network (FNN) is the most popular network model, and the appropriate structure and learning algorithms are the key of its performance. This paper proposes an online self-organizing algorithm for feedforward neural network (OSNN) with a single hidden layer. The proposed OSNN optimizes the structure of FNN for time-varying system including structure design and parameter learning. In structure design, this paper measures the contribution ratios of hidden nodes by local sensitivity analysis based on differentiation method. OSNN merges hidden nodes with the others that have the highest correlation when their contribution ratios are almost zero and adds new hidden nodes by error reparation. For parameter learning, an improved online gradient method (OGM), called online gradient method with fixed memory (FMOGM), is proposed to improve the convergence speed and accuracy of OGM. In addition, this paper calculates the contribution ratios and the network error and estimates the local minima by using the fixed-sized training set of FMOGM instead of one sample at the current time, which can obtain more effective local information and a compact network structure. Finally, the proposed OSNN is verified using a number of benchmark problems and a practical problem for biochemical oxygen demand prediction in wastewater treatment. The experimental results show that OSNN has better convergence speed and accuracy than other algorithms. [ABSTRACT FROM AUTHOR]

This paper investigates how to construct a recurrent radial basis function neural network (RRBFNN) by an information-oriented algorithm (IOA) and how to adjust the parameters by a gradient algorithm simultaneously. In this IOA-based RRBFNN (IOA-RRBFNN), the proposed IOA is used to calculate the information processing strength (IPS) of hidden neurons, such that the independent component contributions between the hidden neurons and output neurons can be extracted. Then, a novel self-organizing strategy is proposed to optimize the structure of RRBFNN based on the input IPS and output IPS of hidden neurons. Meanwhile, a gradient algorithm is developed to update the parameters of IOA-RRBFNN. The proposed IOA-RRBFNN can be used to organize the network structure and adjust the parameters to improve its performance. Finally, several examples are presented to illustrate the effectiveness of IOA-RRBFNN. The results demonstrate that the proposed IOA-RRBFNN is more competitive in solving the nonlinear system modeling problems compared with some existing methods. [ABSTRACT FROM AUTHOR]

In this paper, an adaptive second order algorithm (ASOA) has been developed to train the fuzzy neural network (FNN) to achieve fast and robust convergence for nonlinear system modeling. Different from recent studies, this ASOA-based FNN (ASOA-FNN) has the quasi Hessian matrix and gradient vector which are accumulated as the sum of related sub matrices and vectors, respectively. Meanwhile, the learning rate of ASOA-FNN is designed to accelerate the learning speed. In addition, the convergence of the proposed ASOA-FNN has been proved both in the fixed learning rate phase and the adaptive learning rate phase. Finally, several comparisons have been realized and they have shown that the proposed ASOA-FNN has faster convergence speed and more accurate results than that of some existing methods. [ABSTRACT FROM AUTHOR]

Abstract: An approach, named extended extreme learning machine (ELM), is proposed for training the weights of a class of hierarchical feedforward neural network (HFNN). Unlike conventional single-hidden-layer feedforward networks (SLFNs), this hierarchical ELM (HELM) is based on the hierarchical structure which is capable of hierarchical learning of sequential information online, and one may simply choose hidden layers and then only need to adjust the output weights linking the hidden layer and the output layer. In such HELM implementations, the extended ELM provides better generalization performance during the learning process. Moreover, the proposed extended ELM method is efficient not only for HFNNs with sigmoid hidden nodes but also for HFNNs with radial basis function (RBF) hidden nodes. Finally, the HELM is applied to the activated sludge wastewater treatment processes (WWTPs) for predicting the water qualities. Experimental results and the performance comparison demonstrate the effectiveness of the proposed HELM. [Copyright &y& Elsevier]

Gradient-based algorithms are commonly used for training radial basis function neural network (RBFNN). However, it is still difficult to avoid vanishing gradient to improve the learning performance in the training process. For this reason, in this paper, an accelerated second-order learning (ASOL) algorithm is developed to train RBFNN. First, an adaptive expansion and pruning mechanism (AEPM) of gradient space, based on the integrity and orthogonality of hidden neurons, is designed. Then, the effective gradient information is constantly added to gradient space and the redundant gradient information is eliminated from gradient space. Second, with AEPM, the neurons are generated or pruned accordingly. In this way, a self-organizing RBFNN (SORBFNN) which reduces the structure complexity and improves the generalization ability is obtained. Then, the structure and parameters in the learning process can be optimized by the proposed ASOL-based SORBFNN (ASOL-SORBFNN). Third, some theoretical analyses including the efficiency of the proposed AEPM on avoiding the vanishing gradient and the stability of SORBFNN in the process of structural adjustment are given, then the successful application of the proposed ASOL-SORBFNN is guaranteed. Finally, to illustrate the advantages of the proposed ASOL-SORBFNN, several experimental studies are examined. By comparing with other existing approaches, the results show that ASOL-SORBFNN performs well in terms of both learning speed and prediction accuracy. [ABSTRACT FROM AUTHOR]