Search

Extreme learning machine (ELM) has recently attracted many researchers' interest due to its very fast learning speed, good generalization ability, and ease of implementation. It provides a unified solution that can be used directly to solve regression, binary, and multiclass classification problems. In this paper, we propose a stacked ELMs (S-ELMs) that is specially designed for solving large and complex data problems. The S-ELMs divides a single large ELM network into multiple stacked small ELMs which are serially connected. The S-ELMs can approximate a very large ELM network with small memory requirement. To further improve the testing accuracy on big data problems, the ELM autoencoder can be implemented during each iteration of the S-ELMs algorithm. The simulation results show that the S-ELMs even with random hidden nodes can achieve similar testing accuracy to support vector machine (SVM) while having low memory requirements. With the help of ELM autoencoder, the S-ELMs can achieve much better testing accuracy than SVM and slightly better accuracy than deep belief network (DBN) with much faster training speed.

Extreme learning machines (ELMs) have been proposed for generalized single-hidden-layer feedforward networks which need not be neuron alike and perform well in both regression and classification applications. The problem of determining the suitable network architectures is recognized to be crucial in the successful application of ELMs. This paper first proposes a dynamic ELM (D-ELM) where the hidden nodes can be recruited or deleted dynamically according to their significance to network performance, so that not only the parameters can be adjusted but also the architecture can be self-adapted simultaneously. Then, this paper proves in theory that such D-ELM using Lebesgue p-integrable hidden activation functions can approximate any Lebesgue p-integrable function on a compact input set. Simulation results obtained over various test problems demonstrate and verify that the proposed D-ELM does a good job reducing the network size while preserving good generalization performance.

This brief presents a curve clustering technique based on a new multivariate model. Instead of the usual Gaussian random effect model, our method uses the multivariate t-distribution model which has better robustness to outliers and noise. In our method, we use the B-spline curve to model curve data and apply the mixed-effects model to capture the randomness and covariance of all curves within the same cluster. After fitting the B-spline-based mixed-effects model to the proposed multivariate t -distribution, we derive an expectation-maximization algorithm for estimating the parameters of the model, and apply the proposed approach to the simulated data and the real dataset. The experimental results show that our model yields better clustering results when compared to the conventional Gaussian random effect model.

For a recurrent neural network (RNN), its transient response is a critical issue, especially for real-time signal processing applications. The conventional RNN training algorithms, such as backpropagation through time (BPTT) and real-time recurrent learning (RTRL), have not adequately addressed this problem because they suffer from low convergence speed. While increasing the learning rate may help to improve the performance of the RNN, it can result in unstable training in terms of weight divergence. Therefore, an optimal tradeoff between RNN training speed and weight convergence is desired. In this paper, a robust adaptive gradient-descent (RAGD) training algorithm of RNN is developed based on a novel RNN hybrid training concept. It switches the training patterns between standard real-time online backpropagation (BP) and RTRL according to the derived convergence and stability conditions. The weight convergence and L(2)-stability of the algorithm are derived via the conic sector theorem. The optimized adaptive learning maximizes the training speed of the RNN for each weight update without violating the stability and convergence criteria. Computer simulations are carried out to demonstrate the applicability of the theoretical results.

This paper considers the design of robust neural network tracking controllers for nonlinear systems. The neural network is used in the closed-loop system to estimate the nonlinear system function. We introduce the conic sector theory to establish a robust neural control system, with guaranteed boundedness for both the input/output (I/O) signals and the weights of the neural network. The neural network is trained by the simultaneous perturbation stochastic approximation (SPSA) method instead of the standard backpropagation (BP) algorithm. The proposed neural control system guarantees closed-loop stability of the estimation system, and a good tracking performance. The performance improvement of the proposed system over existing systems can be quantified in terms of preventing weight shifts, fast convergence, and robustness against system disturbance.

We present a multiple compartment, mammillary distributed-parameter model for capillary-tissue exchange, which can be implemented with dynamic contrast-enhanced imaging to study kinetic heterogeneity in tumors. The proposed n-compartment model consists of a vascular distributed-parameter compartment in direct exchange with a number (n - 1) of interstitial compartments. It is applied to a prostate tumor case study to illustrate the possible co-existence of two kinetically distinct compartments in the tumor, and the estimation of useful physiological parameters (such as perfusion, mean transit time, fractional volumes, and transfer and rate constants) associated with tissue microcirculation. The present model exhibits the convenient property of a separable impulse residue response function in time domain, which can be used to provide further insights and understanding on the physiological basis of tissue enhancement parameters commonly used for correlation studies with tumor histological diagnosis.

A robust backpropagation training algorithm with a dead zone scheme is used for the online tuning of the neuralnetwork (NN) tracking control system. This assures the convergence of the multilayered NN in the presence of disturbance. It is proved in this paper that the selection of a smaller range of the dead zone leads to a smaller estimate error of the NN, and hence a smaller tracking error of the NN tracking controller. The proposed algorithm is applied to a three-layered network with adjustable weights and a complete convergence proof is provided. The results can also be extended to the network with more hidden layers.