Search

Abstract: Extreme learning machine (ELM) is one of the most popular and important learning algorithms. It comes from single-hidden-layer feedforward neural networks. It has been proved that ELM can achieve better performance than support vector machine (SVM) in regression and classification. In this paper, mathematically, with regression problem, the step 3 of ELM is studied. First of all, the equation are reformulated as an optimal model. With the optimality, the necessary conditions of optimal solution are presented. The equation is replaced by . We can prove that the latter must have one solution at least. Second, optimal approximation solution is discussed in cases of is column full rank, row full rank, neither column nor row full rank. In the last case, the rank-1 and rank-2 methods are used to get optimal approximation solution. In theory, this paper present a better algorithm for ELM. [Copyright &y& Elsevier]

Abstract: Extreme learning machine (ELM) proposed by Huang et al. was developed for generalized single hidden layer feedforward networks (SLFNs) with a wide variety of hidden nodes. It proved to be very fast and effective especially for solving function approximation problems with a predetermined network structure. However, the method for determining the network structure of preliminary ELM may be tedious and may not lead to a parsimonious solution. In this paper, a systematic two-stage algorithm (named TS-ELM) is introduced to handle the problem. In the first stage, a forward recursive algorithm is applied to select the hidden nodes from the candidates randomly generated in each step and add them to the network until the stopping criterion achieves its minimum. The significance of each hidden node is then reviewed in the second stage and the insignificance ones are removed from the network, which drastically reduces the network complexity. The effectiveness of TS-ELM is verified by the empirical studies in this paper. [ABSTRACT FROM AUTHOR]

Abstract: Low-rank representations have received a lot of interest in the application of kernel-based methods. However, these methods made an assumption that the spectrum of the Gaussian or polynomial kernels decays rapidly. This is not always true and its violation may result in performance degradation. In this paper, we propose an effective technique for learning low-rank Mercer kernels (LMK) with fast-decaying spectrum. What distinguishes our kernels from other classical kernels (Gaussian and polynomial kernels) is that the proposed always yields low-rank Gram matrices whose spectrum decays rapidly, no matter what distribution the data are. Furthermore, the LMK can control the decay rate. Thus, our kernels can prevent performance degradation while using the low-rank approximations. Our algorithm has favorable in scalability—it is linear in the number of data points and quadratic in the rank of the Gram matrix. Empirical results demonstrate that the proposed method learns fast-decaying spectrum and significantly improves the performance. [Copyright &y& Elsevier]

Abstract: In this paper a new learning algorithm is proposed for the problem of simultaneous learning of a function and its derivatives as an extension of the study of error minimized extreme learning machine for single hidden layer feedforward neural networks. Our formulation leads to solving a system of linear equations and its solution is obtained by Moore–Penrose generalized pseudo-inverse. In this approach the number of hidden nodes is automatically determined by repeatedly adding new hidden nodes to the network either one by one or group by group and updating the output weights incrementally in an efficient manner until the network output error is less than the given expected learning accuracy. For the verification of the efficiency of the proposed method a number of interesting examples are considered and the results obtained with the proposed method are compared with that of other two popular methods. It is observed that the proposed method is fast and produces similar or better generalization performance on the test data. [Copyright &y& Elsevier]

Abstract: Image deblurring is a basic and important task of image processing. Traditional filtering based image deblurring methods, e.g. enhancement filters, partial differential equation (PDE) and etc., are limited by the hypothesis that natural images and noise are with low and high frequency terms, respectively. Noise removal and edge protection are always the dilemma for traditional models. In this paper, we study image deblurring problem from a brand new perspective—classification. And we also generalize the traditional PDE model to a more general case, using the theories of calculus of variations. Furthermore, inspired by the theories of approximation of functions, we transform the operator-learning problem into a coefficient-learning problem by means of selecting a group of basis, and build a filter-learning model. Based on extreme learning machine (ELM) , an algorithm is designed and a group of filters are learned effectively. Then a generalized image deblurring model, learned filtering PDE (LF-PDE), is built. The experiments verify the effectiveness of our models and the corresponding learned filters. It is shown that our model can overcome many drawbacks of the traditional models and achieve much better results. [Copyright &y& Elsevier]

Abstract: Variational Bayes learning or mean field approximation is widely used in statistical models which are made of mixtures of exponential distributions, for example, normal mixtures, binomial mixtures, and hidden Markov models. To derive variational Bayes learning algorithm, we need to determine the hyperparameters in the a priori distribution; however, the design method of hyperparameters has not yet been established. In the present paper, we propose two different design methods of hyperparameters which are applied to the different purposes. In the former method, the hyperparameter is determined for minimization of the generalization error. In the latter method, it is chosen so that candidates of hidden structure in training data are extracted. It is experimentally shown that the optimal hyperparameters for two purposes are different from each other. [Copyright &y& Elsevier]

Abstract: In this paper, a robust wavelet-based adaptive neural control (RWANC) with a PI type learning algorithm is proposed. The proposed RWANC system is composed of a wavelet neural controller and a fuzzy compensation controller. The wavelet neural control is utilized to approximate an ideal controller and the fuzzy compensation controller with a fuzzy logic system in it is used to remove the chattering phenomena of conventional sliding-mode control completely. In the RWANC, the learning algorithm is derived based on the Lyapunov function, thus the closed-loop system''s stability can be guaranteed. The chaotic system control has become an emerging topic in engineering community since the uncontrolled system displays complex, noisy-like and unpredictable behavior. Therefore, the proposed RWANC approach is applied to a second-order chaotic nonlinear system to investigate the effectiveness. Through the simulation results, the proposed RWANC scheme can achieve favorable tracking performance and the convergence of the tracking error and control parameters can be accelerated by the developed PI adaptation learning algorithm. [Copyright &y& Elsevier]

Abstract: In a fully complex-valued feed-forward network, the convergence of the Complex-valued Back Propagation (CBP) learning algorithm depends on the choice of the activation function, learning sample distribution, minimization criterion, initial weights and the learning rate. The minimization criteria used in the existing versions of CBP learning algorithm in the literature do not approximate the phase of complex-valued output well in function approximation problems. The phase of a complex-valued output is critical in telecommunication and reconstruction and source localization problems in medical imaging applications. In this paper, the issues related to the convergence of complex-valued neural networks are clearly enumerated using a systematic sensitivity study on existing complex-valued neural networks. In addition, we also compare the performance of different types of split complex-valued neural networks. From the observations in the sensitivity analysis, we propose a new CBP learning algorithm with logarithmic performance index for a complex-valued neural network with exponential activation function. The proposed CBP learning algorithm directly minimizes both the magnitude and phase errors and also provides better convergence characteristics. Performance of the proposed scheme is evaluated using two synthetic complex-valued function approximation problems, the complex XOR problem, and a non-minimum phase equalization problem. Also, a comparative analysis on the convergence of the existing fully complex and split complex networks is presented. [Copyright &y& Elsevier]

Abstract: Existing multi-view learning focuses on the problem of how to learn from data represented by multiple independent sets of attributes (termed as multi-view data), and has been proved to bring an excellent performance. However, in general, we have only a single set of attributes (termed as single-view data) available. The goal of this paper is to employ the multi-view viewpoint to develop a multi-view kernel machine for such a single-view data. The key of doing so is to associate each learning machine with one kernel, take it as one view and thus form a set of learning machines from their corresponding kernels, as a result, a multi-view kernel machine can be developed by synthesizing them into a single learning framework. Further, in the two-view (two-kernel) case, we explore the relationship between the generalization ability of the proposed kernel machine and its associated kernels, in which with the kernel alignment (KA) as a correlation measure between kernels, it is found that superior performance of the proposed machine results from a weaker correlation between the constitutive kernels. To the best of our knowledge, both the multi-view learning on single-view data and the KA measure used here have not appeared in any literature. In practice, we take the kernel modified Ho–Kashyap with squared (KMHKS) approximation of the misclassification errors as a learning machine to develop a multi-view KMHKS (MultiV-KMHKS) on single-view data. [Copyright &y& Elsevier]

Abstract: Recently an incremental algorithm referred to as incremental extreme learning machine (I-ELM) was proposed by Huang et al. [G.-B. Huang, L. Chen, C.-K. Siew, Universal approximation using incremental constructive feedforward networks with random hidden nodes, IEEE Trans. Neural Networks 17(4) (2006) 879–892], which randomly generates hidden nodes and then analytically determines the output weights. Huang et al. [G.-B. Huang, L. Chen, C.-K. Siew, Universal approximation using incremental constructive feedforward networks with random hidden nodes, IEEE Trans. Neural Networks 17(4) (2006) 879–892] have proved in theory that although additive or RBF hidden nodes are generated randomly the network constructed by I-ELM can work as a universal approximator. During our recent study, it is found that some of the hidden nodes in such networks may play a very minor role in the network output and thus may eventually increase the network complexity. In order to avoid this issue and to obtain a more compact network architecture, this paper proposes an enhanced method for I-ELM (referred to as EI-ELM). At each learning step, several hidden nodes are randomly generated and among them the hidden node leading to the largest residual error decreasing will be added to the existing network and the output weight of the network will be calculated in a same simple way as in the original I-ELM. Generally speaking, the proposed enhanced I-ELM works for the widespread type of piecewise continuous hidden nodes. [Copyright &y& Elsevier]