Search

Recently, Extreme learning machine (ELM), an efficient training algorithm for single-hidden-layer feedforward neural networks (SLFN), has gained increasing popularity in machine learning communities. In this paper the ELM based Area Under the ROC Curve (AUC) optimization algorithms are studied so as to further improve the performance of ELM for imbalanced datasets. For binary class problems, a novel ELM algorithm is proposed based on an efficient least square method. For multi-class problems, the following works are done in this paper: First of all, theoretical comparison analysis is proposed for the potential multi-class extensions of AUC; Secondly, a unified objective function for multi-class AUC optimization is proposed following the theoretical analysis; Subsequently, two ELM based multi-class AUC optimization algorithms called E L M M A U C and E L M m a c r o A U C respectively are proposed followed with complexity analyses; Finally, the generalization analysis is established for E L M M A U C in search of theoretical supports. Empirical study on a variety of real-world datasets show the effectiveness of our proposed algorithms. [ABSTRACT FROM AUTHOR]

In a semi-supervised domain adaptation (DA) task, one has access to only few labeled target examples. In this case, the success of DA needs the effective utilization of a large number of unlabeled target data to extract more discriminative information that is useful for generalization. To this end, we exploit in this paper the feature space embeddings of the target data as well as multi-source prior models to augment the discrimination space for the target function learning. Therefore, we propose a novel multi-source adaptation learning framework based on Sparse Feature Space Representation (SFSR), or called SFSR-MSAL for short. Specifically, the SFSR algorithm is first presented for the further construction of robust graph, on which the discriminative information can be smoothly propagated into the unlabeled target data by additionally incorporating the geometric structure of the target data. Considering the robustness in the semi-supervised DA, we replace the traditional l 2 -norm based least squares regression with the l 2, 1 -norm sparse regression, and then construct the SFSR-graph based semi-supervised DA framework with multi-source adaptation constraints. Our framework is universal and can be easily degraded into semi-supervised learning by just tuning the regularization parameter. Moreover, to select the discriminative SFSR-graph Laplacians, we also introduce the ensemble SFSR-graph Laplacians regularization into SFSR-MSAL, thus further improving the performance of SFSR-MSAL. The validity of our methods including semi-supervised and DA learning are examined by several visual recognition tasks on some benchmark datasets, which demonstrate the superiority of our methods in comparison with other related state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

Abstract: In reinforcement learning, when samples are limited in some real applications, Least-Squares Temporal Difference (LSTD) learning is prone to over-fitting, which can be overcome by the introduction of regularization. However, the solution of LSTD with regularization still depends on costly matrix inversion operations. In this paper we investigate the L2-norm regularized LSTD learning and propose an efficient algorithm to avoid expensive computational cost. We derive LSTD using Bellman operator along with projection operator. The L2-norm penalty is introduced to avoid over-fitting. We also describe the difference between Bellman residual minimization and LSTD. Then we propose an efficient recursive least-squares algorithm for L2-norm regularized LSTD, which can eliminate matrix inversion operations and decrease computational complexity effectively. We present empirical comparisons on the Boyan chain problem. The results show that the performance of the new algorithm is better than that of regularized LSTD. [Copyright &y& Elsevier]

Abstract: In this paper, we introduce a new category of fuzzy models called a fuzzy ensemble of parallel polynomial neural network (FEP2N2), which consist of a series of polynomial neural networks weighted by activation levels of information granules formed with the use of fuzzy clustering. The two underlying design mechanisms of the proposed networks rely on information granules resulting from the use of fuzzy C-means clustering (FCM) and take advantage of polynomial neural networks (PNNs). The resulting model comes in the form of parallel polynomial neural networks. In the design procedure, in order to estimate the optimal values of the coefficients of polynomial neural networks we use a weighted least square estimation algorithm. We incorporate various types of structures as the consequent part of the fuzzy model when using the learning algorithm. Among the diverse structures being available, we consider polynomial neural networks, which exhibit highly nonlinear characteristics when being viewed as local learning models. We use FCM to form information granules and to overcome the high dimensionality problem. We adopt PNNs to find the optimal local models, which can describe the relationship between the input variables and output variable within some local region of the input space. We show that the generalization capabilities as well as the approximation abilities of the proposed model are improved as a result of using polynomial neural networks. The performance of the network is quantified through experimentation in which we use a number of benchmarks already exploited within the realm of fuzzy or neurofuzzy modeling. [Copyright &y& Elsevier]

A central problem in machine learning and pattern recognition is the process of recognizing the most important features. In this paper, we provide a new feature selection method (DRPT) that consists of first removing the irrelevant features and then detecting correlations between the remaining features. Let D = A ∣ b be a dataset, where b is the class label and A is a matrix whose columns are the features. We solve A x = b using the least squares method and the pseudo-inverse of A. Each component of x can be viewed as an assigned weight to the corresponding column (feature). We define a threshold based on the local maxima of x and remove those features whose weights are smaller than the threshold. To detect the correlations in the reduced matrix, which we still call A , we consider a perturbation A ̃ of A. We prove that correlations are encoded in Δ x = ∣ x − x ̃ ∣ , where x ̃ is the least squares solution of A ̃ x ̃ = b. We cluster features first based on Δ x and then using the entropy of features. Finally, a feature is selected from each sub-cluster based on its weight and entropy. The effectiveness of DRPT has been verified by performing a series of comparisons with seven state-of-the-art feature selection methods over ten genetic datasets ranging up from 9,117 to 267,604 features. The results show that, over all, the performance of DRPT is favorable in several aspects compared to each feature selection algorithm. [ABSTRACT FROM AUTHOR]