Search

Recently, the Extreme Learning Machine (ELM) algorithm has been applied to various fields due to its rapidity and significant generalization performance. Traditionally, deep learning (DL) and wavelet neural networks (WNN) methods reach a high classification accuracy in machine learning applications. As a result, a new structure based on WNN, deep architecture and ELM is proposed in this paper. The proposed method is based on Extreme Learning Machine Auto-Encoder with DL structure and a composite wavelet activation function used in the hidden nodes. To evaluate the performance of our approach, we used standard benchmark data-sets, namely COIL-20, Pima Indian Diabetes (PID), MNIST and EMNIST. Experimental results show that our method offers satisfactory results and performance compared to other approaches. [ABSTRACT FROM AUTHOR]

Sparse Bayesian learning (SBL) provides state-of-the-art performance in accuracy, sparsity and probabilistic prediction for classification. In SBL, the regularization priors are automatically determined that avoids an exhaustive hyperparameter selection by cross-validation. However, scalability to large problems is a drawback of SBL because of the inversion of a potentially enormous covariance matrix for updating the regularization priors in every iteration. This paper develops an approximate SBL algorithm called ARP-SBL, where the regularization priors are approximated without inverting the covariance matrix. Therefore, our approach can easily scale up to problems with large data size or feature dimension. It alleviates the long training time and high memory complexity in SBL. Based on ARP-SBL, two scalable nonlinear SBL models: scalable relevance vector machine (ARP-RVM) and scalable sparse Bayesian extreme learning machine (ARP-SBELM) are developed for problems of large data size and large feature size respectively. Experiments on a variety of benchmarks have shown that the proposed models are with competitive accuracy compared to existing methods while i) converging faster; ii) requiring thousands of times less memory; iii) without exhaustive regularized hyperparameter selection; and iv) easily scaling up to large data size and high dimensional features. [ABSTRACT FROM AUTHOR]

A novel sparse tensor dictionary learning algorithm and a convolutional neural network (CNN) classification method based on this algorithm are proposed for hyperspectral image (HSI) in this paper. The HSI is intuitively represented as a three-dimensional (3-D) cube, and utilizing the joint spatial-spectral information can significantly improve the accuracy of HSI classification. Therefore, the proposed atom-substituted tensor dictionary learning (ASTDL) algorithm utilizes tensor techniques to extract 3-D joint spatial-spectral features from HSI cubes directly. Sparsity constraint is enforced on the coefficient tensors, which obeys the sparsity attribute of HSI. The proposed ASTDL enhanced CNN (ASTDL-CNN) classification method utilizes a two-dimensional (2-D) CNN to extract deep features from the feature tensors obtained by the ASTDL, and to perform the pixel-wise classification. The use of ASTDL, which extracts intrinsic tensor features before CNN, alleviates within-class spatial-spectral variation and reduces the requirement of CNN for the labeled data. Whereas, the CNN works as a 3-D classifier, and provides the final classification results. Besides, we perform the majority vote on the classification map obtained by CNN to refine the classification. The performance of the proposed method is evaluated on three real HSI data sets. The competitive results to the compared state-of-the-art methods demonstrate that the ASTDL-CNN can provide accurate and robust classification results. [ABSTRACT FROM AUTHOR]

This paper investigates a new bacterial colony-based feature selection algorithm to improve the classification accuracy of customers for personalized products recommendation. An attribute learning strategy is developed in this study to update the feature related population. Specifically, the features can be weighted according to their historic contributions to both the individual- and group-based subsets. Additionally, the frequency of appearance is also recorded for the feature candidates to improve the diversity of feature distribution and avoid the over-fitting. Based on the weight-based feature indexes and frequency of appearance records, the performance of feature subsets are enhanced by replacing the features being repeatedly appeared in a same vector. To explore the feasibility of the proposed method for the missing feature problems, the objective of the optimization is to minimize the classification error using the acceptable number of features. K-Nearest Neighbor is employed as the learning technique to cooperate with the proposed feature selection method. The effectiveness of the proposed feature selection method is demonstrated by performing test on the datasets from UCI machine learning repository and real-world data from Amazon customer reviews of products. Compared with other seven feature selections methods, the proposed feature selection algorithm outperforms the other algorithms by achieving higher classification accuracy rate using smaller features. [ABSTRACT FROM AUTHOR]

Distance metric learning is a branch of machine learning that aims to learn distances from the data, which enhances the performance of similarity-based algorithms. This tutorial provides a theoretical background and foundations on this topic and a comprehensive experimental analysis of the most-known algorithms. We start by describing the distance metric learning problem and its main mathematical foundations, divided into three main blocks: convex analysis, matrix analysis and information theory. Then, we will describe a representative set of the most popular distance metric learning methods used in classification. All the algorithms studied in this paper will be evaluated with exhaustive testing in order to analyze their capabilities in standard classification problems, particularly considering dimensionality reduction and kernelization. The results, verified by Bayesian statistical tests, highlight a set of outstanding algorithms. Finally, we will discuss several potential future prospects and challenges in this field. This tutorial will serve as a starting point in the domain of distance metric learning from both a theoretical and practical perspective. [ABSTRACT FROM AUTHOR]

Sparse representation and dictionary learning have been successfully applied in hyperspectral image classification. Generally, it is more effective to learn the sub-dictionary for each class and utilize multiple scale strategy. However, the sub-dictionary may only consider the within-class information and ignore the discriminative information. For multiscale sparse representation, the shape of the regions would not adaptively change according to context structure. This paper proposes a discriminative sub-dictionary learning algorithm and an adaptive multiscale superpixel classification strategy under sparse representation framework for hyperspectral image classification. In dictionary learning stage, the global constraint term can ensure the learnt sub-dictionaries contain more discriminative information. In sparse representation stage, the adaptive multiscale superpixel strategy can exploit the spatial context according to the structural information. The combination of the two strategies can further explore the spatial and spectral information in hyperspectral image. Experiments on four hyperspectral image datasets prove that the proposed method has a significant improvement in both quantitative and qualitative analysis comparing with state-of-the-art algorithms. Moreover, the proposed algorithm can produce satisfactory results for the imbalanced data problem involved in hyperspectral image. [ABSTRACT FROM AUTHOR]

Many real-world machine learning tasks have very limited labeled data but a large amount of unlabeled data. To take advantage of the unlabeled data for enhancing learning performance, several semi-supervised learning techniques have been developed. In this paper, we propose a novel semi-supervised ensemble learning algorithm, termed Multi-Train, which generates a number of heterogeneous classifiers that use different classification models and/or different features. During the training process, each classifier is refined using unlabeled data, which are labeled by the majority prediction of the rest classifiers. We hypothesize that the use of different models and different input features can promote the diversity of the ensemble, thereby improving the performance compared to existing methods such as the co-training and tri-training algorithms. Experimental results on the UCI datasets clearly demonstrated the effectiveness of using heterogeneous ensembles in semi-supervised learning. [ABSTRACT FROM AUTHOR]

Abstract: This paper investigates the design of a linear-in-the-parameters (LITP) regression classifier for two-class problems. Most existing algorithms generally learn a classifier (model) from the available training data based on some stopping criterions, such as the Akaike''s final prediction error (FPE). The drawback here is that the classifier obtained is then not directly obtained based on its generalization capability. The main objective of this paper is to improve the sparsity and generalization capability of a classifier, while reducing the computational expense in producing it. This is achieved by proposing an automatic two-stage locally regularized classifier construction (TSLRCC) method using the extreme learning machine (ELM). In this new algorithm, the nonlinear parameters in each term, such as the width of the Gaussian function and the power of a polynomial term, are firstly determined by the ELM. An initial classifier is then generated by the direct evaluation of these candidates models according to the leave-one-out (LOO) misclassification rate in the first stage. The significance of each selected regressor term is also checked and insignificant ones are replaced in the second stage. To reduce the computational complexity, a proper regression context is defined which allows fast implementation of the proposed method. Simulation results confirm the effectiveness of the proposed technique. [Copyright &y& Elsevier]

The use of learning algorithms for text classification assumes the availability of a large amount of documents which have been organized and labeled correctly by human experts for use in the training phase. Unless the text documents in question have been in existence for some time, using an expert system is inevitable because manual organizing and labeling of thousands of groups of text documents can be a very labor intensive and intellectually challenging activity. Also, in some new domains, the knowledge to organize and label different classes might not be unavailable. Therefore unsupervised learning schemes for automatically clustering data in the training phase are needed. Furthermore, even when knowledge exists, variation is high when the subject under classification depends on personal opinions and is open to different interpretations. This paper describes a methodology which uses Self Organizing Maps (SOM) and alternatively does the automatic clustering by using the Correlation Coefficient (CorrCoef). Consequently the clusters are used as the labels to train the Support Vector Machine (SVM). Experiments and results are presented based on applying the methodology to some standard text datasets in order to verify the accuracy of the proposed scheme. We will also present results which are used to evaluate the effect that dimensionality reduction and changes in the clustering schemes have on the accuracy of the SVM. Results show that the proposed combination has better accuracy compared to training the learning machine using the expert knowledge. [ABSTRACT FROM AUTHOR]

Abstract: Self-adaptation is an inherent part of any natural and intelligent system. Specifically, it is about the ability of a system to reconcile its requirements or goal of existence with the environment it is interacting with, by adopting an optimal behavior. Self-adaptation becomes crucial when the environment changes dynamically over time. In this paper, we investigate self-adaptation of classification systems at three levels: (1) natural adaptation of the base learners to change in the environment, (2) contributive adaptation when combining the base learners in an ensemble, and (3) structural adaptation of the combination as a form of dynamic ensemble. The present study focuses on neural network classification systems to handle a special facet of self-adaptation, that is, incremental learning (IL). With IL, the system self-adjusts to accommodate new and possibly non-stationary data samples arriving over time. The paper discusses various IL algorithms and shows how the three adaptation levels are inherent in the system''s architecture proposed and how this architecture is efficient in dealing with dynamic change in the presence of various types of data drift when applying these IL algorithms. [Copyright &y& Elsevier]

Cancer is curable if it can be detected early. One way to detect cancer is by analyzing the change in expression of genes in the suspected tissue. Serial analysis of gene expression (SAGE) is a sequencing technique used for measuring the expression levels of genes. Cancer detection problem can be posed as binary classification problem like whether a tissue is cancerous or normal. SAGE libraries contain expression levels of thousands of genes which are the features. It is impossible to consider all these features for classification and also the general feature selection algorithms are not efficient with this data. In this paper, closed frequent itemset mining is proposed as a feature selection technique for identifying a small set of features which can distinguish the two classes efficiently. The performance of the proposed technique is evaluated on SAGE data related to breast tissue and a group of 26 genes are selected as best features. Two well known classifiers, extreme learning machine (ELM) and support vector machine (SVM), are used to evaluate the effectiveness of the selected features in classification and found that the proposed method works well with these classifiers. [ABSTRACT FROM AUTHOR]

A new algorithm for ensemble construction based on adapted restricting a set of weights of examples in training data to avoid overfitting and to reduce a number of iterations is proposed in the paper. The algorithm called IDMBoost (Imprecise Dirichlet Model Boost) applies Walley׳s imprecise Dirichlet model for modifying the restricted sets of weights depending on the number and location of classification errors. Updating of weights within the restricted set (simplex) is carried out by using its extreme points. The proposed algorithm has a double adaptation procedure. The first adaptation is carried out within every restricted simplex like the AdaBoost. The second adaptation reduces and changes the restricted sets of possible weights of examples. Various numerical experiments with real data illustrate the proposed algorithm. [ABSTRACT FROM AUTHOR]

There has been some improvement in the treatment of preterm infants, which has helped to increase their chance of survival. However, the rate of premature births is still globally increasing. As a result, this group of infants is most at risk of developing severe medical conditions that can affect the respiratory, gastrointestinal, immune, central nervous, auditory and visual systems. There is a strong body of evidence emerging that suggests the analysis of uterine electrical signals, from the abdominal surface (Electrohysterography – EHG), could provide a viable way of diagnosing true labour and even predict preterm deliveries. This paper explores this idea further and presents a new dynamic self-organized network immune algorithm that classifies term and preterm records, using an open dataset containing 300 records (38 preterm and 262 term ). Using the dataset, oversampling and cross validation techniques are evaluated against other similar studies. The proposed approach shows an improvement on existing studies with 89% sensitivity , 91% specificity , 90% positive predicted value , 90% negative predicted value , and an overall accuracy of 90%. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, we propose a sequential learning algorithm for a neural network classifier based on human meta-cognitive learning principles. The network, referred to as Meta-cognitive Neural Network (McNN). McNN has two components, namely the cognitive component and the meta-cognitive component. A radial basis function network is the fundamental building block of the cognitive component. The meta-cognitive component controls the learning process in the cognitive component by deciding what-to-learn, when-to-learn and how-to-learn. When a sample is presented at the cognitive component of McNN, the meta-cognitive component chooses the best learning strategy for the sample using estimated class label, maximum hinge error, confidence of classifier and class-wise significance. Also sample overlapping conditions are considered in growth strategy for proper initialization of new hidden neurons. The performance of McNN classifier is evaluated using a set of benchmark classification problems from the UCI machine learning repository and two practical problems, viz., the acoustic emission for signal classification and a mammogram data set for cancer classification. The statistical comparison clearly indicates the superior performance of McNN over reported results in the literature. [Copyright &y& Elsevier]

Abstract: In machine learning the so-called curse of dimensionality, pertinent to many classification algorithms, denotes the drastic increase in computational complexity and classification error with data having a great number of dimensions. In this context, feature selection techniques try to reduce dimensionality finding a new more compact representation of instances selecting the most informative features and removing redundant, irrelevant, and/or noisy features. In this paper, we propose a filter-based feature selection method for working in the multiple-instance learning scenario called ReliefF-MI; it is based on the principles of the well-known ReliefF algorithm. Different extensions are designed and implemented and their performance checked in multiple instance learning. ReliefF-MI is applied as a pre-processing step that is completely independent from the multi-instance classifier learning process and therefore is more efficient and generic than wrapper approaches proposed in this area. Experimental results on five benchmark real-world data sets and 17 classification algorithms confirm the utility and efficiency of this method, both statistically and from the point of view of execution time. [Copyright &y& Elsevier]

Abstract: This paper presents a distributed system for the recognition of human actions using views of the scene grabbed by different cameras. 2D frame descriptors are extracted for each available view to capture the variability in human motion. These descriptors are projected into a lower dimensional space and fed into a probabilistic classifier to output a posterior distribution of the action performed according to the descriptor computed at each camera. Classifier fusion algorithms are then used to merge the posterior distributions into a single distribution. The generated single posterior distribution is fed into a sequence classifier to make the final decision on the performed activity. The system can instantiate different algorithms for the different tasks, as the interfaces between modules are clearly defined. Results on the classification of the actions in the IXMAS dataset are reported. The accuracy of the proposed system is similar to state-of-the-art 3D methods, even though it uses only well-known 2D pattern recognition techniques and does not need to project the data into a 3D space or require camera calibration parameters. [Copyright &y& Elsevier]

Abstract: Although kernel minimum squared error (KMSE) is computationally simple, i.e., it only needs solving a linear equation set, it suffers from the drawback that in the testing phase the computational efficiency decreases seriously as the training samples increase. The underlying reason is that the solution of Naïve KMSE is represented by all the training samples in the feature space. Hence, in this paper, a method of selecting significant nodes for KMSE is proposed. During each calculation round, the presented algorithm prunes the training sample making least contribution to the objective function, hence called as PLOC-KMSE. To accelerate the training procedure, a batch of so-called nonsignificant nodes is pruned instead of one by one in PLOC-KMSE, and this speedup algorithm is named MPLOC-KMSE for short. To show the efficacy and feasibility of the proposed PLOC-KMSE and MPLOC-KMSE, the experiments on benchmark data sets and real-world instances are reported. The experimental results demonstrate that PLOC-KMSE and MPLOC-KMSE require the fewest significant nodes compared with other algorithms. That is to say, their computational efficiency in the testing phase is best, thus suitable for environments having a strict demand of computational efficiency. In addition, from the performed experiments, it is easily known that the proposed MPLOC-KMSE accelerates the training procedure without sacrificing the computational efficiency of testing phase to reach the almost same generalization performance. Finally, although PLOC and MPLOC are proposed in regression domain, they can be easily extended to classification problem and other algorithms such as kernel ridge regression. [Copyright &y& Elsevier]

Abstract: To conduct binary classification with highly imbalanced data is a very common problem, especially when the examples of interest are relatively rare. In this paper, we proposed the “Meta Imbalanced Classification Ensemble (MICE)” algorithm in order to dilute the effect of imbalanced data. In the MICE, the majority group is partitioned based on the transformed features from “inner product” to retain the geometric relation between two groups. The empirical results show that the performance of MICE is better than some renowned classification methods in terms of the specificity and the sensitivity. [Copyright &y& Elsevier]

Abstract: Feature selection plays an important role in pattern classification. Its purpose is to remove redundant features from data set as many as possible. The presence of useless features may not only deteriorate the performance of learning algorithms, but also obscure important information (e.g., intrinsic structure) behind data. Along with new and emerging techniques, data sets in many domains are becoming larger and larger and many irrelevant features are often prevailing in these data sets. This, however, poses great challenges to traditional learning algorithms, such as low efficiency and over-fitting. Thus, it becomes apparent that an efficient technique is needed to eliminate redundant or irrelevant features from the data sets. Currently, many endeavors to cope with this problem have been attempted and various outstanding feature selection methods have been proposed. Unlike other selection methods, in this paper we propose a general scheme of boosting feature selection method using information metric. The primary characteristic of our method is that it exploits weight of data to select salient features. Furthermore, the weight of data will be dynamically changed after each candidate feature has been selected. Thus, the information criteria used in feature selector can exactly represent the relevant degree between features and the class labels. As a result, the selected feature subset has maximal relevance to the class labels. Simulation studies carried out on UCI data sets show that the classification performance achieved by our proposed method is better than those of other selection methods in most cases. [Copyright &y& Elsevier]

Abstract: This paper proposes a hybrid neural network model using a possible combination of different transfer projection functions (sigmoidal unit, SU, product unit, PU) and kernel functions (radial basis function, RBF) in the hidden layer of a feed-forward neural network. An evolutionary algorithm is adapted to this model and applied for learning the architecture, weights and node typology. Three different combined basis function models are proposed with all the different pairs that can be obtained with SU, PU and RBF nodes: product–sigmoidal unit (PSU) neural networks, product–radial basis function (PRBF) neural networks, and sigmoidal–radial basis function (SRBF) neural networks; and these are compared to the corresponding pure models: product unit neural network (PUNN), multilayer perceptron (MLP) and the RBF neural network. The proposals are tested using ten benchmark classification problems from well known machine learning problems. Combined functions using projection and kernel functions are found to be better than pure basis functions for the task of classification in several datasets. [Copyright &y& Elsevier]