Search

• An analysis of lexicographic optimization for feature selection and data classification. • Cooperative co-evolution for feature selection while the classifier hyperparameters are also being optimized. • Optimization of multiple objectives using a simple evolutionary algorithm. • A new lexicographic ranking methodology to compare the results of different feature selection methods. • Quite good results for real high-dimensional datasets. This paper starts with two hypotheses. The first one is that the simultaneous optimization of the hyperparameters regulating the classifier within a wrapper method, while the best subset of features is being determined, should improve the results with respect to those obtained with a pre-parameterized classifier. The second one is that solving these two problems can be formulated as a lexicographic optimization problem, allowing the use of a simple single-objective evolutionary algorithm to solve this multi-objective problem. The fitness function is of key importance for such wrapper methods. It is responsible for guiding the search towards potentially good solutions and it also consumes most of the runtime. Having these issues in mind, this paper also proposes a new lexicographic fitness function, designed to minimize the runtime of the algorithm and also to avoid over-fitting. Furthermore, the execution time and the quality of the results obtained by the wrapper procedure also depend on some algorithmic hyperparameters: the similarity thresholds used when comparing two different solutions lexicographically and the percentage of data samples used for validation during the training process. Thus, an experimental analysis has been carried out to find adequate values for these hyperparameters. Finally, the lexicographic cooperative co-evolutionary wrapper approach, using the new fitness function proposed in this paper, has been tested with several datasets belonging to the University of California, Irvine (UCI) repository and also with some real high-dimensional datasets, obtaining quite good results, compared to other state-of-the-art wrapper methods. The comparison has also been made lexicographically, with a new methodology proposed in this paper. [ABSTRACT FROM AUTHOR]

In this paper, we study the decentralized AdaBoost problem over sensor networks, and propose a fully decentralized AdaBoost algorithm, where each sensor can obtain the centralized global solution without transmission of private dataset. By decomposing the centralized cost function into a summation of local ones, we convert decentralized AdaBoost problem into a distributed optimization problem, and design a distributed alternating minimization method to solve it. In order to improve convergence rate, motivated by Nesterov gradient descent method, we propose a fast decentralized AdaBoost algorithm. Then, we prove the convergence of proposed algorithms. Moreover, we deduce decentralized AdaBoost algorithm for logistic regression in detail. The simulations with Spam-Email dataset illustrate the effectiveness of proposed algorithms. [ABSTRACT FROM AUTHOR]

Recently, the Extreme Learning Machine (ELM) algorithm has been applied to various fields due to its rapidity and significant generalization performance. Traditionally, deep learning (DL) and wavelet neural networks (WNN) methods reach a high classification accuracy in machine learning applications. As a result, a new structure based on WNN, deep architecture and ELM is proposed in this paper. The proposed method is based on Extreme Learning Machine Auto-Encoder with DL structure and a composite wavelet activation function used in the hidden nodes. To evaluate the performance of our approach, we used standard benchmark data-sets, namely COIL-20, Pima Indian Diabetes (PID), MNIST and EMNIST. Experimental results show that our method offers satisfactory results and performance compared to other approaches. [ABSTRACT FROM AUTHOR]

• An algorithm that transforms long-tailed problem into hierarchical learning problem. • An effective knowledge transfer paradigm is proposed to help hierarchical learning. • The performance of our model outperforms the state-of-the-art methods. Deep visual recognition methods have achieved excellent performance on artificially constructed image datasets where the data distribution is balanced. However, in real-world scenarios, data distribution is usually extremely imbalanced and exhibit a long-tailed distribution where data in each head class is more than the class in the tail. Many efficient deep learning methods fail to work normally, i.e., they perform well in the head class while poor in the tail class. In this paper, we propose a two-layer Hierarchical-Learning Long-Tailed Recognition (HL-LTR) algorithm which transforms the long-tailed problem into a hierarchical classification problem by constructing a hierarchical superclass tree in which each layer corresponds to a recognition task. In the first layer of the tree, the degree of data imbalance is largely decreased. The recognition task of the second layer is the original long-tailed recognition problem. The training of HL-LTR is top-down. The knowledge learned by the first layer transfers to classes of the second layer and guides the feature learning of the second layer by using attention mechanism module and knowledge distillation method. Compared with directly solving the most difficult long-tailed recognition task, HL-LTR achieves better performance due to its progressive learning method from easy to difficult and effective knowledge transfer strategy. [ABSTRACT FROM AUTHOR]

Sparse Bayesian learning (SBL) provides state-of-the-art performance in accuracy, sparsity and probabilistic prediction for classification. In SBL, the regularization priors are automatically determined that avoids an exhaustive hyperparameter selection by cross-validation. However, scalability to large problems is a drawback of SBL because of the inversion of a potentially enormous covariance matrix for updating the regularization priors in every iteration. This paper develops an approximate SBL algorithm called ARP-SBL, where the regularization priors are approximated without inverting the covariance matrix. Therefore, our approach can easily scale up to problems with large data size or feature dimension. It alleviates the long training time and high memory complexity in SBL. Based on ARP-SBL, two scalable nonlinear SBL models: scalable relevance vector machine (ARP-RVM) and scalable sparse Bayesian extreme learning machine (ARP-SBELM) are developed for problems of large data size and large feature size respectively. Experiments on a variety of benchmarks have shown that the proposed models are with competitive accuracy compared to existing methods while i) converging faster; ii) requiring thousands of times less memory; iii) without exhaustive regularized hyperparameter selection; and iv) easily scaling up to large data size and high dimensional features. [ABSTRACT FROM AUTHOR]

A novel sparse tensor dictionary learning algorithm and a convolutional neural network (CNN) classification method based on this algorithm are proposed for hyperspectral image (HSI) in this paper. The HSI is intuitively represented as a three-dimensional (3-D) cube, and utilizing the joint spatial-spectral information can significantly improve the accuracy of HSI classification. Therefore, the proposed atom-substituted tensor dictionary learning (ASTDL) algorithm utilizes tensor techniques to extract 3-D joint spatial-spectral features from HSI cubes directly. Sparsity constraint is enforced on the coefficient tensors, which obeys the sparsity attribute of HSI. The proposed ASTDL enhanced CNN (ASTDL-CNN) classification method utilizes a two-dimensional (2-D) CNN to extract deep features from the feature tensors obtained by the ASTDL, and to perform the pixel-wise classification. The use of ASTDL, which extracts intrinsic tensor features before CNN, alleviates within-class spatial-spectral variation and reduces the requirement of CNN for the labeled data. Whereas, the CNN works as a 3-D classifier, and provides the final classification results. Besides, we perform the majority vote on the classification map obtained by CNN to refine the classification. The performance of the proposed method is evaluated on three real HSI data sets. The competitive results to the compared state-of-the-art methods demonstrate that the ASTDL-CNN can provide accurate and robust classification results. [ABSTRACT FROM AUTHOR]

This paper investigates a new bacterial colony-based feature selection algorithm to improve the classification accuracy of customers for personalized products recommendation. An attribute learning strategy is developed in this study to update the feature related population. Specifically, the features can be weighted according to their historic contributions to both the individual- and group-based subsets. Additionally, the frequency of appearance is also recorded for the feature candidates to improve the diversity of feature distribution and avoid the over-fitting. Based on the weight-based feature indexes and frequency of appearance records, the performance of feature subsets are enhanced by replacing the features being repeatedly appeared in a same vector. To explore the feasibility of the proposed method for the missing feature problems, the objective of the optimization is to minimize the classification error using the acceptable number of features. K-Nearest Neighbor is employed as the learning technique to cooperate with the proposed feature selection method. The effectiveness of the proposed feature selection method is demonstrated by performing test on the datasets from UCI machine learning repository and real-world data from Amazon customer reviews of products. Compared with other seven feature selections methods, the proposed feature selection algorithm outperforms the other algorithms by achieving higher classification accuracy rate using smaller features. [ABSTRACT FROM AUTHOR]

Distance metric learning is a branch of machine learning that aims to learn distances from the data, which enhances the performance of similarity-based algorithms. This tutorial provides a theoretical background and foundations on this topic and a comprehensive experimental analysis of the most-known algorithms. We start by describing the distance metric learning problem and its main mathematical foundations, divided into three main blocks: convex analysis, matrix analysis and information theory. Then, we will describe a representative set of the most popular distance metric learning methods used in classification. All the algorithms studied in this paper will be evaluated with exhaustive testing in order to analyze their capabilities in standard classification problems, particularly considering dimensionality reduction and kernelization. The results, verified by Bayesian statistical tests, highlight a set of outstanding algorithms. Finally, we will discuss several potential future prospects and challenges in this field. This tutorial will serve as a starting point in the domain of distance metric learning from both a theoretical and practical perspective. [ABSTRACT FROM AUTHOR]

A key issue in RGBD classification is how to fuse the RGB and depth modalities. A popular way is to extract the private features of unimodality and the common features between the two modalities. Most of them use low order algebraic metrics to find the common part of the RGB and depth signals. In this paper, adversarial network is used to learn the common features between the RGB and depth modalities. A modality discriminator is designed to compete with the feature encoder so as to generate the modality-invariant information, which can be regarded as the common features. With this concept, we present a M ulti- M odal F eature L earning algorithm with A dversarial N etwork (MMFLAN) to decouple the RGBD signals and obtain the fused features. Comprehensive experiments based on three datasets are used to evaluate the effectiveness and robustness of our MMFLAN. [ABSTRACT FROM AUTHOR]

Sparse representation and dictionary learning have been successfully applied in hyperspectral image classification. Generally, it is more effective to learn the sub-dictionary for each class and utilize multiple scale strategy. However, the sub-dictionary may only consider the within-class information and ignore the discriminative information. For multiscale sparse representation, the shape of the regions would not adaptively change according to context structure. This paper proposes a discriminative sub-dictionary learning algorithm and an adaptive multiscale superpixel classification strategy under sparse representation framework for hyperspectral image classification. In dictionary learning stage, the global constraint term can ensure the learnt sub-dictionaries contain more discriminative information. In sparse representation stage, the adaptive multiscale superpixel strategy can exploit the spatial context according to the structural information. The combination of the two strategies can further explore the spatial and spectral information in hyperspectral image. Experiments on four hyperspectral image datasets prove that the proposed method has a significant improvement in both quantitative and qualitative analysis comparing with state-of-the-art algorithms. Moreover, the proposed algorithm can produce satisfactory results for the imbalanced data problem involved in hyperspectral image. [ABSTRACT FROM AUTHOR]

Link prediction in a graph is the problem of detecting the missing links or the ones that would be formed in the near future. Using a graph representation of the data, we can convert the problem of classification to the problem of link prediction which aims at finding the missing links between the unlabeled data (unlabeled nodes) and their classes. To our knowledge, despite the fact that numerous algorithms use the graph representation of the data for classification, none are using link prediction as the heart of their classifying procedure. In this work, we propose a novel algorithm called CULP (C lassification U sing L ink P rediction) which uses a new structure namely Label Embedded Graph or LEG and a link predictor to find the class of the unlabeled data. Different link predictors along with Compatibility Score - a new link predictor we proposed that is designed specifically for our settings - has been used and showed promising results for classifying different datasets. This paper further improved CULP by designing an extension called CULM which uses a majority vote (hence the M in the acronym) procedure with weights proportional to the predictions' confidences to use the predictive power of multiple link predictors and also exploits the low level features of the data. Extensive experimental evaluations shows that both CULP and CULM are highly accurate and competitive with the cutting edge graph classifiers and general classifiers. [ABSTRACT FROM AUTHOR]

Drawing from memory the face of a friend you have not seen in years is a difficult task. However, if you happen to cross paths, you would easily recognize each other. The biological memory is equipped with an impressive compression algorithm that can store the essential, and then infer the details to match perception. The Willshaw Memory is a simple abstract model for cortical computations which implements mechanisms of biological memories. Using our recently proposed sparse coding prescription for visual patterns [34] , this model can store and retrieve an impressive amount of real-world data in a fault-tolerant manner. In this paper, we extend the capabilities of the basic Associative Memory Model by using a Multiple-Modality framework. In this setting, the memory stores several modalities (e.g., visual, or textual) of each pattern simultaneously. After training, the memory can be used to infer missing modalities when just a subset is perceived. Using a simple encoder-memory-decoder architecture, and a newly proposed iterative retrieval algorithm for the Willshaw Model, we perform experiments on the MNIST dataset. By storing both the images and labels as modalities, a single Memory can be used not only to retrieve and complete patterns but also to classify and generate new ones. We further discuss how this model could be used for other learning tasks, thus serving as a biologically-inspired framework for learning. [ABSTRACT FROM AUTHOR]

The probabilistic group nearest neighbor(PGNN) query , which returns all the uncertain objects whose probabilities of being the group nearest neighbor (GNN) results exceed a user-specified threshold, is widely used in uncertain database. Most existing work for answering PGNN queries adopted a general framework which consist of three phases: spatial pruning, probabilistic pruning, refinement . In the probabilistic pruning phase, dividing the uncertain regions into many partitions to derive a tighter probabilities bounds is a common method. However, there is a tradeoff between the computational cost of probabilistic pruning phase and refinement phase controlled by the granularity of the partitions. In this paper, we study the problem of setting the optimal granularity of the partitions for uncertain objects, and propose a new framework for PGNN queries based on granularity classification using ELM such that the overall cost is minimized. In addition, to improve the accuracy of classification and make the classifier applicable to the dynamic environment, a plurality voting method and a dynamic classification strategy are proposed respectively. Extensive experiments shows that compared with the default granularities of the partitions, the granularities chosen by ELM classifiers are more proper, which further improves the performance of PGNN query algorithm. In addition, ELM outperforms SVM with regard to both the response time and classification accuracy. [ABSTRACT FROM AUTHOR]

Many real-world machine learning tasks have very limited labeled data but a large amount of unlabeled data. To take advantage of the unlabeled data for enhancing learning performance, several semi-supervised learning techniques have been developed. In this paper, we propose a novel semi-supervised ensemble learning algorithm, termed Multi-Train, which generates a number of heterogeneous classifiers that use different classification models and/or different features. During the training process, each classifier is refined using unlabeled data, which are labeled by the majority prediction of the rest classifiers. We hypothesize that the use of different models and different input features can promote the diversity of the ensemble, thereby improving the performance compared to existing methods such as the co-training and tri-training algorithms. Experimental results on the UCI datasets clearly demonstrated the effectiveness of using heterogeneous ensembles in semi-supervised learning. [ABSTRACT FROM AUTHOR]

This paper presents a competitive co-evolutionary (ComCoE) that engages a double elimination tournament (DET) to evolve artificial neural networks (ANNs) for undertaking data classification problems. The proposed model performs a global search by a ComCoE approach to find near optimal solutions. During the global search process, two populations of different ANNs compete and fitness evaluation of each ANN is made in a subjective manner based on their participations throughout a DET which promotes competitive interactions among individual ANNs. The adaptation and fitness evaluation processes drive the global search for a more competent ANN classifier. A winning ANN is identified from the global search. Then, the Scaled Conjugate Backpropagation algorithm, which is a local search, is performed to further train the winning ANN to obtain a precise solution. The performance of the proposed classification model is evaluated rigorously; its performance is compared with the baseline ANNs of the proposed model as well as other classifiers. The results indicate that the proposed model could construct an ANN which could produce high classification accuracy rates with a compact network structure. [ABSTRACT FROM AUTHOR]

Abstract: This paper investigates the design of a linear-in-the-parameters (LITP) regression classifier for two-class problems. Most existing algorithms generally learn a classifier (model) from the available training data based on some stopping criterions, such as the Akaike''s final prediction error (FPE). The drawback here is that the classifier obtained is then not directly obtained based on its generalization capability. The main objective of this paper is to improve the sparsity and generalization capability of a classifier, while reducing the computational expense in producing it. This is achieved by proposing an automatic two-stage locally regularized classifier construction (TSLRCC) method using the extreme learning machine (ELM). In this new algorithm, the nonlinear parameters in each term, such as the width of the Gaussian function and the power of a polynomial term, are firstly determined by the ELM. An initial classifier is then generated by the direct evaluation of these candidates models according to the leave-one-out (LOO) misclassification rate in the first stage. The significance of each selected regressor term is also checked and insignificant ones are replaced in the second stage. To reduce the computational complexity, a proper regression context is defined which allows fast implementation of the proposed method. Simulation results confirm the effectiveness of the proposed technique. [Copyright &y& Elsevier]

It has been known that one of the important steps in training a complex-valued radial basis function neural network is to effectively determine its centers and widths of neurons in the hidden layer. In this paper, an improved maximum spread algorithm is propose to solve this issue. Its basic idea is that the choice of centers not only depends on the distances between samples from different classes, but also is heavily affected by the average distance between samples in the same class. The relationship between external and inner distances is taken into account when determining centers. The performance of this algorithm is tested on several datasets. It is shown that much better performance can be achieved by the developed algorithm than by some existing ones. [ABSTRACT FROM AUTHOR]

The use of learning algorithms for text classification assumes the availability of a large amount of documents which have been organized and labeled correctly by human experts for use in the training phase. Unless the text documents in question have been in existence for some time, using an expert system is inevitable because manual organizing and labeling of thousands of groups of text documents can be a very labor intensive and intellectually challenging activity. Also, in some new domains, the knowledge to organize and label different classes might not be unavailable. Therefore unsupervised learning schemes for automatically clustering data in the training phase are needed. Furthermore, even when knowledge exists, variation is high when the subject under classification depends on personal opinions and is open to different interpretations. This paper describes a methodology which uses Self Organizing Maps (SOM) and alternatively does the automatic clustering by using the Correlation Coefficient (CorrCoef). Consequently the clusters are used as the labels to train the Support Vector Machine (SVM). Experiments and results are presented based on applying the methodology to some standard text datasets in order to verify the accuracy of the proposed scheme. We will also present results which are used to evaluate the effect that dimensionality reduction and changes in the clustering schemes have on the accuracy of the SVM. Results show that the proposed combination has better accuracy compared to training the learning machine using the expert knowledge. [ABSTRACT FROM AUTHOR]

Abstract: Relations between objects in various systems, such as hyperlinks connecting web pages, citations of scientific papers, conversations via email or social interactions in Web 2.0 portals are commonly modeled by networks. One of many interesting problems currently studied for such domains is node classification. Due to the nature of the networked data and the unavailability of collection of nodes'' broad representation for training in majority of environments, only a very limited data may remain useful for classification. Therefore, there is a need for accurate and efficient algorithms that are able to perform good classification based only on scanty knowledge of network nodes. A new approach of sampling algorithm—LDGibbs, used in the context of collective classification with application of label-dependent features, is proposed in the paper in order to provide more accurate generalization for sparse datasets. Additionally, a new LDBootstrapping algorithm based on label-dependent features has been developed. Both new algorithms include additional steps to extract new input features based on graph structures but limited only to the nodes of a given label. It means that a separate set of structural features is provided for each label. The comparison with the other approaches, in particular with standard Gibbs Sampling and bootstrapping provided satisfactory results and revealed LDGibbs''s superiority. [Copyright &y& Elsevier]

Abstract: Self-adaptation is an inherent part of any natural and intelligent system. Specifically, it is about the ability of a system to reconcile its requirements or goal of existence with the environment it is interacting with, by adopting an optimal behavior. Self-adaptation becomes crucial when the environment changes dynamically over time. In this paper, we investigate self-adaptation of classification systems at three levels: (1) natural adaptation of the base learners to change in the environment, (2) contributive adaptation when combining the base learners in an ensemble, and (3) structural adaptation of the combination as a form of dynamic ensemble. The present study focuses on neural network classification systems to handle a special facet of self-adaptation, that is, incremental learning (IL). With IL, the system self-adjusts to accommodate new and possibly non-stationary data samples arriving over time. The paper discusses various IL algorithms and shows how the three adaptation levels are inherent in the system''s architecture proposed and how this architecture is efficient in dealing with dynamic change in the presence of various types of data drift when applying these IL algorithms. [Copyright &y& Elsevier]

Abstract: Support vector machine (SVM) is a powerful technique for data classification. Despite of its good theoretic foundations and high classification accuracy, normal SVM is not suitable for classification of large data sets, because the training complexity of SVM is highly dependent on the size of data set. This paper presents a novel SVM classification approach for large data sets by using minimum enclosing ball clustering. After the training data are partitioned by the proposed clustering method, the centers of the clusters are used for the first time SVM classification. Then we use the clusters whose centers are support vectors or those clusters which have different classes to perform the second time SVM classification. In this stage most data are removed. Several experimental results show that the approach proposed in this paper has good classification accuracy compared with classic SVM while the training is significantly faster than several other SVM classifiers. [Copyright &y& Elsevier]

Discriminant analysis is an important and well-studied algorithm in pattern recognition area, and many linear discriminant analysis methods have been proposed over the last few decades. However, in the previous works, the between-scatter matrix is not updated when seeking the discriminant vectors, which causes redundancy for the well separated pairs. In this paper, a between-scatter matrix updating scheme is proposed based on the separable status of the obtained vectors. In our scheme, separable status determination of obtained vectors is decisive. Here, we notice that appropriate separation of a multi-dimensional feature (with homoscedastic Gaussian distribution) may help to find better discriminant vectors, and the separability of a multi-dimensional feature can be deduced from the separability of its elements. To make the discriminant vectors statistically uncorrelated, the algorithm is applied to the S t -orthogonal space of the obtained vectors in an iterative way. We also extend our method to more general cases, like heteroscedastic distributions, by an appropriate kernel function. Experimental results on multiple databases demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Due to the influence of environment, data obtained in real world are not completely reliable sometimes. This paper focuses on tackling the feature selection problem with unreliable data. First, the problem is formulated as an multi-objective optimization one with two objectives: the reliability and the classification accuracy. Then, an effective multi-objective feature selection algorithm based on bare-bones particle swarm optimization is proposed by incorporating two new operators. One is a reinforced memory strategy, which is designed to overcome the degradation phenomenon of particles. Another is a hybrid mutation, which is designed to improve the search ability of the proposed algorithm. Finally, two state-of-the-art multi-objective optimization algorithms are also applied to this kind of problem, and comparison results suggest that the proposed algorithm is highly competitive for the feature selection problem with unreliable data. [ABSTRACT FROM AUTHOR]

This paper brings up a very important issue for active learning in practice. Traditional active learning mechanism is based on the assumption that the number of classes happens to be known in advance, and thus selective sampling is confined to the determinate model. However, as is the case for many applications, the model class is usually indeterminate and there is every chance that the hypothesis itself is inappropriate. To address this problem, we propose a novel indeterminate multi-class active learning algorithm, which comprehensively evaluates the instance based on both the value in refining the existing model and the potential in triggering model rectification. In this way, balance is effectively achieved between model update and model upgrade. Advantage of the proposed algorithm is demonstrated by experiments of classification tasks on both synthetic and real-world dataset. [ABSTRACT FROM AUTHOR]

In many applications, the training data, from which one needs to learn a classifier, is corrupted with label noise. Many standard algorithms such as SVM perform poorly in the presence of label noise. In this paper we investigate the robustness of risk minimization to label noise. We prove a sufficient condition on a loss function for the risk minimization under that loss to be tolerant to uniform label noise. We show that the 0–1 loss, sigmoid loss, ramp loss and probit loss satisfy this condition though none of the standard convex loss functions satisfy it. We also prove that, by choosing a sufficiently large value of a parameter in the loss function, the sigmoid loss, ramp loss and probit loss can be made tolerant to non-uniform label noise also if we can assume the classes to be separable under noise-free data distribution. Through extensive empirical studies, we show that risk minimization under the 0–1 loss, the sigmoid loss and the ramp loss has much better robustness to label noise when compared to the SVM algorithm. [ABSTRACT FROM AUTHOR]

Cancer is curable if it can be detected early. One way to detect cancer is by analyzing the change in expression of genes in the suspected tissue. Serial analysis of gene expression (SAGE) is a sequencing technique used for measuring the expression levels of genes. Cancer detection problem can be posed as binary classification problem like whether a tissue is cancerous or normal. SAGE libraries contain expression levels of thousands of genes which are the features. It is impossible to consider all these features for classification and also the general feature selection algorithms are not efficient with this data. In this paper, closed frequent itemset mining is proposed as a feature selection technique for identifying a small set of features which can distinguish the two classes efficiently. The performance of the proposed technique is evaluated on SAGE data related to breast tissue and a group of 26 genes are selected as best features. Two well known classifiers, extreme learning machine (ELM) and support vector machine (SVM), are used to evaluate the effectiveness of the selected features in classification and found that the proposed method works well with these classifiers. [ABSTRACT FROM AUTHOR]

A new algorithm for ensemble construction based on adapted restricting a set of weights of examples in training data to avoid overfitting and to reduce a number of iterations is proposed in the paper. The algorithm called IDMBoost (Imprecise Dirichlet Model Boost) applies Walley׳s imprecise Dirichlet model for modifying the restricted sets of weights depending on the number and location of classification errors. Updating of weights within the restricted set (simplex) is carried out by using its extreme points. The proposed algorithm has a double adaptation procedure. The first adaptation is carried out within every restricted simplex like the AdaBoost. The second adaptation reduces and changes the restricted sets of possible weights of examples. Various numerical experiments with real data illustrate the proposed algorithm. [ABSTRACT FROM AUTHOR]

There has been some improvement in the treatment of preterm infants, which has helped to increase their chance of survival. However, the rate of premature births is still globally increasing. As a result, this group of infants is most at risk of developing severe medical conditions that can affect the respiratory, gastrointestinal, immune, central nervous, auditory and visual systems. There is a strong body of evidence emerging that suggests the analysis of uterine electrical signals, from the abdominal surface (Electrohysterography – EHG), could provide a viable way of diagnosing true labour and even predict preterm deliveries. This paper explores this idea further and presents a new dynamic self-organized network immune algorithm that classifies term and preterm records, using an open dataset containing 300 records (38 preterm and 262 term ). Using the dataset, oversampling and cross validation techniques are evaluated against other similar studies. The proposed approach shows an improvement on existing studies with 89% sensitivity , 91% specificity , 90% positive predicted value , 90% negative predicted value , and an overall accuracy of 90%. [ABSTRACT FROM AUTHOR]

The image parsing process gives labels to image regions, as well as information including shape, semantics and context. Although it is one of the most important features in the human visual system, automatic image parsing using computer vision techniques remains difficult due to computational issue. In this paper we introduce a novel method to address this limitation. Our system models an image as a set of regions and uses a novel hypotheses generation algorithm to get possible image parsing solutions for final re-scoring. Our proposed hypotheses generation algorithm, called Loopy Dynamic Programming (LDP), handles large search space efficiently and gives good parsing hypotheses for testing. With such capacity, we are able to apply more precise and complex image models to achieve better performance. In addition, our system can perform image segmentation, detection and classification simultaneously. Experimental results using Pascal VOC 2007 dataset show that the proposed technique achieves very promising performance in all the three tasks. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, a novel supervised feature extraction and classification fusion algorithm based on neighborhood preserving embedding (NPE) and sparse representation is proposed. Specifically, an optimal dictionary is adaptively learned to bate the trivial information of the original training data; then, in order to obtain the sparse representation coefficients, a sparse preserving embedding map is sought to reduce the dimensionality of high-dimensional data, and the test data is classified by the corresponding sparse representation coefficients. Finally, the novel supervised fusion algorithm is applied to the land cover recognition of the off-land scenario. Experimental results show that the proposed method leads to promising results in fusing feature extraction and classification. [Copyright &y& Elsevier]

Abstract: Least squares support vector machines (LS-SVMs) are sensitive to outliers or noise in the training dataset. Weighted least squares support vector machines (WLS-SVMs) can partly overcome this shortcoming by assigning different weights to different training samples. However, it is a difficult task for WLS-SVMs to set the weights of the training samples, which greatly influences the robustness of WLS-SVMs. In order to avoid setting weights, in this paper, a novel robust LS-SVM (RLS-SVM) is presented based on the truncated least squares loss function for regression and classification with noise. Based on its equivalent model, we theoretically analyze the reason why the robustness of RLS-SVM is higher than that of LS-SVMs and WLS-SVMs. In order to solve the proposed RLS-SVM, we propose an iterative algorithm based on the concave–convex procedure (CCCP) and the Newton algorithm. The statistical tests of the experimental results conducted on fourteen benchmark regression datasets and ten benchmark classification datasets show that compared with LS-SVMs, WLS-SVMs and iteratively reweighted LS-SVM (IRLS-SVM), the proposed RLS-SVM significantly reduces the effect of the noise in the training dataset and provides superior robustness. [Copyright &y& Elsevier]

Abstract: Individuals from Pittsburgh rule-based classifiers represent a complete solution to the classification problem and each individual is a variable-length set of rules. Therefore, these systems usually demand a high level of computational resources and run-time, which increases as the complexity and the size of the data sets. It is known that this computational cost is mainly due to the recurring evaluation process of the rules and the individuals as rule sets. In this paper we propose a parallel evaluation model of rules and rule sets on GPUs based on the NVIDIA CUDA programming model which significantly allows reducing the run-time and speeding up the algorithm. The results obtained from the experimental study support the great efficiency and high performance of the GPU model, which is scalable to multiple GPU devices. The GPU model achieves a rule interpreter performance of up to 64 billion operations per second and the evaluation of the individuals is speeded up of up to 3.461×when compared to the CPU model. This provides a significant advantage of the GPU model, especially addressing large and complex problems within reasonable time, where the CPU run-time is not acceptable. [Copyright &y& Elsevier]

Abstract: Semi-supervised classification has become an active topic recently and a number of algorithms, such as Self-training, have been proposed to improve the performance of supervised classification using unlabeled data. In this paper, we propose a semi-supervised learning framework which combines clustering and classification. Our motivation is that clustering analysis is a powerful knowledge-discovery tool and it may reveal the underlying data space structure from unlabeled data. In our framework, semi-supervised clustering is integrated into Self-training classification to help train a better classifier. In particular, the semi-supervised fuzzy c-means algorithm and support vector machines are used for clustering and classification, respectively. Experimental results on artificial and real datasets demonstrate the advantages of the proposed framework. [Copyright &y& Elsevier]

Abstract: This paper introduces Sphere Support Vector Machines (SVMs) as the new fast classification algorithm based on combining a minimal enclosing ball approach, state of the art nearest point problem solvers and probabilistic techniques. The blending of the three significantly speeds up the training phase of SVMs and also attains practically the same accuracy as the other classification models over several large real datasets within the strict validation frame of a double (nested) cross-validation. The results shown are promoting SphereSVM as outstanding alternatives for handling large and ultra-large datasets in a reasonable time without switching to various parallelization schemes for SVM algorithms recently proposed. [Copyright &y& Elsevier]

Abstract: The parametric-margin vector machine is a useful classifier in many cases, especially when the noise is heteroscedastic. In this paper, the geometric interpretation for the is described, which is equivalent to finding a couple of points in two disjoint convex hulls by simultaneously minimizing the square distance and maximizing the square norm of their sum with a weight factor given by users. Motivated by the Gilbert–Schlesinger–Kozinec (GSK) and Mitchell–Dem''yanov–Malozemov (MDM) algorithms, two geometric algorithms, called the parametric and parametric algorithms, are introduced to solve the . Computational results on several synthetic as well as benchmark datasets demonstrate the significant performance of the proposed algorithms in terms of both kernel operations and classification accuracy. [Copyright &y& Elsevier]

Abstract: In this paper, we propose a sequential learning algorithm for a neural network classifier based on human meta-cognitive learning principles. The network, referred to as Meta-cognitive Neural Network (McNN). McNN has two components, namely the cognitive component and the meta-cognitive component. A radial basis function network is the fundamental building block of the cognitive component. The meta-cognitive component controls the learning process in the cognitive component by deciding what-to-learn, when-to-learn and how-to-learn. When a sample is presented at the cognitive component of McNN, the meta-cognitive component chooses the best learning strategy for the sample using estimated class label, maximum hinge error, confidence of classifier and class-wise significance. Also sample overlapping conditions are considered in growth strategy for proper initialization of new hidden neurons. The performance of McNN classifier is evaluated using a set of benchmark classification problems from the UCI machine learning repository and two practical problems, viz., the acoustic emission for signal classification and a mammogram data set for cancer classification. The statistical comparison clearly indicates the superior performance of McNN over reported results in the literature. [Copyright &y& Elsevier]

Abstract: In this paper, we investigate the decision making ability of a fully complex-valued radial basis function (FC-RBF) network in solving real-valued classification problems. The FC-RBF classifier is a single hidden layer fully complex-valued neural network with a nonlinear input layer, a nonlinear hidden layer, and a linear output layer. The neurons in the input layer of the classifier employ the phase encoded transformation to map the input features from the Real domain to the Complex domain. The neurons in the hidden layer employ a fully complex-valued Gaussian-like activation function of the type of hyperbolic secant (sech). The classification ability of the classifier is first studied analytically and it is shown that the decision boundaries of the FC-RBF classifier are orthogonal to each other. Then, the performance of the FC-RBF classifier is studied experimentally using a set of real-valued benchmark problems and also a real-world problem. The study clearly indicates the superior classification ability of the FC-RBF classifier. [Copyright &y& Elsevier]

Abstract: The local tangent space alignment (LTSA) has demonstrated promising results in finding meaningful low-dimensional structures hidden in high-dimensional data. However, LTSA may have a limited effectiveness on the data which are organized in multiple classes or contain noisy points. In this paper, the distances between the samples and their neighbors are rescaled by using the reconstruction weights to overcome the limitation. An extension of LTSA is proposed based on the local rescaled distance matrix. Numerical experiments on both synthetic and real-world data sets are used to show the improvement of our extension for classification and the robustness to noisy data. [Copyright &y& Elsevier]

Abstract: Sparse representation has attracted great attention in the past few years. Sparse representation based classification (SRC) algorithm was developed and successfully used for classification. In this paper, a kernel sparse representation based classification (KSRC) algorithm is proposed. Samples are mapped into a high dimensional feature space first and then SRC is performed in this new feature space by utilizing kernel trick. Since samples in the high dimensional feature space are unknown, we cannot perform KSRC directly. In order to overcome this difficulty, we give the method to solve the problem of sparse representation in the high dimensional feature space. If an appropriate kernel is selected, in the high dimensional feature space, a test sample is probably represented as the linear combination of training samples of the same class more accurately. Therefore, KSRC has more powerful classification ability than SRC. Experiments of face recognition, palmprint recognition and finger-knuckle-print recognition demonstrate the effectiveness of KSRC. [Copyright &y& Elsevier]

Abstract: In machine learning the so-called curse of dimensionality, pertinent to many classification algorithms, denotes the drastic increase in computational complexity and classification error with data having a great number of dimensions. In this context, feature selection techniques try to reduce dimensionality finding a new more compact representation of instances selecting the most informative features and removing redundant, irrelevant, and/or noisy features. In this paper, we propose a filter-based feature selection method for working in the multiple-instance learning scenario called ReliefF-MI; it is based on the principles of the well-known ReliefF algorithm. Different extensions are designed and implemented and their performance checked in multiple instance learning. ReliefF-MI is applied as a pre-processing step that is completely independent from the multi-instance classifier learning process and therefore is more efficient and generic than wrapper approaches proposed in this area. Experimental results on five benchmark real-world data sets and 17 classification algorithms confirm the utility and efficiency of this method, both statistically and from the point of view of execution time. [Copyright &y& Elsevier]

Abstract: This paper presents a distributed system for the recognition of human actions using views of the scene grabbed by different cameras. 2D frame descriptors are extracted for each available view to capture the variability in human motion. These descriptors are projected into a lower dimensional space and fed into a probabilistic classifier to output a posterior distribution of the action performed according to the descriptor computed at each camera. Classifier fusion algorithms are then used to merge the posterior distributions into a single distribution. The generated single posterior distribution is fed into a sequence classifier to make the final decision on the performed activity. The system can instantiate different algorithms for the different tasks, as the interfaces between modules are clearly defined. Results on the classification of the actions in the IXMAS dataset are reported. The accuracy of the proposed system is similar to state-of-the-art 3D methods, even though it uses only well-known 2D pattern recognition techniques and does not need to project the data into a 3D space or require camera calibration parameters. [Copyright &y& Elsevier]

Abstract: Support vector machines (SVMs) are a widely used technique for classification, clustering and data analysis. While efficient algorithms for training SVM are available, dealing with large datasets makes training and classification a computationally challenging problem. In this paper we exploit modern processor architectures to improve the training speed of , a well known implementation of the sequential minimal optimization algorithm. We describe , an optimized version of which takes advantage of the peculiar architecture of the Cell Broadband Engine. We assess the performance of on real-world training problems, and we show how this optimization is particularly effective on large, dense datasets. [Copyright &y& Elsevier]

Abstract: This paper presents a feature selection method for data classification, which combines a model-based variable selection technique and a fast two-stage subset selection algorithm. The relationship between a specified (and complete) set of candidate features and the class label is modeled using a non-linear full regression model which is linear-in-the-parameters. The performance of a sub-model measured by the sum of the squared-errors (SSE) is used to score the informativeness of the subset of features involved in the sub-model. The two-stage subset selection algorithm approaches a solution sub-model with the SSE being locally minimized. The features involved in the solution sub-model are selected as inputs to support vector machines (SVMs) for classification. The memory requirement of this algorithm is independent of the number of training patterns. This property makes this method suitable for applications executed in mobile devices where physical RAM memory is very limited. An application was developed for activity recognition, which implements the proposed feature selection algorithm and an SVM training procedure. Experiments are carried out with the application running on a PDA for human activity recognition using accelerometer data. A comparison with an information gain-based feature selection method demonstrates the effectiveness and efficiency of the proposed algorithm. [Copyright &y& Elsevier]

Abstract: Although kernel minimum squared error (KMSE) is computationally simple, i.e., it only needs solving a linear equation set, it suffers from the drawback that in the testing phase the computational efficiency decreases seriously as the training samples increase. The underlying reason is that the solution of Naïve KMSE is represented by all the training samples in the feature space. Hence, in this paper, a method of selecting significant nodes for KMSE is proposed. During each calculation round, the presented algorithm prunes the training sample making least contribution to the objective function, hence called as PLOC-KMSE. To accelerate the training procedure, a batch of so-called nonsignificant nodes is pruned instead of one by one in PLOC-KMSE, and this speedup algorithm is named MPLOC-KMSE for short. To show the efficacy and feasibility of the proposed PLOC-KMSE and MPLOC-KMSE, the experiments on benchmark data sets and real-world instances are reported. The experimental results demonstrate that PLOC-KMSE and MPLOC-KMSE require the fewest significant nodes compared with other algorithms. That is to say, their computational efficiency in the testing phase is best, thus suitable for environments having a strict demand of computational efficiency. In addition, from the performed experiments, it is easily known that the proposed MPLOC-KMSE accelerates the training procedure without sacrificing the computational efficiency of testing phase to reach the almost same generalization performance. Finally, although PLOC and MPLOC are proposed in regression domain, they can be easily extended to classification problem and other algorithms such as kernel ridge regression. [Copyright &y& Elsevier]

Abstract: Recently, sparsity preserving projections (SPP) algorithm has been proposed, which combines l 1-graph preserving the sparse reconstructive relationship of the data with the classical dimensionality reduction algorithm. However, when applied to classification problem, SPP only focuses on the sparse structure but ignores the label information of samples. To enhance the classification performance, a new algorithm termed discriminative learning by sparse representation projections or DLSP for short is proposed in this paper. DLSP algorithm incorporates the merits of both local interclass geometrical structure and sparsity property. That makes it possess the advantages of the sparse reconstruction, and more importantly, it has better capacity of discrimination, especially when the size of the training set is small. Extensive experimental results on serval publicly available data sets show the feasibility and effectiveness of the proposed algorithm. [Copyright &y& Elsevier]

Abstract: The nearest neighbor classification is a simple and yet effective technique for pattern recognition. Performance of this technique depends significantly on the distance function used to compute similarity between examples. Some techniques were developed to learn weights of features for changing the distance structure of samples in nearest neighbor classification. In this paper, we propose an approach to learning sample weights for enlarging margin by using a gradient descent algorithm to minimize margin based classification loss. Experimental analysis shows that the distances trained in this way reduce the loss of the margin and enlarge the hypothesis margin on several datasets. Moreover, the proposed approach consistently outperforms nearest neighbor classification and some other state-of-the-art methods. [Copyright &y& Elsevier]

Abstract: Multisurface proximal support vector machine via generalized eigenvalues (GEPSVM), being an effective classification tool for supervised learning, tries to seek two nonparallel planes that are determined by solving two generalized eigenvalue problems (GEPs). The GEPs may lead to an instable classification performance, due to matrix singularity. Proximal support vector machine using local information (LIPSVM), as a variant of GEPSVM, attempts to avoid the above shortcoming through adopting a similar formulation to the Maximum Margin Criterion (MMC). The solution to an LIPSVM follows directly from solving two standard eigenvalue problems. Actually, an LIPSVM can be viewed as a reduced algorithm, because it uses the selectively generated points to train the classifier. A major advantage of an LIPSVM is that it is resistant to outliers. In this paper, following the geometric intuition of an LIPSVM, a novel multi-plane learning approach called Localized Twin SVM via Convex Minimization (LCTSVM) is proposed. This approach determines two nonparallel planes by solving two newly formed SVM-type problems. In addition to keeping the superior characteristics of an LIPSVM, an LCTSVM still has its additional edges: (1) it has similar or better classification capability compared to LIPSVM, TWSVM and LSTSVM; (2) each plane is generated from a quadratic programming problem (QPP) instead of a special convex difference optimization arising from an LIPSVM; (3) the solution can be reduced to solving two systems of linear equations, resulting in considerably lesser computational cost; and (4) it can find the global minimum. Experiments carried out on both toy and real-world problems disclose the effectiveness of an LCTSVM. [Copyright &y& Elsevier]

Abstract: Mass spectrometry (MS)-based proteomics has been established as a standard way for biomarker discovery and early detection of disease from the proteome level. However, the data generated by MS technology are noisy, have a lot of redundancy, and most importantly have low sample-to-dimension ratio. Therefore, successful analysis of MS data heavily relies on the algorithmic and data mining techniques used to extract information. In this paper, we briefly discuss some of the commonly used algorithms applied to mass-to-charge (m/z) feature selection, and propose a k-means clustering based hybrid algorithm to address the high-dimension and high-correlation issues associated with this task. Our k-means clustering based hybrid algorithm incorporates the advantages of both filter- and wrapper-based feature selection algorithms. A special iterative procedure is introduced to overcome the instability of k-means clustering and genetic algorithm. By comparing the proposed hybrid system with several popular m/z feature selection algorithms using 10 different classifiers, we show that the proposed algorithm is able to select m/z features with lower correlation while achieving higher sample classification accuracies. The m/z feature selection results also indicate that the proposed hybrid algorithm is very stable despite its stochastic nature. [ABSTRACT FROM AUTHOR]

Abstract: To conduct binary classification with highly imbalanced data is a very common problem, especially when the examples of interest are relatively rare. In this paper, we proposed the “Meta Imbalanced Classification Ensemble (MICE)” algorithm in order to dilute the effect of imbalanced data. In the MICE, the majority group is partitioned based on the transformed features from “inner product” to retain the geometric relation between two groups. The empirical results show that the performance of MICE is better than some renowned classification methods in terms of the specificity and the sensitivity. [Copyright &y& Elsevier]

Abstract: Feature selection plays an important role in pattern classification. Its purpose is to remove redundant features from data set as many as possible. The presence of useless features may not only deteriorate the performance of learning algorithms, but also obscure important information (e.g., intrinsic structure) behind data. Along with new and emerging techniques, data sets in many domains are becoming larger and larger and many irrelevant features are often prevailing in these data sets. This, however, poses great challenges to traditional learning algorithms, such as low efficiency and over-fitting. Thus, it becomes apparent that an efficient technique is needed to eliminate redundant or irrelevant features from the data sets. Currently, many endeavors to cope with this problem have been attempted and various outstanding feature selection methods have been proposed. Unlike other selection methods, in this paper we propose a general scheme of boosting feature selection method using information metric. The primary characteristic of our method is that it exploits weight of data to select salient features. Furthermore, the weight of data will be dynamically changed after each candidate feature has been selected. Thus, the information criteria used in feature selector can exactly represent the relevant degree between features and the class labels. As a result, the selected feature subset has maximal relevance to the class labels. Simulation studies carried out on UCI data sets show that the classification performance achieved by our proposed method is better than those of other selection methods in most cases. [Copyright &y& Elsevier]