Search

• An analysis of lexicographic optimization for feature selection and data classification. • Cooperative co-evolution for feature selection while the classifier hyperparameters are also being optimized. • Optimization of multiple objectives using a simple evolutionary algorithm. • A new lexicographic ranking methodology to compare the results of different feature selection methods. • Quite good results for real high-dimensional datasets. This paper starts with two hypotheses. The first one is that the simultaneous optimization of the hyperparameters regulating the classifier within a wrapper method, while the best subset of features is being determined, should improve the results with respect to those obtained with a pre-parameterized classifier. The second one is that solving these two problems can be formulated as a lexicographic optimization problem, allowing the use of a simple single-objective evolutionary algorithm to solve this multi-objective problem. The fitness function is of key importance for such wrapper methods. It is responsible for guiding the search towards potentially good solutions and it also consumes most of the runtime. Having these issues in mind, this paper also proposes a new lexicographic fitness function, designed to minimize the runtime of the algorithm and also to avoid over-fitting. Furthermore, the execution time and the quality of the results obtained by the wrapper procedure also depend on some algorithmic hyperparameters: the similarity thresholds used when comparing two different solutions lexicographically and the percentage of data samples used for validation during the training process. Thus, an experimental analysis has been carried out to find adequate values for these hyperparameters. Finally, the lexicographic cooperative co-evolutionary wrapper approach, using the new fitness function proposed in this paper, has been tested with several datasets belonging to the University of California, Irvine (UCI) repository and also with some real high-dimensional datasets, obtaining quite good results, compared to other state-of-the-art wrapper methods. The comparison has also been made lexicographically, with a new methodology proposed in this paper. [ABSTRACT FROM AUTHOR]

In this paper, we study the decentralized AdaBoost problem over sensor networks, and propose a fully decentralized AdaBoost algorithm, where each sensor can obtain the centralized global solution without transmission of private dataset. By decomposing the centralized cost function into a summation of local ones, we convert decentralized AdaBoost problem into a distributed optimization problem, and design a distributed alternating minimization method to solve it. In order to improve convergence rate, motivated by Nesterov gradient descent method, we propose a fast decentralized AdaBoost algorithm. Then, we prove the convergence of proposed algorithms. Moreover, we deduce decentralized AdaBoost algorithm for logistic regression in detail. The simulations with Spam-Email dataset illustrate the effectiveness of proposed algorithms. [ABSTRACT FROM AUTHOR]

Recently, the Extreme Learning Machine (ELM) algorithm has been applied to various fields due to its rapidity and significant generalization performance. Traditionally, deep learning (DL) and wavelet neural networks (WNN) methods reach a high classification accuracy in machine learning applications. As a result, a new structure based on WNN, deep architecture and ELM is proposed in this paper. The proposed method is based on Extreme Learning Machine Auto-Encoder with DL structure and a composite wavelet activation function used in the hidden nodes. To evaluate the performance of our approach, we used standard benchmark data-sets, namely COIL-20, Pima Indian Diabetes (PID), MNIST and EMNIST. Experimental results show that our method offers satisfactory results and performance compared to other approaches. [ABSTRACT FROM AUTHOR]

• An algorithm that transforms long-tailed problem into hierarchical learning problem. • An effective knowledge transfer paradigm is proposed to help hierarchical learning. • The performance of our model outperforms the state-of-the-art methods. Deep visual recognition methods have achieved excellent performance on artificially constructed image datasets where the data distribution is balanced. However, in real-world scenarios, data distribution is usually extremely imbalanced and exhibit a long-tailed distribution where data in each head class is more than the class in the tail. Many efficient deep learning methods fail to work normally, i.e., they perform well in the head class while poor in the tail class. In this paper, we propose a two-layer Hierarchical-Learning Long-Tailed Recognition (HL-LTR) algorithm which transforms the long-tailed problem into a hierarchical classification problem by constructing a hierarchical superclass tree in which each layer corresponds to a recognition task. In the first layer of the tree, the degree of data imbalance is largely decreased. The recognition task of the second layer is the original long-tailed recognition problem. The training of HL-LTR is top-down. The knowledge learned by the first layer transfers to classes of the second layer and guides the feature learning of the second layer by using attention mechanism module and knowledge distillation method. Compared with directly solving the most difficult long-tailed recognition task, HL-LTR achieves better performance due to its progressive learning method from easy to difficult and effective knowledge transfer strategy. [ABSTRACT FROM AUTHOR]

Sparse Bayesian learning (SBL) provides state-of-the-art performance in accuracy, sparsity and probabilistic prediction for classification. In SBL, the regularization priors are automatically determined that avoids an exhaustive hyperparameter selection by cross-validation. However, scalability to large problems is a drawback of SBL because of the inversion of a potentially enormous covariance matrix for updating the regularization priors in every iteration. This paper develops an approximate SBL algorithm called ARP-SBL, where the regularization priors are approximated without inverting the covariance matrix. Therefore, our approach can easily scale up to problems with large data size or feature dimension. It alleviates the long training time and high memory complexity in SBL. Based on ARP-SBL, two scalable nonlinear SBL models: scalable relevance vector machine (ARP-RVM) and scalable sparse Bayesian extreme learning machine (ARP-SBELM) are developed for problems of large data size and large feature size respectively. Experiments on a variety of benchmarks have shown that the proposed models are with competitive accuracy compared to existing methods while i) converging faster; ii) requiring thousands of times less memory; iii) without exhaustive regularized hyperparameter selection; and iv) easily scaling up to large data size and high dimensional features. [ABSTRACT FROM AUTHOR]

A novel sparse tensor dictionary learning algorithm and a convolutional neural network (CNN) classification method based on this algorithm are proposed for hyperspectral image (HSI) in this paper. The HSI is intuitively represented as a three-dimensional (3-D) cube, and utilizing the joint spatial-spectral information can significantly improve the accuracy of HSI classification. Therefore, the proposed atom-substituted tensor dictionary learning (ASTDL) algorithm utilizes tensor techniques to extract 3-D joint spatial-spectral features from HSI cubes directly. Sparsity constraint is enforced on the coefficient tensors, which obeys the sparsity attribute of HSI. The proposed ASTDL enhanced CNN (ASTDL-CNN) classification method utilizes a two-dimensional (2-D) CNN to extract deep features from the feature tensors obtained by the ASTDL, and to perform the pixel-wise classification. The use of ASTDL, which extracts intrinsic tensor features before CNN, alleviates within-class spatial-spectral variation and reduces the requirement of CNN for the labeled data. Whereas, the CNN works as a 3-D classifier, and provides the final classification results. Besides, we perform the majority vote on the classification map obtained by CNN to refine the classification. The performance of the proposed method is evaluated on three real HSI data sets. The competitive results to the compared state-of-the-art methods demonstrate that the ASTDL-CNN can provide accurate and robust classification results. [ABSTRACT FROM AUTHOR]

This paper investigates a new bacterial colony-based feature selection algorithm to improve the classification accuracy of customers for personalized products recommendation. An attribute learning strategy is developed in this study to update the feature related population. Specifically, the features can be weighted according to their historic contributions to both the individual- and group-based subsets. Additionally, the frequency of appearance is also recorded for the feature candidates to improve the diversity of feature distribution and avoid the over-fitting. Based on the weight-based feature indexes and frequency of appearance records, the performance of feature subsets are enhanced by replacing the features being repeatedly appeared in a same vector. To explore the feasibility of the proposed method for the missing feature problems, the objective of the optimization is to minimize the classification error using the acceptable number of features. K-Nearest Neighbor is employed as the learning technique to cooperate with the proposed feature selection method. The effectiveness of the proposed feature selection method is demonstrated by performing test on the datasets from UCI machine learning repository and real-world data from Amazon customer reviews of products. Compared with other seven feature selections methods, the proposed feature selection algorithm outperforms the other algorithms by achieving higher classification accuracy rate using smaller features. [ABSTRACT FROM AUTHOR]

Drawing from memory the face of a friend you have not seen in years is a difficult task. However, if you happen to cross paths, you would easily recognize each other. The biological memory is equipped with an impressive compression algorithm that can store the essential, and then infer the details to match perception. The Willshaw Memory is a simple abstract model for cortical computations which implements mechanisms of biological memories. Using our recently proposed sparse coding prescription for visual patterns [34] , this model can store and retrieve an impressive amount of real-world data in a fault-tolerant manner. In this paper, we extend the capabilities of the basic Associative Memory Model by using a Multiple-Modality framework. In this setting, the memory stores several modalities (e.g., visual, or textual) of each pattern simultaneously. After training, the memory can be used to infer missing modalities when just a subset is perceived. Using a simple encoder-memory-decoder architecture, and a newly proposed iterative retrieval algorithm for the Willshaw Model, we perform experiments on the MNIST dataset. By storing both the images and labels as modalities, a single Memory can be used not only to retrieve and complete patterns but also to classify and generate new ones. We further discuss how this model could be used for other learning tasks, thus serving as a biologically-inspired framework for learning. [ABSTRACT FROM AUTHOR]