Search

• We propose an improved differential evolution (DE) algorithm by composite binary crossover and Pareto refining operations to the DE. • DE solves the quantifying fraudulent transactions problem by minimizing the selected feature cost while maximizing the quantification ability. • We show that the proposed DE outperforms four other variants of DE and five state-of-the-art EAs on various credit datasets. • The results are also statically validated through the Friedman test and Wilcoxon post-hoc analysis. [Display omitted] Identification of fraudulent credit card transactions is a complex problem mainly due to the following factors: 1) The relative behavior of customers and fraudsters may alter over time. 2) The ratio of legitimate to fraudulent transactions is highly imbalanced, and 3) Investigators examine a small segment of transactions in a reasonable time frame. Researchers have proposed various algorithms to identify potential fraud in a new incoming transaction. However, these approaches require significant human investigator effort and are sometimes misleading. To address this issue, this paper proposes an improved multiobjective differential evolution (DE) algorithm to estimate the distribution of fraudulent transactions in a set of new incoming transactions, referred to as quantifying fraudulent transactions. Our paper has three major novelties. First, we present the problem formulation of cost-based feature selection with maximum quantification ability. Second, we improve the DE by applying effective trial vector generation algorithms to the random control parameter settings to exploit the advantage of individual DE variants. Third, we develop the maximum-relevancy-minimum-redundancy-based Pareto refining operator to enhance the self-learning ability of individuals in Pareto solutions. We compare our approach against four other modifications of DE and five state-of-the-art evolutionary algorithms on real-time credit datasets in streaming and non-streaming frameworks using hyper-volume, two-set coverage, and spread performance metrics. [ABSTRACT FROM AUTHOR]

• The first attempt to design a GEP-based codematrix generation algorithm. The individual structure of GEP allows the codematrix to be optimized in a much faster and more effective way. • The feature selection scheme is combined with the design of codematrix, to accomplish the feature selection and codematrix optimization simultaneously. • A greedy strategy-based adaptive optimization method to further improve the individual's quality to deal with hard classes by appending proper columns to the codematrix. The multi-class classification task is one of the most common tasks in machine learning. As a typical solution based on a partitioning strategy, Error-Correcting Output Codes (ECOC) can transform a multi-class classification problem into multiple binary classification problems. The key of ECOC is to construct an effective codematrix to represent a set of class decomposition schemes, which transforms a multiclass problem into a group of binary class problems. Consequently, the design of a fast and effective ECOC codematrix generation method is of great research significance and value for solving multi-class classification problems. In ECOC algorithms, the design of codematrix is treated as a combination problem between different code columns, in which the evolutionary algorithm shows a great advantage. Based on this consideration, the Gene Expression Programming (GEP) is applied to search for the codematrix with high performance because its expressive tree structure makes it well represent codematrcies for subsequent optimization operations. This paper proposes an adaptive ECOC algorithm based on Gene Expression Programming (GEP) and similarity measurement matrix, named GEP-ECOC. In our GEP, each individual represents a set of columns to form a random ECOC codematrix, which is optimized in the evolutionary process. Meanwhile, the crossover and mutation operations are modified to include a legality checking process to ensure that the generated codematrix satisfies the ECOC constraints. The GEP-based ECOC codematrix generation algorithm can quickly produce a codematrix with better performance, which ensures the efficiency of the algorithm to a certain extent. In addition, an adaptive algorithm based on a similarity measurement matrix is proposed to add new columns to the current codematrix, aiming to better handle hard classes. Our algorithm is compared with other algorithms on various data sets, and the experimental results confirm that our GEP-ECOC can balance the efficiency and performance of the algorithm and achieve higher performance. [ABSTRACT FROM AUTHOR]

Highlights • Unlike common multi-label feature selection, the proposed method derives from different cognitive standpoint. • Feature selection process is directly done on multi-label data, and there is no need to data transformation. • The proposed method tries to find label-specific features which are the most discriminative features for each label. • Also, an extension of our method is presented which selects a pre-defined number of features. • The proposed method is appropriate to both numerical and nominal features. • The proposed method is effective and fast. Abstract In multi-label data, each instance is associated with a set of labels, instead of one label. Similar to single-label data, feature selection plays an important role in improving classification performance. In multi-label classification, each class label might be specified by some particular characteristics of its own which are called label-specific features. In this paper, a fast accurate filter-based feature selection method is exclusively designed for multi-label datasets to find label-specific features. It maps the features to a multi-dimensional space based on a filter method, and selects the most salient features with the help of Pareto-dominance concepts from multi-objective optimization domain. Our proposed method can be used as online feature selection that deals with problems in which features arrive sequentially while the number of data samples is fixed. In this method, the number of features to be selected is specified during the process of feature selection. However, sometimes it is desired to predefine the number of features. For this reason, an extension of the proposed method is presented to solve this problem. To prove the performance of the proposed methods, several experiments are conducted on some multi-label datasets and the results are compared to five well-established multi-label feature selection methods. The results show the superiority of the proposed methods in terms of different multi-label classification criteria and execution time. [ABSTRACT FROM AUTHOR]

Highlights • For Alzheimer's disease (AD) diagnosis, we consider the integration of multi-modality imaging and genetic data which encode different level of knowledge. • With a focus on taking advantage of both phenotype and genotype information, a novel structured sparsity, defined by ℓ 1, p -norm (p > 1), regularized multiple kernel learning method is designed. • An efficient block coordinate descent algorithm applicable to any case with p > 1 was derived to solve the proposed formulation. Abstract Multimodal data fusion has shown great advantages in uncovering information that could be overlooked by using single modality. In this paper, we consider the integration of high-dimensional multi-modality imaging and genetic data for Alzheimer's disease (AD) diagnosis. With a focus on taking advantage of both phenotype and genotype information, a novel structured sparsity, defined by ℓ 1, p -norm (p > 1), regularized multiple kernel learning method is designed. Specifically, to facilitate structured feature selection and fusion from heterogeneous modalities and also capture feature-wise importance, we represent each feature with a distinct kernel as a basis, followed by grouping the kernels according to modalities. Then, an optimally combined kernel presentation of multimodal features is learned in a data-driven approach. Contrary to the Group Lasso (i.e., ℓ 2, 1 -norm penalty) which performs sparse group selection, the proposed regularizer enforced on kernel weights is to sparsely select concise feature set within each homogenous group and fuse the heterogeneous feature groups by taking advantage of dense norms. We have evaluated our method using data of subjects from Alzheimer's Disease Neuroimaging Initiative (ADNI) database. The effectiveness of the method is demonstrated by the clearly improved prediction diagnosis and also the discovered brain regions and SNPs relevant to AD. [ABSTRACT FROM AUTHOR]

Relief-like algorithms have been widely used as feature selection to reduce the dimension of high-dimensional data which involves thousands of irrelevant variables because of their low computational cost and high accuracy. Classical Relief algorithms have not exactly shown the dynamic procedure that updates weight iteratively. This paper proposes an innovative feature weight estimation method, called dynamic representation and neighbor sparse reconstruction-based Relief (DRNSR-Relief). Similar to the classical Relief algorithms, the goal of DRNSR-Relief is to maximize the expected margin in the weighted feature space. A dynamic representation framework is introduced to show the dynamic relationship between the expected margin vector and the weight vector. To achieve better neighbor reconstruction, DRNSR-Relief decomposes a nonlinear problem into a set of locally linear ones through local hyperplane with l 1 regularization and then estimates feature weights in a large margin framework. With the help of gradient ascent method, we can guarantee the convergence of DRNSR-Relief. To demonstrate the validity and the effectiveness of our formulation for feature selection in supervised learning, we perform extensive experiments on synthetic and real-world datasets. Experimental results indicate that DRNSR-Relief is very promising. [ABSTRACT FROM AUTHOR]

Feature selection remains a popular method for quantity reduction of attributes of high-dimensional data, to reduce computational costs in classifications. A new feature selection method based on the joint maximal information entropy between features and class (FS-JMIE) is proposed in this paper. Firstly, the joint maximal information entropy (JMIE) is defined to measure a feature subset. Next, a binary particle swarm optimization (BPSO) algorithm is introduced to search the optimal feature subset. Finally, classification is performed on UCI corpora to verify the performance of our proposed method compared to the traditional mutual information (MI) method, CHI method, as well as a binary version of particle swarm optimization-support vector machines (BPSO-SVMs) feature selection. Experiments show that FS-JMIE achieves an equal or better performance than MI, CHI, and BPSO-SVM. Further, FS-JMIE manifests relatively better robustness to the number of classes. Moreover, the method shows higher consistency and better time-efficiency than BPSO-SVM. [ABSTRACT FROM AUTHOR]

Streaming feature selection (SFS), is the task of selecting the most informative features in dealing with high-dimensional or incrementally growing problems. Several SFS algorithms have been proposed in the literature. However, they do not consider all feature subsets at the redundancy analysis step due to computational concerns. Moreover, they do not reconsider previously removed features which leads to losing most of the useful information. In this paper, the redundancy analysis step is defined as a binary optimization problem. Then, a binary bat algorithm (BBA) is adopted to find the minimal informative subsets. In this way, a large number of feature subsets can be considered effectively at the redundancy analysis step. In addition, an effective priority list is used to maintain previously removed redundant features. Such a list allows the re-examination of informative features. As a result, it is possible to consider the mutual information between features that are not streamed in an small time interval. Experimental studies on fifteen different types of datasets show that our approach is superior to state-of-the-art online and offline streaming feature selection methods in terms of classification accuracy. [ABSTRACT FROM AUTHOR]

• A unified model for sparse optimal scoring is proposed by employing L u -norm (0 ≤ q ≤ 1) regular term where L o − norm and(L u) -norm (0 ≤ q ≤ 1) will be selected adaptively to find more sparser solutions. • We derive an efficient iterative algorithm based on alternating direction method of multipliers (ADMM) for the new formulation. The new proposed method can solve (L o −) norm regularized problem directly rather than using convex or nonconvex approximations of (L o −) norm. • The convergence of the algorithm is analyzed theoretically. • Extensive numerical experiments show that our algorithm is efficient not only in classification accuracy but also in sparsity. Optimal scoring (OS), an equivalent form of linear discriminant analysis (LDA), is an important supervised learning method and dimensionality reduction tool. However, it is still a challenge for the classical OS on small sample size (SSS) datasets. In this paper, to find sparse discriminant vectors, we propose a unified model for sparse optimal scoring (SOS) by virtue of the generalized ℓ q -norm (0 ≤ q ≤ 1). To overcome the difficulty in treating the generalized ℓ q -norm, we propose an efficient alternative direction method of multipliers (ADMM), where proximity operator of ℓ q -norm is employed for different q values. Meanwhile, the convergence results of our method are also established. Numerical experiments on artificial and benchmark datasets demonstrate the effectiveness and feasibility of our proposed method. [ABSTRACT FROM AUTHOR]

• A hybrid Machine Learning (ML) approach for an efficient wrapper feature selection is proposed for enhancing the performance and reducing the computation time of IDS while increasing the accuracy to detect the intruder in IoT networks. • The hybrid ML method consists of two techniques: Differential Evolution (DE) and Extreme Learning Machine (ELM). The DE is responsible for choosing the beneficial features while the ELM is employed to assess the selected features. • Extensive experiments are implemented to evaluate the proposed approach using different performance metrics. In addition, to verify the efficiency of proposed model, the comparison is achieved with some related existing methods such as CFS + SVM [20] , CFS + ANN [19] , GA + Bagging [18] , CFS + RF [30] , and Chi-squared + RF [30]. The intrusion detection system (IDS) has gained a rapid increase of interest due to its widely recognized potential in various security fields, however, it suffers from several challenges. Different network datasets have several redundant and irrelevant features that affect the decision of the IDS classifier. Therefore, it is essential to decrease these features to improve the system performance. In this paper, an efficient wrapper feature selection method is proposed for improving the performance and decreasing the processing time of the IDS. The proposed approach employs a differential evaluation algorithm to select the useful features whilst the extreme learning machine classifier is applied after feature selection to evaluate the selected features. Many experiments are performed using the full NSL-KDD dataset to evaluate the performance of the proposed method. The results prove that the proposed approach can efficiently reduce the features, increase the accuracy, reduce the false alarm rates, and improve the processing time of the IDS in comparison to other recent related works. [ABSTRACT FROM AUTHOR]

Facial wrinkles are important features of aging human skin which can be incorporated in several image-based applications related to aging. Facial wrinkles are 3D features of skin and appear as subtle discontinuities or cracks in surrounding skin texture. However, facial wrinkles can easily be masked by illumination/acquisition conditions in 2D images due to the specific nature of skin surface texture and its reflective properties. Existing approaches to image-based analysis of aging skin are based on the analysis of wrinkles as texture and not as curvilinear discontinuity/crack features. Previously, we proposed a stochastic approach based on Marked Point Processes (MPP) to localize facial wrinkles as curves. In this paper, we present a fast deterministic algorithm based on Gabor filters and image morphology to improve localization results. We propose image features based on Gabor filter bank to highlight the subtle curvilinear discontinuities in skin texture caused by wrinkles. Then, image morphology is used to incorporate geometric constraints to localize curvilinear shapes of wrinkles at image sites of large Gabor filter responses. Experiments are conducted on two sets of low and high resolution images and results are compared with those of MPP modeling. Experiments show that the proposed algorithm not only is significantly faster than MPP-based approach but also provides visually better results. [ABSTRACT FROM AUTHOR]

In this paper, we develop a computer aided diagnosis algorithm to detect and classify the abnormalities in vision-based endoscopic examination. We focus on analyzing the traditional gastroscope data and help the medical experts improve the accuracy of medical diagnosis with our analysis tool. To achieve this, we first segment the image into superpixels, then extract various color and texture features from them and combine the features into one feature vector to represent the images. This approach is more flexible and accurate than the traditional patch-based image representation. Then we design a novel feature selection model with group sparsity, Deep Sparse SVM (DSSVM) that not only can assign a suitable weight to the feature dimensions like the other traditional feature selection models, but also directly exclude useless features from the feature pool. Thus, our DSSVM model can maintain the accuracy while reducing the computation complexity. Moreover, the image quality is also pre-assessed. For the experiments, we build a new gastroscope dataset with a total of about 3800 images from 1284 volunteers, and conducted various experiments and comparisons with other algorithms to justify the effectiveness and efficiency of our algorithm. [ABSTRACT FROM AUTHOR]

• A novel unsupervised feature selection method is proposed which aims at preserving the local invariance of feature subspace by local linear embedding and manifold regularization simultaneously. • An alternating algorithm based on alternative direction method of multiplier s with computational complexity analysis is designed to handle the resulting optimization problem. • A variety of experiments on several publicly available real world datasets are conducted to demonstrate the effectiveness and superiority of the proposed method. As one of the important dimensionality reduction techniques, unsupervised feature selection (UFS) has enjoyed amounts of popularity over the last few decades, which can not only improve learning performance, but also enhance interpretability and reduce computational costs. The existing UFS methods often model the data in the original feature space, which cannot fully exploit the discriminative information. In this paper, to address this issue, we investigate how to strengthen the relationship between UFS and the feature subspace, so as to select relevant features more straightforwardly and effectively. Methodologically, a novel UFS approach, referred to as Graph Regularized Local Linear Embedding (GLLE), is proposed by integrating local linear embedding (LLE) and manifold regularization constrained in feature subspace into a unified framework. To be more specific, we explicitly define a feature selection matrix composed of 0 and 1, which can realize the process of UFS. For the purpose of modelling the feature selection matrix, we propose to preserve the local linear reconstruction relationship among neighboring data points in the feature subspace, which corresponds to LLE constrained in the feature subspace. To make the feature selection matrix more accurate, we propose to use manifold regularization as an assistant of LLE to find the relevant and representative features such that the selected features can make each sample under the feature subspace be accordance with the manifold assumption. A tailored iterative algorithm based on Alternative Direction Method of Multipliers (ADMM) is designed to solve the proposed optimization problem. Extensive experiments on twelve real-world benchmark datasets are conducted, and the more promising results are achieved compared with the state-of-the-arts approaches. [ABSTRACT FROM AUTHOR]

Abstract: This paper presents a novel feature-selection algorithm for data regression with a lot of irrelevant features. The proposed method is based on well-established machine-learning technique without any assumption about the underlying data distribution. The key idea in this method is to decompose an arbitrarily complex nonlinear problem into a set of locally linear ones through local information, and to learn globally feature relevance within the least squares loss framework. In contrast to other feature-selection algorithms for data regression, the learning of this method is efficient since the solution can be readily found through gradient descent with a simple update rule. Experiments on some synthetic and real-world data sets demonstrate the viability of our formulation of the feature-selection problem and the effectiveness of our algorithm. [Copyright &y& Elsevier]

• Presenting a new multi-label feature selection method that efficiently combines manifold regularization and dependence maximization. • Introducing a HSIC-based measurement to evaluate the dependence between the manifold space and label space. • Developing an iterative optimization method to solve the objective function of our method MRDM with good convergence. • Conducting extensive experiments on various multi-label data sets to demonstrate the superiority of the proposed method. Feature selection is able to select more discriminative features for classification and plays an important role in multi-label learning to alleviate the effect of the curse of dimensionality. Recently, the multi-label feature selection methods based on the sparse regression model have received increasing attentions. However, most of these methods directly project original data space to label space in the regression model, which is inappropriate because the linear assumption between data space and label space doesn't hold in most cases. In the paper, we propose a feature selection method named multi-label feature selection via manifold regularization and dependence maximization (MRDM). In the regression model of MRDM, the original data space is projected to a low-dimensional manifold space, which not only has the same topological structure with the original data, but also has a strong dependence with the class labels. Then, an objective function involving l 2 , 1 -norm regularization is formulated, and an alternating optimization-based iterative algorithm is designed to obtain the sparse coefficients for multi-label feature selection. Extensive experiments on various multi-label data sets demonstrate the superiority of the proposed method compared with some state-of-the-art multi-label feature selection methods. [ABSTRACT FROM AUTHOR]

• We propose a new feature selection method based on graph-based feature representations and the fused lasso framework. • Our approach can accommodate structural relationship between pairs of samples through graph-based features. • Our method can enhance the trade-off between the relevance of each feature and the redundancy between pairwise features. • An iterative algorithm is developed to identify the most discriminative features. • Experiments demonstrate that our proposed approach can outperform its competitors on benchmark datasets. Most state-of-the-art feature selection methods tend to overlook the structural relationship between a pair of samples associated with each feature dimension, which may encapsulate useful information for refining the performance of feature selection. Moreover, they usually consider candidate feature relevancy equivalent to selected feature relevancy, and therefore, some less relevant features may be misinterpreted as salient features. To overcome these issues, we propose a new feature selection method based on graph-based feature representations and the Fused Lasso framework in this paper. Unlike state-of-the-art feature selection approaches, our method has two main advantages. First, it can accommodate structural relationship between a pair of samples through a graph-based feature representation. Second, our method can enhance the trade-off between the relevancy of each individual feature on the one hand and its redundancy between pairwise features on the other. This is achieved through the use of a Fused Lasso framework applied to features reordered on the basis of their relevance with respect to the target feature. To effectively solve the optimization problem, an iterative algorithm is developed to identify the most discriminative features. Experiments demonstrate that our proposed approach can outperform its competitors on benchmark datasets. [ABSTRACT FROM AUTHOR]

Abstract: Feature subset selection is often required as a preliminary work for many pattern recognition problems. In this paper, a novel filter framework is presented to select optimal feature subset based on a maximum weight and minimum redundancy (MWMR) criterion. Since the weight of each feature indicates its importance for some ad hoc tasks (such as clustering and classification) and the redundancy represents the correlations among features. Through the proposed MWMR, we can select the feature subset in which the features are most beneficial to the subsequent tasks while the redundancy among them is minimal. Moreover, a pair-wise updating based iterative algorithm is introduced to solve our framework effectively. In the experiments, three feature weighting algorithms (Laplacian score, Fisher score and Constraint score) are combined with two redundancy measurement methods (Pearson correlation coefficient and Mutual information) to test the performances of proposed MWMR. The experimental results on five different databases (CMU PIE, Extended YaleB, Colon, DLBCL and PCMAC) demonstrate the advantage and efficiency of our MWMR. [Copyright &y& Elsevier]

Abstract: We propose a new supervised object retrieval method based on the selection of local visual features learned with the BLasso algorithm. BLasso is a boosting-like procedure that efficiently approximates the Lasso path through backward regularization steps. The advantage compared to a classical boosting strategy is that it produces a sparser selection of visual features. This allows us to improve the efficiency of the retrieval and, as discussed in the paper, it facilitates human visual interpretation of the models generated. We carried out our experiments on the Caltech-256 dataset with state-of-the-art local visual features. We show that our method outperforms AdaBoost in effectiveness while significantly reducing the model complexity and the prediction time. We discuss the evaluation of the visual models obtained in terms of human interpretability. [Copyright &y& Elsevier]

Abstract: The common spatial pattern (CSP) algorithm is effective in decoding the spatial patterns of the corresponding neuronal activities from electroencephalogram (EEG) signal patterns in brain–computer interfaces (BCIs). However, its effectiveness depends on the subject-specific time segment relative to the visual cue and on the temporal frequency band that is often selected manually or heuristically. This paper presents a novel statistical method to automatically select the optimal subject-specific time segment and temporal frequency band based on the mutual information between the spatial–temporal patterns from the EEG signals and the corresponding neuronal activities. The proposed method comprises four progressive stages: multi-time segment and temporal frequency band-pass filtering, CSP spatial filtering, mutual information-based feature selection and naïve Bayesian classification. The proposed mutual information-based selection of optimal spatial–temporal patterns and its one-versus-rest multi-class extension were evaluated on single-trial EEG from the BCI Competition IV Datasets IIb and IIa respectively. The results showed that the proposed method yielded relatively better session-to-session classification results compared against the best submission. [Copyright &y& Elsevier]

Abstract: Pedestrian counting plays an important role in public safety and intelligent transportation. Most pedestrian counting algorithms based on supervised learning require much labeling work and rarely exploit the topological information of unlabelled data in a video. In this paper, we propose a Semi-Supervised Elastic Net (SSEN) regression method by utilizing sequential information between unlabelled samples and their temporally neighboring samples as a regularization term. Compared with a state-of-the-art algorithm, extensive experiments indicate that our algorithm can not only select sparse representative features from the original feature space without losing their interpretability, but also attain superior prediction performance with only very few labelled frames. [Copyright &y& Elsevier]

Abstract: Feature selection is an important preprocessing step for building efficient, generalizable and interpretable classifiers on high dimensional data sets. Given the assumption on the sufficient labelled samples, the Markov Blanket provides a complete and sound solution to the selection of optimal features, by exploring the conditional independence relationships among the features. In real-world applications, unfortunately, it is usually easy to get unlabelled samples, but expensive to obtain the corresponding accurate labels on the samples. This leads to the potential waste of valuable classification information buried in unlabelled samples. In this paper, we propose a new BAyesian Semi-SUpervised Method, or BASSUM in short, to exploit the values of unlabelled samples on classification feature selection problem. Generally speaking, the inclusion of unlabelled samples helps the feature selection algorithm on (1) pinpointing more specific conditional independence tests involving fewer variable features and (2) improving the robustness of individual conditional independence tests with additional statistical information. Our experimental results show that BASSUM enhances the efficiency of traditional feature selection methods and overcomes the difficulties on redundant features in existing semi-supervised solutions. [Copyright &y& Elsevier]

Abstract: Feature and instance selection are two effective data reduction processes which can be applied to classification tasks obtaining promising results. Although both processes are defined separately, it is possible to apply them simultaneously. This paper proposes an evolutionary model to perform feature and instance selection in nearest neighbor classification. It is based on cooperative coevolution, which has been applied to many computational problems with great success. The proposed approach is compared with a wide range of evolutionary feature and instance selection methods for classification. The results contrasted through non-parametric statistical tests show that our model outperforms previously proposed evolutionary approaches for performing data reduction processes in combination with the nearest neighbor rule. [Copyright &y& Elsevier]

Abstract: Currently the best algorithms for transcription factor binding site prediction within sequences of regulatory DNA are severely limited in accuracy. In this paper, we integrate original binding site prediction algorithms, and use a ‘window’ of consecutive predictions in order to contextualise the neighbouring results. We combine either random selection or Tomek links under-sampling with SMOTE over-sampling techniques. In addition, we investigate the behaviour of four feature selection filtering methods: bi-normal separation, correlation coefficients, F-Score and a cross entropy based algorithm. Finally, we remove some of the final predicted binding sites on the basis of their biological plausibility. The results show that we can generate a new prediction that significantly improves on the performance of any one of the individual algorithms. [Copyright &y& Elsevier]

Abstract: The problem of selecting a subset of relevant features is classic and found in many branches of science including—examples in pattern recognition. In this paper, we propose a new feature selection criterion based on low-loss nearest neighbor classification and a novel feature selection algorithm that optimizes the margin of nearest neighbor classification through minimizing its loss function. At the same time, theoretical analysis based on energy-based model is presented, and some experiments are also conducted on several benchmark real-world data sets and facial data sets for gender classification to show that the proposed feature selection method outperforms other classic ones. [Copyright &y& Elsevier]

Abstract: Feature set partitioning generalizes the task of feature selection by partitioning the feature set into subsets of features that are collectively useful, rather than by finding a single useful subset of features. This paper presents a novel feature set partitioning approach that is based on a genetic algorithm. As part of this new approach a new encoding schema is also proposed and its properties are discussed. We examine the effectiveness of using a Vapnik–Chervonenkis dimension bound for evaluating the fitness function of multiple, oblivious tree classifiers. The new algorithm was tested on various datasets and the results indicate the superiority of the proposed algorithm to other methods. [Copyright &y& Elsevier]

Abstract: Feature selection is an important preprocessing step in mining high-dimensional data. Generally, supervised feature selection methods with supervision information are superior to unsupervised ones without supervision information. In the literature, nearly all existing supervised feature selection methods use class labels as supervision information. In this paper, we propose to use another form of supervision information for feature selection, i.e. pairwise constraints, which specifies whether a pair of data samples belong to the same class (must-link constraints) or different classes (cannot-link constraints). Pairwise constraints arise naturally in many tasks and are more practical and inexpensive than class labels. This topic has not yet been addressed in feature selection research. We call our pairwise constraints guided feature selection algorithm as Constraint Score and compare it with the well-known Fisher Score and Laplacian Score algorithms. Experiments are carried out on several high-dimensional UCI and face data sets. Experimental results show that, with very few pairwise constraints, Constraint Score achieves similar or even higher performance than Fisher Score with full class labels on the whole training data, and significantly outperforms Laplacian Score. [Copyright &y& Elsevier]

Abstract: In many systems, such as fuzzy neural network, we often adopt the language labels (such as large, medium, small, etc.) to split the original feature into several fuzzy features. In order to reduce the computation complexity of the system after the fuzzification of features, the optimal fuzzy feature subset should be selected. In this paper, we propose a new heuristic algorithm, where the criterion is based on min–max learning rule and fuzzy extension matrix is designed as the search strategy. The algorithm is proved in theory and has shown its high performance over several real-world benchmark data sets. [Copyright &y& Elsevier]

Abstract: In this paper we present a study of structural features of handwriting extracted from three characters “d”, “y”, and “f” and grapheme “th”. The features used are based on the standard features used by forensic document examiners. The process of feature extraction is presented along with the results. Analysis of the usefulness of features was conducted via searching the optimal feature sets using the wrapper method. A neural network was used as a classifier and a genetic algorithm was used to search for optimal feature sets. It is shown that most of the structural micro features studied, do possess discriminative power, which justifies their use in forensic analysis of handwriting. The results also show that the grapheme possessed significantly higher discriminating power than any of the three single characters studied, which supports the opinion that a character form is affected by its adjacent characters. [Copyright &y& Elsevier]

Abstract: Hybrid hierarchical clustering techniques which combine the characteristics of different partitional clustering techniques or partitional and hierarchical clustering techniques are interesting. In this paper, efficient bottom-up hybrid hierarchical clustering (BHHC) techniques have been proposed for the purpose of prototype selection for protein sequence classification. In the first stage, an incremental partitional clustering technique such as leader algorithm (ordered leader no update (OLNU) method) which requires only one database (db) scan is used to find a set of subcluster representatives. In the second stage, either a hierarchical agglomerative clustering (HAC) scheme or a partitional clustering algorithm—‘K-medians’ is used on these subcluster representatives to obtain a required number of clusters. Thus, this hybrid scheme is scalable and hence would be suitable for clustering large data sets and we also get a hierarchical structure consisting of clusters and subclusters and the representatives of which are used for pattern classification. Even if more number of prototypes are generated, classification time does not increase much as only a part of the hierarchical structure is searched. The experimental results (classification accuracy (CA) using the prototypes obtained and the computation time) of the proposed algorithms are compared with that of the hierarchical agglomerative schemes, K-medians and nearest neighbour classifier (NNC) methods. The proposed methods are found to be computationally efficient with reasonably good CA. [Copyright &y& Elsevier]

• We propose a new simultaneous instance and feature selection algorithm. • The method achieves better storage reduction and testing error than previous approaches. • The method is scalable to datasets with millions of features. Data reduction is becoming increasingly relevant due to the enormous amounts of data that are constantly being produced in many fields of research. Instance selection is one of the most widely used methods for this task. At the same time, most recent pattern recognition problems involve highly complex datasets with a large number of possible explanatory variables. For many reasons, this abundance of variables significantly hinders classification and recognition tasks. There are efficiency issues, too, because the speed of many classification algorithms is greatly improved when the complexity of the data is reduced. Thus, feature selection is also a widely used method for data reduction and for gaining an understanding of feature information. Although most methods address instance and feature selection separately, the two problems are interwoven, and benefits are expected from performing these two tasks jointly. However, few algorithms have been proposed for simultaneously addressing the tasks of instance and feature selection. Furthermore, most of those methods are based on complex heuristics that are very difficult to scale up even to moderately large datasets. This paper proposes a new algorithm for dealing with many instances and many features simultaneously by performing joint instance and feature selection using a simple heuristic search and several scaling-up mechanisms that can be successfully applied to datasets with millions of features and instances. In the proposed method, a forward selection search is performed in the feature space jointly with the application of standard instance selection in a constructive subspace built stepwise. Several simplifications are adopted in the search to obtain a scalable method. An extensive comparison using 95 large datasets shows the usefulness of our method and its ability to deal with millions of instances and features simultaneously. The method is able to obtain better classification performance results than state-of-the-art approaches while achieving considerable data reduction. [ABSTRACT FROM AUTHOR]

• We propose a new unsupervised feature selection based on information theory. • A single optimization problem is designed considering dependence among features. • We analyze the convergence of the proposed iterative algorithm. Many research topics present very high dimensional data. Because of the heavy execution times and large memory requirements, many machine learning methods have difficulty in processing these data. In this paper, we propose a new unsupervised feature selection method considering the pairwise dependence of features (feature dependency-based unsupervised feature selection, or DUFS). To avoid selecting redundant features, the proposed method calculates the dependence among features and applies this information to a regression-based unsupervised feature selection process. We can select small feature set with the dependence among features by eliminating redundant features. To consider the dependence among features, we used mutual information widely used in supervised feature selection area. To our best knowledge, it is the first study to consider the pairwise dependence of features in the unsupervised feature selection method. Experimental results for six data sets demonstrate that the proposed method outperforms existing state-of-the-art unsupervised feature selection methods in most cases. [ABSTRACT FROM AUTHOR]

• The research reveals the main problem of the classic null space LDA method: the intrinsic overfitting problem. • A new approach, representative null space LDA (RNLDA), is proposed to solve the overfitting problem. • Practical and efficient RNLDA algorithms and an automatic parameter setting algorithm are proposed. Null space Linear Discriminant Analysis (NLDA) was proposed twenty years ago to overcome the singularity problem of LDA in practical applications. With two decades of technique development, many Discriminative Dimensionality Reduction (DDR) methods that outperform NLDA have been proposed. This paper provides new insight into NLDA and illustrates that NLDA is much more powerful after solving its inherent problem. The main problem of NLDA is the intrinsic overfitting problem. An ideal NLDA model is proposed to analyze its overfitting problem. Based on the ideal NLDA model, a more reasonable Representative NLDA (RNLDA) method is proposed to prevent overfitting. Two simple but efficient RNLDA algorithms are proposed to implement the RNLDA method with a theoretical proof. This study theoretically analyzed and indicated that applying the classical but simple hold-out pretraining method can automatically set the only parameter to achieve high performance. Extensive experiments with eight databases demonstrate the superior performance of the RNLDA method over state-of-the-art DDR methods. [ABSTRACT FROM AUTHOR]

• A novel genetic programming (GP) based ECOC algorithm is proposed with the aid of ternary digit operators. • In the GP, every terminal node represents an ECOC column along with a feature subset, and every nonterminal node is a ternary digit operator. • Each individual is transformed into a codematrix associated with a set of feature subsets. • A local optimization algorithm is proposed to remove redundant or irrelevant columns and append necessary columns to tackle hard classes. • TOGP-ECOC outperforms other well-known ECOC algorithms on various data sets. The key to the success of an Error-Correcting Output Code (ECOC) algorithm is the effective codematrix, which represents a set of class reassignment schemes for decomposing a multiclass problem into a set of binary class problems. This paper proposes a new method, which uses Ternary digit Operators based Genetic Programming (GP) to generate effective ECOC codematrix (TOGP-ECOC for short). In our GP, each terminal node stores a ternary digit string, representing a column and a related feature subset; each non-terminal node represents a ternary digit operator, which produces a new column based on its child nodes. In this way, each individual is interpreted as an ECOC codematrix along with a set of corresponding feature subsets, serving the solution for the multiclass classification task. When a new individual is produced, a legality checking process is carried out to verify whether the transformed codematrix follows the ECOC constraints. The illegal one is corrected according to different strategies. Besides, a local optimization algorithm is designed to prune redundant columns and improve the performance of each individual. Our experiments compared TOGP-ECOC with some well known ECOC algorithms on various data sets, and the results confirm the superiority of our algorithm. Our source code is available at: https://github.com/MLDMXM2017/TOGP-ECOC. [ABSTRACT FROM AUTHOR]

• A novel tensor regression model is introduced to simultaneously capture the underlying low-rank and sparse structure of the coefficient tensor. • Unlike traditional tensor regression models that impose sparse penalty on the factor matrices of the coefficient tensor, our model directly imposes sparse penalty on the coefficient tensor. • An orthonormality constraint is imposed on the factor matrices to make our model identifiable. • We evaluate the proposed model on synthetic and real data sets. The results show that our model achieves competitive performance. Many applications in biomedical informatics deal with data in the tensor form. Traditional regression methods which take vectors as covariates may encounter difficulties in handling tensors due to their ultrahigh dimensionality and complex structure. In this paper, we introduce a novel sparse regularized Tucker tensor regression model to exploit the structure of tensor covariates and perform feature selection on tensor data. Based on Tucker decomposition of the regression coefficient tensor, we reduce the ultrahigh dimensionality to a manageable level. To make our model identifiable, we impose the orthonormality constraint on the factor matrices. Unlike previous tensor regression models that impose sparse penalty on the factor matrices of the coefficient tensor, our model directly imposes sparse penalty on the coefficient tensor to select the relevant features on tensor data. An efficient optimization algorithm based on alternating direction method of multiplier (ADMM) algorithm is designed to solve our proposed model. The performance of our model is evaluated on both synthetic and real genomic data. Experiment results on synthetic data demonstrate that our model could identify the true related signals more accurately than other state-of-the-art regression models. The analysis on genomic data of melanoma demonstrates that our model can achieve better prediction performance and identify markers with important implications. Our model and the associated studies can provide useful insights to disease or pathogenesis mechanisms, and will benefit further studies in variable selection. [ABSTRACT FROM AUTHOR]

• A binary coyote optimization algorithm for feature selection problem is proposed. • The binarization method is based on hyperbolic transfer function. • We test the proposed wrapper with the Naïve Bayes classifier using seven data sets. • The wrapper presents high training accuracy. • The wrapper shows relatively low standard deviation and subsets with few features. The Coyote Optimization Algorithm (COA) is a bio-inspired optimization algorithm based on the intelligent behavior of coyotes. COA was proposed recently and it considers the social organization of the coyotes and its adaptation to the environment in order to solve continuous optimization problems. In addition, it is a population-based algorithm and it can be classified as both, swarm intelligence and evolutionary heuristics, because contributes with a different algorithmic structure. This paper proposes a binary version of the COA, named Binary COA (BCOA) applying to select the optimal feature subset for classification, based on the hyperbolic transfer function in a wrapper model. By this way, the features are selected based on the performance evaluation of a classification algorithm. We tested the effectiveness of the BCOA wrapper with the Naïve Bayes classifier and were used seven public domain benchmark datasets to compare the proposed approach in terms of classification accuracy, number of selected features and computational cost with other state-of-art algorithms of the literature. The results shown that BCOA was able to find subsets with few features while it still performs well in terms of classification accuracy. [ABSTRACT FROM AUTHOR]

Abstract: This paper presents a novel feature selection method based on the use of a multilayer perceptron (MLP). The algorithm identifies a subset of relevant, non-redundant attributes for supervised pattern classification by estimating the relative contribution of the input units (those representing the attributes) to the output neurons (those corresponding to the problem classes). The experimental results suggest that the proposed method works well on a variety of real-world domains. [Copyright &y& Elsevier]