Search

In numerous domains, there is ample evidence indicating that a significant portion of real-world data exhibits temporal variations, such as medical research and meteorological studies. Particularly in the era of big data, not only does the size of data change dynamically but also its dimensions at an unprecedented pace. Consequently, employing traditional methods to handle such dynamic data becomes highly impractical. To address this limitation, this paper proposes an incremental feature selection algorithm tailored for dynamic feature variations. For scenarios involving an increase in features, the novel algorithm efficiently identifies a target subset with effective features. In order to showcase the efficacy of the proposed algorithm, this paper conduct experiments using four commonly used classifiers and five UCI data sets. The experimental results further validate both the feasibility and efficiency of the new approach. [ABSTRACT FROM AUTHOR]

• An analysis of lexicographic optimization for feature selection and data classification. • Cooperative co-evolution for feature selection while the classifier hyperparameters are also being optimized. • Optimization of multiple objectives using a simple evolutionary algorithm. • A new lexicographic ranking methodology to compare the results of different feature selection methods. • Quite good results for real high-dimensional datasets. This paper starts with two hypotheses. The first one is that the simultaneous optimization of the hyperparameters regulating the classifier within a wrapper method, while the best subset of features is being determined, should improve the results with respect to those obtained with a pre-parameterized classifier. The second one is that solving these two problems can be formulated as a lexicographic optimization problem, allowing the use of a simple single-objective evolutionary algorithm to solve this multi-objective problem. The fitness function is of key importance for such wrapper methods. It is responsible for guiding the search towards potentially good solutions and it also consumes most of the runtime. Having these issues in mind, this paper also proposes a new lexicographic fitness function, designed to minimize the runtime of the algorithm and also to avoid over-fitting. Furthermore, the execution time and the quality of the results obtained by the wrapper procedure also depend on some algorithmic hyperparameters: the similarity thresholds used when comparing two different solutions lexicographically and the percentage of data samples used for validation during the training process. Thus, an experimental analysis has been carried out to find adequate values for these hyperparameters. Finally, the lexicographic cooperative co-evolutionary wrapper approach, using the new fitness function proposed in this paper, has been tested with several datasets belonging to the University of California, Irvine (UCI) repository and also with some real high-dimensional datasets, obtaining quite good results, compared to other state-of-the-art wrapper methods. The comparison has also been made lexicographically, with a new methodology proposed in this paper. [ABSTRACT FROM AUTHOR]

To improve the classification performance of existing adaptive boosting (AdaBoost) based algorithms effectively, an improved AdaBoost algorithm based on misclassified samples oriented feature selection and weighted non-negative matrix factorization (WNMF) is proposed in this paper. Firstly, in order to consider the effects of sample weights, a misclassified samples oriented feature selection (called MOFS) is proposed to select the most discriminative features which occur in the samples with high weights. Secondly, the explicit features and the part-based features of the training samples are both considered, and the WNMF algorithm is introduced and combined with MOFS to reduce the dimension of the training sample set. Finally, the concept of misclassification degree is introduced and a fine grained sample weight updating method is proposed to distinguish the samples with different misclassification degrees. Numerical experiments show that the proposed MOFS method achieves higher accuracy compared to traditional feature selection methods, and the proposed MOFS and WNMF based AdaBoost method obtains significant improvement on classification accuracy when comparing with typical existing AdaBoost based algorithms using different classifiers. [ABSTRACT FROM AUTHOR]

• An advanced shuffled frog leaping algorithm (DSSRLFLA) is developed for model evaluation and feature selection. • A dynamic step size adjustment strategy based on historical information is designed to balance exploration and exploitation. • Specular reflection learning mechanism increases the possibility of searching for valid solutions and enhances the search ability. • An improved simulated annealing strategy is executed for each memetic to improve the efficiency of local exploitation. • DSSRLFLA can obtain higher quality optimal solutions compared to some competitive algorithms. The shuffled frog leaping algorithm is a new optimization algorithm proposed to solve the combinatorial optimization problem, which effectively combines the memetic algorithm based on a memetic algorithm and the particle swarm algorithm based on population behavior. The algorithm is widely used because it is easy to implement and requires few parameters to be adjusted. However, there are still some characteristics of this method that need to be improved because it is easy to fall into local optimization or poor search ability. To alleviate this limitation, a new version of the improved SFLA is proposed in this paper, which incorporates a dynamic step size adjustment strategy based on historical information, a specular reflection learning mechanism, and a simulated annealing mechanism based on chaotic mapping and levy flight. Firstly, the dynamic step size adjustment strategy based on historical information effectively helps to balance local exploration and global exploitation and alleviates the problem of falling into local optimum. Second, the specular reflection learning mechanism increases the possibility of searching for valid solutions in feasible domains and enhances the search ability of individuals in the population. Finally, an improved simulated annealing strategy is executed for each memetic, which improves the efficiency of local exploitation. In order to test the performance of the proposed algorithm, 31 test functions were selected from IEEE CEC2014 and 23 essential benchmark functions, and comparative experiments were carried out from the two dimensions of 30 and 100. Furthermore, a series of competing algorithms are selected, which involve nine classical standard algorithms, including PSO, BA, SSA, FA, SCA, WOA, GWO, MFO, and SFLA, as well as, six well-known improved algorithms, including LSFLA, DDSFLA, GOTLBO, ALCPSO, BLPSO, CLPSO. Furthermore, Wilcoxon signed-rank test and Friedman test are used as testing tools to illustrate the scalability of the proposed algorithm. From the analysis of the results, it can be seen that this proposed method has been effectively improved concerning stability and the quality of the optimal solution obtained from the search, both in low and high dimensions, and the ability to jump out of the local optimum has also been enhanced. In addition, to prove that this method has a reliable performance in discrete problems and continuous problems, DSSRLFLA is mapped into a discrete space, and 24 UCI data sets are also selected to evaluate the performance of the new feature selection method. The experimental results illustrate that this improved method can obtain fewer features and higher classification accuracy than some well-known feature selection methods. [ABSTRACT FROM AUTHOR]

The technology for Visual Odometry (VO) that estimates the position and orientation of the moving object through analyzing the image sequences captured by on-board cameras, has been well investigated with the rising interest in autonomous driving. This paper studies monocular VO from the perspective of Deep Learning (DL). Unlike most current learning-based methods, our approach, called DeepAVO, is established on the intuition that features contribute discriminately to different motion patterns. Specifically, we present a novel four-branch network to learn the rotation and translation by leveraging Convolutional Neural Networks (CNNs) to focus on different quadrants of optical flow input. To enhance the ability of feature selection, we further introduce an effective channel-spatial attention mechanism to force each branch to explicitly distill related information for specific Frame to Frame (F2F) motion estimation. Experiments on various datasets involving outdoor driving and indoor walking scenarios show that the proposed DeepAVO outperforms the state-of-the-art monocular methods by a large margin, demonstrating competitive performance to the stereo VO algorithm and verifying promising potential for generalization. [ABSTRACT FROM AUTHOR]

Fuzzy rough set is a theoretical framework of fuzzy uncertainty management, and discernibility matrix offers a mathematical foundation for algorithm construction of feature learning. The approaches of fuzzy rough set and discernibility matrix have been successfully applied in single-label learning. However, few works have been done on investigating the foundation of fuzzy rough discernibility matrix on multi-label data. There will be two pivotal problems to be addressed when using fuzzy rough discernibility matrix for multi-label data analysis. One is how to extract sample-level and label-level correlations; and the other is how to utilize the discernibility matrix for algorithm construction. For this reason, in this paper the fuzzy rough discrimination matrix is introduced to deal with the problem of multi-label feature selection. First, the significance of labels in the label space is captured based on the label correlation. Labels with different significances contribute to different weights for measuring the similarity between samples. Hence, a sample similarity matrix in the label space is computed based on the label weighting strategy. Then, a framework of a fuzzy decision system is formalized, in which the discernibility matrix of fuzzy rough sets is introduced as a foundation to evaluate the sample discrimination ability of features. Under the discernibility matrix criterion, a multi-label learning algorithm is developed to select discriminative features from multi-label data. A series of experimental analysis verifies the effectiveness and efficiency of the proposed method. [ABSTRACT FROM AUTHOR]

Sparse Bayesian learning (SBL) provides state-of-the-art performance in accuracy, sparsity and probabilistic prediction for classification. In SBL, the regularization priors are automatically determined that avoids an exhaustive hyperparameter selection by cross-validation. However, scalability to large problems is a drawback of SBL because of the inversion of a potentially enormous covariance matrix for updating the regularization priors in every iteration. This paper develops an approximate SBL algorithm called ARP-SBL, where the regularization priors are approximated without inverting the covariance matrix. Therefore, our approach can easily scale up to problems with large data size or feature dimension. It alleviates the long training time and high memory complexity in SBL. Based on ARP-SBL, two scalable nonlinear SBL models: scalable relevance vector machine (ARP-RVM) and scalable sparse Bayesian extreme learning machine (ARP-SBELM) are developed for problems of large data size and large feature size respectively. Experiments on a variety of benchmarks have shown that the proposed models are with competitive accuracy compared to existing methods while i) converging faster; ii) requiring thousands of times less memory; iii) without exhaustive regularized hyperparameter selection; and iv) easily scaling up to large data size and high dimensional features. [ABSTRACT FROM AUTHOR]

This paper investigates a new bacterial colony-based feature selection algorithm to improve the classification accuracy of customers for personalized products recommendation. An attribute learning strategy is developed in this study to update the feature related population. Specifically, the features can be weighted according to their historic contributions to both the individual- and group-based subsets. Additionally, the frequency of appearance is also recorded for the feature candidates to improve the diversity of feature distribution and avoid the over-fitting. Based on the weight-based feature indexes and frequency of appearance records, the performance of feature subsets are enhanced by replacing the features being repeatedly appeared in a same vector. To explore the feasibility of the proposed method for the missing feature problems, the objective of the optimization is to minimize the classification error using the acceptable number of features. K-Nearest Neighbor is employed as the learning technique to cooperate with the proposed feature selection method. The effectiveness of the proposed feature selection method is demonstrated by performing test on the datasets from UCI machine learning repository and real-world data from Amazon customer reviews of products. Compared with other seven feature selections methods, the proposed feature selection algorithm outperforms the other algorithms by achieving higher classification accuracy rate using smaller features. [ABSTRACT FROM AUTHOR]

Abstract Feature (gene) selection of microarray data is a very important and challenging task. This paper proposes a new feature selection method, partial maximum correlation information (PMCI), for microarray data classification. The proposed method extracts several orthogonal components from the feature space to evaluate the importance of each feature. In PMCI method, the component extraction is based on the correlation between feature space and class coding space. Meanwhile, this paper also provides a new generic class-encoding scheme for multi-class problems. This type of encoding scheme can ensure the smooth implementation for PMCI and has a clear geometric meaning. To validate the performance of the proposed method, this paper selects six state-of-the-art algorithms for comparison based on ten widely used microarray benchmark datasets for experiment. The experimental results show that PMCI is an effective feature selection method with good performance and lower time complexity. [ABSTRACT FROM AUTHOR]

Abstract Dealing with high-dimensional dataset has always been an important problem and feature selection is one of useful tools. In this paper, we develop a new filter based supervised feature selection method by combining Trace Ratio Criterion of Linear Discriminant Analysis (TRC-LDA) and group sparsity regularization. The filter based supervised feature selection method is a classifier-independent method while the TRC-LDA criterion is a recently developed criterion for dimensionality reduction that can well preserve discriminative information of dataset. However, there are seldom methods by utilizing TRC-LDA criterion for feature selection. On the other hand, imposing the l2,0-norm to the projection matrix of TRC-LDA will force some rows in it to be zero while keep other rows nonzero making the index of nonzero rows to be the selected features, however, l 2,0 -nom minimizing problem is NP-hard and intractable. To solve the above problem, in this paper, we develop a new method, namely, Trace Ratio Criterion Discriminative Feature Selection (TRC-DFS), for feature selection. The proposed TRC-DFS has imposed l 2,1 -norm, i.e. an approximation of l 2,0 -norm, to the projection matrix W of TRC-LDA to achieve feature selection. As a result, the proposed TRC-DFS can both achieve feature selection as well as capture the discriminative structure of data. We also extend the proposed method with l 2, p -norm (0 < p < 1) regularization to grasp more sparse property and develop an iterative approach to calculate the optimal solution, which is rigorously proved to be converged. Extensive simulations based on several real-world datasets verify the effectiveness of the proposed methods. [ABSTRACT FROM AUTHOR]

Hyperparameter optimization is a challenging process that has the potential to improve machine learning algorithms. Since it creates a remarkable computational burden for machine learning tasks, there have been few works coping with tuning strategies of a specific algorithm. In this paper, an improved Stochastic Gradient Descent (SGD) based on Fisher Maximization is developed for tuning hyperparameters of an Echo State Network (ESN) which has a wide range of applications. The results of the method are then compared with those of traditional Gradient Descent and Grid Search. According to the obtained results; 1) The scale of the data sets greatly affects the reliability of hyperparameter optimization results; 2) Feature selection is critical in terms of mean error of training when hyperparameter optimization is applied on some methods such as ESN; 3) SGD falls in a good local minima if Fisher Maximization is performed to find a good starting point. [ABSTRACT FROM AUTHOR]

Traditional unsupervised feature selection methods usually construct a fixed similarity matrix. This matrix is sensitive to noise and becomes unreliable, which affects the performance of feature selection. The researches have shown that both the global reconstruction information and local structure information are important for feature selection. To solve the above problem effectively and make use of the global and local information of data simultaneously, a novel algorithm is proposed in this paper, called subspace learning for unsupervised feature selection via adaptive structure learning and rank approximation (SLASR). Specifically, SLASR learns the manifold structure adaptively, thus the preserved local geometric structure can be more accurate and more robust to noise. As a result, the learning of the similarity matrix and the low-dimensional embedding is completed in one step, which improves the effect of feature selection. Meanwhile, SLASR adopts the matrix factorization subspace learning framework. By minimizing the reconstruction error of subspace learning residual matrix, the global reconstruction information of data is preserved. Then, to guarantee more accurate manifold structure of the similarity matrix, a rank constraint is used to constrain the Laplacian matrix. Additionally, the l 2,1/2 regularization term is used to constrain the projection matrix to select the most sparse and robust features. Experimental results on twelve benchmark datasets show that SLASR is superior to the six comparison algorithms from the literature. [ABSTRACT FROM AUTHOR]

• We propose structured OMP (SOMP), adding a flexible weight to adjust the correlations of selected features. • We theoretically and experimentally analyze the influence of the weight on feature selection. • SOMP exhibits robust and good performance on various datasets with distinct classifiers. Feature selection techniques are widely adopted to deal with high-dimensional data. One of the most popular algorithms is orthogonal matching pursuit (OMP). OMP tends to select only one from correlated features, because the next selected feature relies on a residual that is orthogonal to previous selected features. Practically, it is very common that some strongly correlated features are significantly beneficial to the classification performance. Hence, OMP might exhibit poor performance in these cases. To alleviate this issue, in this paper, we propose a novel yet simple feature selection algorithm, namely structured OMP (SOMP), by considering correlations among distinct features for multi-class classification. Specifically, we extend OMP to tackle multi-class tasks, and add a flexible weight to adjust the correlation among selected features in order to find better feature combination for classification. Meanwhile, we theoretically and experimentally analyze the effects of the added weight on feature selection. Extensive experiments on multiple benchmark databases demonstrate the effectiveness and efficiency of SOMP. [ABSTRACT FROM AUTHOR]

Abstract A Deep Boltzmann Machine is a model of a Deep Neural Network formed from multiple layers of neurons with nonlinear activation functions. The structure of a Deep Boltzmann Machine enables it to learn very complex relationships between features and facilitates advanced performance in learning of high-level representation of features, compared to conventional Artificial Neural Networks. Feature selection at the input level of Deep Neural Networks has not been well studied, despite its importance in reducing the input features processed by the deep learning model, which facilitates understanding of the data. This paper proposes a novel algorithm, Deep Feature Selection (Deep-FS), which is capable of removing irrelevant features from large datasets in order to reduce the number of inputs which are modelled during the learning process. The proposed Deep-FS algorithm utilizes a Deep Boltzmann Machine, and uses knowledge which is acquired during training to remove features at the beginning of the learning process. Reducing inputs is important because it prevents the network from learning the associations between the irrelevant features which negatively impact on the acquired knowledge of the network about the overall distribution of the data. The Deep-FS method embeds feature selection in a Restricted Boltzmann Machine which is used for training a Deep Boltzmann Machine. The generative property of the Restricted Boltzmann Machine is used to reconstruct eliminated features and calculate reconstructed errors, in order to evaluate the impact of eliminating features. The performance of the proposed approach was evaluated with experiments conducted using the MNIST, MIR-Flickr, GISETTE, MADELON and PANCAN datasets. The results revealed that the proposed Deep-FS method enables improved feature selection without loss of accuracy on the MIR-Flickr dataset, where Deep-FS reduced the number of input features by removing 775 features without reduction in performance. With regards to the MNIST dataset, Deep-FS reduced the number of input features by more than 45%; it reduced the network error from 0.97% to 0.90%, and also reduced processing and classification time by more than 5.5%. Additionally, when compared to classical feature selection methods, Deep-FS returned higher accuracy. The experimental results on GISETTE, MADELON and PANCAN showed that Deep-FS reduced 81%, 57% and 77% of the number of input features, respectively. Moreover, the proposed feature selection method reduced the classifier training time by 82%, 70% and 85% on GISETTE, MADELON and PANCAN datasets, respectively. Experiments with various datasets, comprising a large number of features and samples, revealed that the proposed Deep-FS algorithm overcomes the main limitations of classical feature selection algorithms. More specifically, most classical methods require, as a prerequisite, a pre-specified number of features to retain, however in Deep-FS this number is identified automatically. Deep-FS performs the feature selection task faster than classical feature selection algorithms which makes it suitable for deep learning tasks. In addition, Deep-FS is suitable for finding features in large and big datasets which are normally stored in data batches for faster and more efficient processing. [ABSTRACT FROM AUTHOR]

Abstract Feature selection is used in many applications in machine learning and bioinformatics. As a popular approach, feature selection can be implemented in the filter-manner based on the sparse solution of the l 1 regularization. Most study of the l 1 regularization concentrates on investigating the iteration solution of the problem or focuses on adapting sparsity to different applications. It is necessary to explore more deeply about how the sparsity learned with the l 1 regularization contributes to feature selection. In this paper, we make an effort to analyze the role of the l 1 regularization in feature selection from the perspective of information theory. We discover that the l 1 regularization contributes to minimizing the redundancy in feature selection. To avoid the complex computation of the l 1 optimization, we propose a novel feature selection algorithm, i.e. the Laplacian regularization based on mutual information (LRMI), which realizes the minimization of the redundancy in a new way, and incorporates the l 2 norm to achieve automatic grouping. Extensive experimental results demonstrate the superiority of LRMI over several traditional l 1 regularization based feature selection algorithms with less time consumption. [ABSTRACT FROM AUTHOR]

Feature selection has playing an important role in many pattern recognition and machine learning applications, where meaningful features are desired to be extracted from high dimensional raw data and noisy ones are expected to be eliminated. l 2,1 -norm regularization based Robust Feature Selection (RFS) has extracted a lot of attention due to its efficiency and high performance of joint sparsity. In this paper, we propose a more general framework for robust and discriminative multi-class feature selection. Four types of weighting, which are based on correlation information between features and labels, are adopted to strengthen the discriminative performance of l 2,1 -norm joint sparsity. F-norm regularization, which is extended from multi-class Elastic Net, is added to improve the stability of the method. An efficient algorithm and its corresponding convergence proof are provided. Experimental results on several two-class and multi-class datasets are performed to verify the effectiveness of the proposed feature selection method. [ABSTRACT FROM AUTHOR]

DNA microarray technology provides an efficient way to diagnose cancer. However, microarray gene expression data face the challenges of class imbalance and high dimension. The class imbalance problem usually leads to inaccurate results when using traditional feature selection and classification algorithms. Due to fast learning speed and good classification performance, extreme learning machine (ELM) has become one of the best classification algorithms and weighted ELM has been recently presented to deal with the class imbalance. However, they ignored the negative impact of imbalanced feature set. This paper proposes a hybrid method based on WELM to handle the multi class imbalance problem of cancer microarray data at both feature and algorithmic levels. At feature level, a corrected feature subset is searched for each class using class oriented feature selection method, so that the features correlated with the minority class are explicitly selected. At algorithmic level, WELM is further modified to strengthen the input nodes with high discrimination power, and an ensemble model is trained to improve the generalization. That is, multiple modified WELM models are trained on the datasets characterized by different feature subsets; in order to encourage the ensemble diversity, the models with low dissimilarity are removed and the reserved ones are combined as an ensemble model. The experiments are conducted on eight gene expression datasets with multiple cancer types and classification results show that our method significantly outperforms ELM and several recent works. [ABSTRACT FROM AUTHOR]

Naïve Bayes, k-nearest neighbors, Adaboost, support vector machines and neural networks are five among others commonly used text classifiers. Evaluation of these classifiers involves a variety of factors to be considered including benchmark used, feature selections, parameter settings of algorithms, and the measurement criteria employed. Researchers have demonstrated that some algorithms outperform others on some corpus, however, inconsistency of human labeling and high dimensionality of feature spaces are two issues to be addressed in text categorization. This paper focuses on evaluating the five commonly used text classifiers by using an automatically generated text document collection which is labeled by a group of experts to alleviate subjectivity of human category assignments, and at the same time to examine the influence of the number of features on the performance of the algorithms. [ABSTRACT FROM AUTHOR]

Feature selection is important and necessary especially for processing large scale data. Existing feature selection methods generally compute a discriminant value with respect to class variable for a feature to indicate its classification ability. Such a scalar value can hardly reveal the multi-faceted classification abilities of a feature for the different subproblems in a classification task. In this paper, an effective way is proposed for feature selection based on measurement of ability to classify subproblems and discrimination structure complementarity of features. The classification abilities of a feature for different subproblems are calculated respectively. Hence for the feature, a discrimination structure vector representing its classification abilities for all subproblems can be obtained. In feature selection, the features, which can individually classify as many subproblems as possible, are firstly evaluated and selected. Subsequently, their complementary features are selectively chosen, which can complementarily classify the subproblems that the selected features cannot classify. Two algorithms are designed for progressively selecting features, by firstly eliminating irrelevant features and then abandoning redundant features based on discrimination structure complementarity. The proposed algorithms are compared with some related methods for feature selection on some open gene expression datasets and UCI datasets. Experimental results demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Feature selection for large-scale data sets has been conceived as a very important data preprocessing step in the area of machine learning. Data sets in real databases usually take on hybrid forms, i.e., the coexistence of categorical and numerical data. In this paper, based on the idea of decomposition and fusion, an efficient feature selection approach for large-scale hybrid data sets is studied. According to this approach, one can get an effective feature subset in a much shorter time. By employing two common classifiers as the evaluation function, experiments have been carried out on twelve UCI data sets. The experimental results show that the proposed approach is effective and efficient. [ABSTRACT FROM AUTHOR]

Linear Discriminant Analysis (LDA) is a standard tool for classification and dimension reduction in many applications. However, the problem of high dimension is still a great challenge for the classical LDA. In this paper we consider the supervised pattern classification in the high dimensional setting, in which the number of features is much larger than the number of observations and present a novel approach to the sparse optimal scoring problem using the zero-norm. The difficulty in treating the zero-norm is overcome by using appropriate continuous approximations such that the resulting problems are solved by alternating schemes based on DC (Difference of Convex functions) programming and DCA (DC Algorithms). The experimental results on both simulated and real datasets show the efficiency of the proposed algorithms compared to the five state-of-the-art methods. [ABSTRACT FROM AUTHOR]

In recent years, there has been a great interest in prostate segmentation, which is an important and challenging task for CT image guided radiotherapy. In this paper, a learning-based segmentation method via joint transductive feature selection and transductive regression is presented, which incorporates the physician׳s simple manual specification (only taking a few seconds), to aid accurate segmentation, especially for the case with large irregular prostate motion. More specifically, for the current treatment image, experienced physician is first allowed to manually assign the labels for a small subset of prostate and non-prostate voxels, especially in the first and last slices of the prostate regions. Then, the proposed method follows the two step: in prostate-likelihood estimation step, two novel algorithms, tLasso and wLapRLS, will be sequentially employed for transductive feature selection and transductive regression, respectively, aiming to generate the prostate-likelihood map. In multi-atlases based label fusion step, the final segmentation result will be obtained according to the corresponding prostate-likelihood map and the previous images of the same patient. The proposed method has been substantially evaluated on a real prostate CT dataset including 24 patients with 330 CT images, and compared with several state-of-the-art methods. Experimental results show that the proposed method outperforms the state-of-the-arts in terms of higher Dice ratio, higher true positive fraction, and lower centroid distances. Also, the results demonstrate that simple manual specification can help improve the segmentation performance, which is clinically feasible in real practice. [ABSTRACT FROM AUTHOR]

Due to the influence of environment, data obtained in real world are not completely reliable sometimes. This paper focuses on tackling the feature selection problem with unreliable data. First, the problem is formulated as an multi-objective optimization one with two objectives: the reliability and the classification accuracy. Then, an effective multi-objective feature selection algorithm based on bare-bones particle swarm optimization is proposed by incorporating two new operators. One is a reinforced memory strategy, which is designed to overcome the degradation phenomenon of particles. Another is a hybrid mutation, which is designed to improve the search ability of the proposed algorithm. Finally, two state-of-the-art multi-objective optimization algorithms are also applied to this kind of problem, and comparison results suggest that the proposed algorithm is highly competitive for the feature selection problem with unreliable data. [ABSTRACT FROM AUTHOR]

Feature selection is an important preprocessing step in data mining. Mutual information-based feature selection is a kind of popular and effective approaches. In general, most existing mutual information-based techniques are greedy methods, which are proven to be efficient but suboptimal. In this paper, mutual information-based feature selection is transformed into a global optimization problem, which provides a new idea for solving feature selection problems. First, a single-objective feature selection algorithm combining relevance and redundancy is presented, which has well global searching ability and high computational efficiency. Furthermore, to improve the performance of feature selection, we propose a multi-objective feature selection algorithm. The method can meet different requirements and achieve a tradeoff among multiple conflicting objectives. On this basis, a hybrid feature selection framework is adopted for obtaining a final solution. We compare the performance of our algorithm with related methods on both synthetic and real datasets. Simulation results show the effectiveness and practicality of the proposed method. [ABSTRACT FROM AUTHOR]

Cancer is curable if it can be detected early. One way to detect cancer is by analyzing the change in expression of genes in the suspected tissue. Serial analysis of gene expression (SAGE) is a sequencing technique used for measuring the expression levels of genes. Cancer detection problem can be posed as binary classification problem like whether a tissue is cancerous or normal. SAGE libraries contain expression levels of thousands of genes which are the features. It is impossible to consider all these features for classification and also the general feature selection algorithms are not efficient with this data. In this paper, closed frequent itemset mining is proposed as a feature selection technique for identifying a small set of features which can distinguish the two classes efficiently. The performance of the proposed technique is evaluated on SAGE data related to breast tissue and a group of 26 genes are selected as best features. Two well known classifiers, extreme learning machine (ELM) and support vector machine (SVM), are used to evaluate the effectiveness of the selected features in classification and found that the proposed method works well with these classifiers. [ABSTRACT FROM AUTHOR]

Bayesian Ying–Yang (BYY) harmony learning system is a powerful tool for statistical learning. Via the BYY harmony leaning of finite mixtures, model selection, i.e., the selection of an appropriate number of components for the mixture, can be made automatically during parameter learning on a given dataset. In this paper, an adaptive gradient BYY harmony learning algorithm is proposed for log-normal mixtures to implement parameter learning with automated model selection. It is demonstrated by the experiments on both synthetic and real-world datasets that the proposed BYY harmony learning algorithm not only has the ability of automated model selection, but also leads to a rather good estimation of the parameters in the original log-normal mixture. [ABSTRACT FROM AUTHOR]

Named Data Networking (NDN) is a promising network architecture being considered as a possible replacement for the current IP-based (host-centric) Internet infrastructure. NDN can overcome the fundamental limitations of the current Internet, in particular, Denial-of-Service (DoS) attacks. However, NDN can be subject to new type of DoS attacks namely Interest flooding attacks and content poisoning. These types of attacks exploit key architectural features of NDN. This paper presents a new intelligent hybrid algorithm for proactive detection of DoS attacks and adaptive mitigation reaction in NDN. In the detection phase, a combination of multiobjective evolutionary optimization algorithm with PSO in the context of the RBF neural network has been applied in order to improve the accuracy of DoS attack prediction. Performance of the proposed hybrid approach is also evaluated successfully by some benchmark problems. In the adaptive reaction phase, we introduced a framework for mitigating DoS attacks based on the misbehaving type of network nodes. The evaluation through simulations shows that the proposed intelligent hybrid algorithm (proactive detection and adaptive reaction) can quickly and effectively respond and mitigate DoS attacks in adverse conditions in terms of the applied performance criteria. [ABSTRACT FROM AUTHOR]

Abstract: The power of common spatial pattern (CSP) has been widely validated in electroencephalogram (EEG) based brain–computer interface (BCI). However, its effectiveness is highly dependent on subject-specific time segment, channel configuration and frequency band. Hence, the preprocessing procedure of CSP algorithm is critical to enhance the performance of BCI system. This paper proposes a feature extraction and selection method based on common spatial and spectral pattern for motor imagery brain–computer interface (BCI). We formulate the optimization of spatial spectral patterns, channel configuration and time segment as maximizing the proposed criterions including mutual information algorithm, Fisher ratio algorithm and wrapper method. The proposed method is evaluated on single trial EEG from dataset IVa of BCI competition III. The results show that best features are selected by a wrapper method and these features in cross-validation yield better performance compared to most of the reported results. [Copyright &y& Elsevier]

Abstract: Feature selection is one of the core issues in designing pattern recognition systems and has attracted considerable attention in the literature. Most of the feature selection methods in the literature only handle relevance and redundancy analysis from the point of view of the whole class, which neglects the relation of features and the separate classes. In this paper, we propose a novel feature selection framework to explicitly handle the relevance and redundancy analysis for each class label. Then we propose two simple and effective feature selection algorithms based on this framework and Kullback–Leibler divergence. An empirical study is conducted to evaluate the efficiency and effectiveness of our algorithms comparing with five representative feature selection algorithms. Empirical results show that our proposed algorithms are efficient and outperform the selected algorithms in most cases, and show the superiority of our proposed feature selection framework. [Copyright &y& Elsevier]

Abstract: Feature selection is an important preprocessing task for many machine learning and pattern recognition applications, including regression and classification. Missing data are encountered in many real-world problems and have to be considered in practice. This paper addresses the problem of feature selection in prediction problems where some occurrences of features are missing. To this end, the well-known mutual information criterion is used. More precisely, it is shown how a recently introduced nearest neighbors based mutual information estimator can be extended to handle missing data. This estimator has the advantage over traditional ones that it does not directly estimate any probability density function. Consequently, the mutual information may be reliably estimated even when the dimension of the space increases. Results on artificial as well as real-world datasets indicate that the method is able to select important features without the need for any imputation algorithm, under the assumption of missing completely at random data. Moreover, experiments show that selecting the features before imputing the data generally increases the precision of the prediction models, in particular when the proportion of missing data is high. [Copyright &y& Elsevier]

Abstract: Structural health monitoring aims to detect damages in man-made engineering structures by monitoring changes in their vibration response. Unsupervised learning algorithms can be used to obtain a model of the undamaged condition and detect which new samples of the structure are not in agreement with it. However, in real structures with a sensor network configuration, the number of candidate features usually becomes large. Therefore, complexity increases and it is necessary to perform feature selection and/or dimensionality reduction to achieve good detection accuracy. In this paper, we propose to exploit the three-way structure of data and apply a true multi-way data analysis algorithm: Parallel Factor Analysis. A simple model is obtained and used to train novelty detectors. The methods are tested both with real and simulated structural data to assess that the three-way analysis can be successfully used in structural health monitoring. Furthermore, the usefulness of the approach for feature selection is also analyzed. [Copyright &y& Elsevier]

Abstract: In machine learning the so-called curse of dimensionality, pertinent to many classification algorithms, denotes the drastic increase in computational complexity and classification error with data having a great number of dimensions. In this context, feature selection techniques try to reduce dimensionality finding a new more compact representation of instances selecting the most informative features and removing redundant, irrelevant, and/or noisy features. In this paper, we propose a filter-based feature selection method for working in the multiple-instance learning scenario called ReliefF-MI; it is based on the principles of the well-known ReliefF algorithm. Different extensions are designed and implemented and their performance checked in multiple instance learning. ReliefF-MI is applied as a pre-processing step that is completely independent from the multi-instance classifier learning process and therefore is more efficient and generic than wrapper approaches proposed in this area. Experimental results on five benchmark real-world data sets and 17 classification algorithms confirm the utility and efficiency of this method, both statistically and from the point of view of execution time. [Copyright &y& Elsevier]

Abstract: During the last few years, nonparallel plane classifiers, such as Multisurface Proximal Support Vector Machine via Generalized Eigenvalues (GEPSVM), and Least Squares TWSVM (LSTSVM), have attracted much attention. However, there are not any modifications of them that have been presented to automatically select the input features. This motivates the rush towards new classifiers. In this paper, we develop a new nonparallel plane classifier, which is designed for automatically selecting the relevant features. We first introduce a Tikhonov regularization (TR) term that is usually used for regularizing least squares into the LSTSVM learning framework, and then convert this formulation to a linear programming (LP) problem. By minimizing an exterior penalty (EP) problem of the dual of the LP formulation and using a fast generalized Newton algorithm, our method yields very sparse solutions, such that it generates a classifier that depends on only a smaller number of input features. In other words, this approach is capable of suppressing input features. This makes the classifier easier to store and faster to compute in the classification phase. Lastly, experiments on both toy and real problems disclose the effectiveness of our method. [Copyright &y& Elsevier]

Abstract: This paper presents a new feature selection (FS) algorithm based on the wrapper approach using neural networks (NNs). The vital aspect of this algorithm is the automatic determination of NN architectures during the FS process. Our algorithm uses a constructive approach involving correlation information in selecting features and determining NN architectures. We call this algorithm as constructive approach for FS (CAFS). The aim of using correlation information in CAFS is to encourage the search strategy for selecting less correlated (distinct) features if they enhance accuracy of NNs. Such an encouragement will reduce redundancy of information resulting in compact NN architectures. We evaluate the performance of CAFS on eight benchmark classification problems. The experimental results show the essence of CAFS in selecting features with compact NN architectures. [ABSTRACT FROM AUTHOR]

Abstract: Mass spectrometry (MS)-based proteomics has been established as a standard way for biomarker discovery and early detection of disease from the proteome level. However, the data generated by MS technology are noisy, have a lot of redundancy, and most importantly have low sample-to-dimension ratio. Therefore, successful analysis of MS data heavily relies on the algorithmic and data mining techniques used to extract information. In this paper, we briefly discuss some of the commonly used algorithms applied to mass-to-charge (m/z) feature selection, and propose a k-means clustering based hybrid algorithm to address the high-dimension and high-correlation issues associated with this task. Our k-means clustering based hybrid algorithm incorporates the advantages of both filter- and wrapper-based feature selection algorithms. A special iterative procedure is introduced to overcome the instability of k-means clustering and genetic algorithm. By comparing the proposed hybrid system with several popular m/z feature selection algorithms using 10 different classifiers, we show that the proposed algorithm is able to select m/z features with lower correlation while achieving higher sample classification accuracies. The m/z feature selection results also indicate that the proposed hybrid algorithm is very stable despite its stochastic nature. [ABSTRACT FROM AUTHOR]

Abstract: This paper introduces a novel conservative feature subset selection method with incomplete data sets. The method is conservative in the sense that it selects the minimal subset of features that renders the rest of the features independent of the target (the class variable) without making any assumption about the missing data mechanism. This is achieved in the context of determining the Markov blanket of the target that reflects the worst-case assumption about the missing data mechanism, including the case when data are not missing at random. An application of the method on synthetic and real-world incomplete data is carried out to illustrate its practical relevance. The method is compared against state-of-the-art approaches such as the expectation–maximization (EM) algorithm and the available case technique. [Copyright &y& Elsevier]

Abstract: Feature selection plays an important role in pattern classification. Its purpose is to remove redundant features from data set as many as possible. The presence of useless features may not only deteriorate the performance of learning algorithms, but also obscure important information (e.g., intrinsic structure) behind data. Along with new and emerging techniques, data sets in many domains are becoming larger and larger and many irrelevant features are often prevailing in these data sets. This, however, poses great challenges to traditional learning algorithms, such as low efficiency and over-fitting. Thus, it becomes apparent that an efficient technique is needed to eliminate redundant or irrelevant features from the data sets. Currently, many endeavors to cope with this problem have been attempted and various outstanding feature selection methods have been proposed. Unlike other selection methods, in this paper we propose a general scheme of boosting feature selection method using information metric. The primary characteristic of our method is that it exploits weight of data to select salient features. Furthermore, the weight of data will be dynamically changed after each candidate feature has been selected. Thus, the information criteria used in feature selector can exactly represent the relevant degree between features and the class labels. As a result, the selected feature subset has maximal relevance to the class labels. Simulation studies carried out on UCI data sets show that the classification performance achieved by our proposed method is better than those of other selection methods in most cases. [Copyright &y& Elsevier]

Abstract: In this paper, one novel genetic algorithm dynamic chain-like agent genetic algorithm (CAGA) is proposed for solving global numerical optimization problem and feature selection problem. The CAGA combines the chain-like agent structure with dynamic neighboring genetic operators to get higher optimization capability. An agent in chain-like agent structure represents a candidate solution to the optimization problem. Any agent interacts with neighboring agents to evolve. With dynamic neighboring genetic operators, they compete and cooperate with their neighbors, and can use knowledge to increase energies. Global numerical optimization problem and feature selection problem are the most important problems for evolutionary algorithm, especially for genetic algorithm. Hence, the experiments of global numerical optimization and feature selection are necessary to verify the performance of genetic algorithms. Corresponding experiments have been done and show that CAGA is suitable for real coding and binary coding optimization problems, and has more precise and more stable optimization results. [Copyright &y& Elsevier]

Abstract: In many computer vision tasks like face recognition and image retrieval, one is often confronted with high-dimensional data. Procedures that are analytically or computationally manageable in low-dimensional spaces can become completely impractical in a space of several hundreds or thousands dimensions. Thus, various techniques have been developed for reducing the dimensionality of the feature space in the hope of obtaining a more manageable problem. The most popular feature selection and extraction techniques include Fisher score, Principal Component Analysis (PCA), and Laplacian score. Among them, PCA and Laplacian score are unsupervised methods, while Fisher score is supervised method. None of them can take advantage of both labeled and unlabeled data points. In this paper, we introduce a novel semi-supervised feature selection algorithm, which makes use of both labeled and unlabeled data points. Specifically, the labeled points are used to maximize the margin between data points from different classes, while the unlabeled points are used to discover the geometrical structure of the data space. We compare our proposed algorithm with Fisher score and Laplacian score on face recognition. Experimental results demonstrate the efficiency and effectiveness of our algorithm. [Copyright &y& Elsevier]

Abstract: The gene expression data obtained from microarrays have shown useful in cancer classification problems. DNA microarray data have extremely high dimensionality compared with the small number of available samples. An important step in microarray studies is to select a small number of features (genes) for cancer classifications. In this paper, we propose a novel feature selection algorithm to identify a small set of genes from microarray data for cancer classification. This new wrapper method uses margins to evaluate subsets of genes; bootstrap methods are applied in genetic search to alleviate small sample size problems and support vector machines with maximum margins are used for classifications. The results of experiments carried out show that the proposed method successfully identifies genes informative for cancer classification and provides very reliable cancer classification results. [Copyright &y& Elsevier]