Search

Fuzzy rough set is a theoretical framework of fuzzy uncertainty management, and discernibility matrix offers a mathematical foundation for algorithm construction of feature learning. The approaches of fuzzy rough set and discernibility matrix have been successfully applied in single-label learning. However, few works have been done on investigating the foundation of fuzzy rough discernibility matrix on multi-label data. There will be two pivotal problems to be addressed when using fuzzy rough discernibility matrix for multi-label data analysis. One is how to extract sample-level and label-level correlations; and the other is how to utilize the discernibility matrix for algorithm construction. For this reason, in this paper the fuzzy rough discrimination matrix is introduced to deal with the problem of multi-label feature selection. First, the significance of labels in the label space is captured based on the label correlation. Labels with different significances contribute to different weights for measuring the similarity between samples. Hence, a sample similarity matrix in the label space is computed based on the label weighting strategy. Then, a framework of a fuzzy decision system is formalized, in which the discernibility matrix of fuzzy rough sets is introduced as a foundation to evaluate the sample discrimination ability of features. Under the discernibility matrix criterion, a multi-label learning algorithm is developed to select discriminative features from multi-label data. A series of experimental analysis verifies the effectiveness and efficiency of the proposed method. [ABSTRACT FROM AUTHOR]

Sparse Bayesian learning (SBL) provides state-of-the-art performance in accuracy, sparsity and probabilistic prediction for classification. In SBL, the regularization priors are automatically determined that avoids an exhaustive hyperparameter selection by cross-validation. However, scalability to large problems is a drawback of SBL because of the inversion of a potentially enormous covariance matrix for updating the regularization priors in every iteration. This paper develops an approximate SBL algorithm called ARP-SBL, where the regularization priors are approximated without inverting the covariance matrix. Therefore, our approach can easily scale up to problems with large data size or feature dimension. It alleviates the long training time and high memory complexity in SBL. Based on ARP-SBL, two scalable nonlinear SBL models: scalable relevance vector machine (ARP-RVM) and scalable sparse Bayesian extreme learning machine (ARP-SBELM) are developed for problems of large data size and large feature size respectively. Experiments on a variety of benchmarks have shown that the proposed models are with competitive accuracy compared to existing methods while i) converging faster; ii) requiring thousands of times less memory; iii) without exhaustive regularized hyperparameter selection; and iv) easily scaling up to large data size and high dimensional features. [ABSTRACT FROM AUTHOR]

This paper investigates a new bacterial colony-based feature selection algorithm to improve the classification accuracy of customers for personalized products recommendation. An attribute learning strategy is developed in this study to update the feature related population. Specifically, the features can be weighted according to their historic contributions to both the individual- and group-based subsets. Additionally, the frequency of appearance is also recorded for the feature candidates to improve the diversity of feature distribution and avoid the over-fitting. Based on the weight-based feature indexes and frequency of appearance records, the performance of feature subsets are enhanced by replacing the features being repeatedly appeared in a same vector. To explore the feasibility of the proposed method for the missing feature problems, the objective of the optimization is to minimize the classification error using the acceptable number of features. K-Nearest Neighbor is employed as the learning technique to cooperate with the proposed feature selection method. The effectiveness of the proposed feature selection method is demonstrated by performing test on the datasets from UCI machine learning repository and real-world data from Amazon customer reviews of products. Compared with other seven feature selections methods, the proposed feature selection algorithm outperforms the other algorithms by achieving higher classification accuracy rate using smaller features. [ABSTRACT FROM AUTHOR]

Abstract Dealing with high-dimensional dataset has always been an important problem and feature selection is one of useful tools. In this paper, we develop a new filter based supervised feature selection method by combining Trace Ratio Criterion of Linear Discriminant Analysis (TRC-LDA) and group sparsity regularization. The filter based supervised feature selection method is a classifier-independent method while the TRC-LDA criterion is a recently developed criterion for dimensionality reduction that can well preserve discriminative information of dataset. However, there are seldom methods by utilizing TRC-LDA criterion for feature selection. On the other hand, imposing the l2,0-norm to the projection matrix of TRC-LDA will force some rows in it to be zero while keep other rows nonzero making the index of nonzero rows to be the selected features, however, l 2,0 -nom minimizing problem is NP-hard and intractable. To solve the above problem, in this paper, we develop a new method, namely, Trace Ratio Criterion Discriminative Feature Selection (TRC-DFS), for feature selection. The proposed TRC-DFS has imposed l 2,1 -norm, i.e. an approximation of l 2,0 -norm, to the projection matrix W of TRC-LDA to achieve feature selection. As a result, the proposed TRC-DFS can both achieve feature selection as well as capture the discriminative structure of data. We also extend the proposed method with l 2, p -norm (0 < p < 1) regularization to grasp more sparse property and develop an iterative approach to calculate the optimal solution, which is rigorously proved to be converged. Extensive simulations based on several real-world datasets verify the effectiveness of the proposed methods. [ABSTRACT FROM AUTHOR]

Hyperparameter optimization is a challenging process that has the potential to improve machine learning algorithms. Since it creates a remarkable computational burden for machine learning tasks, there have been few works coping with tuning strategies of a specific algorithm. In this paper, an improved Stochastic Gradient Descent (SGD) based on Fisher Maximization is developed for tuning hyperparameters of an Echo State Network (ESN) which has a wide range of applications. The results of the method are then compared with those of traditional Gradient Descent and Grid Search. According to the obtained results; 1) The scale of the data sets greatly affects the reliability of hyperparameter optimization results; 2) Feature selection is critical in terms of mean error of training when hyperparameter optimization is applied on some methods such as ESN; 3) SGD falls in a good local minima if Fisher Maximization is performed to find a good starting point. [ABSTRACT FROM AUTHOR]

Abstract Feature selection is used in many applications in machine learning and bioinformatics. As a popular approach, feature selection can be implemented in the filter-manner based on the sparse solution of the l 1 regularization. Most study of the l 1 regularization concentrates on investigating the iteration solution of the problem or focuses on adapting sparsity to different applications. It is necessary to explore more deeply about how the sparsity learned with the l 1 regularization contributes to feature selection. In this paper, we make an effort to analyze the role of the l 1 regularization in feature selection from the perspective of information theory. We discover that the l 1 regularization contributes to minimizing the redundancy in feature selection. To avoid the complex computation of the l 1 optimization, we propose a novel feature selection algorithm, i.e. the Laplacian regularization based on mutual information (LRMI), which realizes the minimization of the redundancy in a new way, and incorporates the l 2 norm to achieve automatic grouping. Extensive experimental results demonstrate the superiority of LRMI over several traditional l 1 regularization based feature selection algorithms with less time consumption. [ABSTRACT FROM AUTHOR]

Feature selection has playing an important role in many pattern recognition and machine learning applications, where meaningful features are desired to be extracted from high dimensional raw data and noisy ones are expected to be eliminated. l 2,1 -norm regularization based Robust Feature Selection (RFS) has extracted a lot of attention due to its efficiency and high performance of joint sparsity. In this paper, we propose a more general framework for robust and discriminative multi-class feature selection. Four types of weighting, which are based on correlation information between features and labels, are adopted to strengthen the discriminative performance of l 2,1 -norm joint sparsity. F-norm regularization, which is extended from multi-class Elastic Net, is added to improve the stability of the method. An efficient algorithm and its corresponding convergence proof are provided. Experimental results on several two-class and multi-class datasets are performed to verify the effectiveness of the proposed feature selection method. [ABSTRACT FROM AUTHOR]

Naïve Bayes, k-nearest neighbors, Adaboost, support vector machines and neural networks are five among others commonly used text classifiers. Evaluation of these classifiers involves a variety of factors to be considered including benchmark used, feature selections, parameter settings of algorithms, and the measurement criteria employed. Researchers have demonstrated that some algorithms outperform others on some corpus, however, inconsistency of human labeling and high dimensionality of feature spaces are two issues to be addressed in text categorization. This paper focuses on evaluating the five commonly used text classifiers by using an automatically generated text document collection which is labeled by a group of experts to alleviate subjectivity of human category assignments, and at the same time to examine the influence of the number of features on the performance of the algorithms. [ABSTRACT FROM AUTHOR]

In recent years, there has been a great interest in prostate segmentation, which is an important and challenging task for CT image guided radiotherapy. In this paper, a learning-based segmentation method via joint transductive feature selection and transductive regression is presented, which incorporates the physician׳s simple manual specification (only taking a few seconds), to aid accurate segmentation, especially for the case with large irregular prostate motion. More specifically, for the current treatment image, experienced physician is first allowed to manually assign the labels for a small subset of prostate and non-prostate voxels, especially in the first and last slices of the prostate regions. Then, the proposed method follows the two step: in prostate-likelihood estimation step, two novel algorithms, tLasso and wLapRLS, will be sequentially employed for transductive feature selection and transductive regression, respectively, aiming to generate the prostate-likelihood map. In multi-atlases based label fusion step, the final segmentation result will be obtained according to the corresponding prostate-likelihood map and the previous images of the same patient. The proposed method has been substantially evaluated on a real prostate CT dataset including 24 patients with 330 CT images, and compared with several state-of-the-art methods. Experimental results show that the proposed method outperforms the state-of-the-arts in terms of higher Dice ratio, higher true positive fraction, and lower centroid distances. Also, the results demonstrate that simple manual specification can help improve the segmentation performance, which is clinically feasible in real practice. [ABSTRACT FROM AUTHOR]

Cancer is curable if it can be detected early. One way to detect cancer is by analyzing the change in expression of genes in the suspected tissue. Serial analysis of gene expression (SAGE) is a sequencing technique used for measuring the expression levels of genes. Cancer detection problem can be posed as binary classification problem like whether a tissue is cancerous or normal. SAGE libraries contain expression levels of thousands of genes which are the features. It is impossible to consider all these features for classification and also the general feature selection algorithms are not efficient with this data. In this paper, closed frequent itemset mining is proposed as a feature selection technique for identifying a small set of features which can distinguish the two classes efficiently. The performance of the proposed technique is evaluated on SAGE data related to breast tissue and a group of 26 genes are selected as best features. Two well known classifiers, extreme learning machine (ELM) and support vector machine (SVM), are used to evaluate the effectiveness of the selected features in classification and found that the proposed method works well with these classifiers. [ABSTRACT FROM AUTHOR]

Bayesian Ying–Yang (BYY) harmony learning system is a powerful tool for statistical learning. Via the BYY harmony leaning of finite mixtures, model selection, i.e., the selection of an appropriate number of components for the mixture, can be made automatically during parameter learning on a given dataset. In this paper, an adaptive gradient BYY harmony learning algorithm is proposed for log-normal mixtures to implement parameter learning with automated model selection. It is demonstrated by the experiments on both synthetic and real-world datasets that the proposed BYY harmony learning algorithm not only has the ability of automated model selection, but also leads to a rather good estimation of the parameters in the original log-normal mixture. [ABSTRACT FROM AUTHOR]

Abstract: In machine learning the so-called curse of dimensionality, pertinent to many classification algorithms, denotes the drastic increase in computational complexity and classification error with data having a great number of dimensions. In this context, feature selection techniques try to reduce dimensionality finding a new more compact representation of instances selecting the most informative features and removing redundant, irrelevant, and/or noisy features. In this paper, we propose a filter-based feature selection method for working in the multiple-instance learning scenario called ReliefF-MI; it is based on the principles of the well-known ReliefF algorithm. Different extensions are designed and implemented and their performance checked in multiple instance learning. ReliefF-MI is applied as a pre-processing step that is completely independent from the multi-instance classifier learning process and therefore is more efficient and generic than wrapper approaches proposed in this area. Experimental results on five benchmark real-world data sets and 17 classification algorithms confirm the utility and efficiency of this method, both statistically and from the point of view of execution time. [Copyright &y& Elsevier]

Abstract: Feature selection plays an important role in pattern classification. Its purpose is to remove redundant features from data set as many as possible. The presence of useless features may not only deteriorate the performance of learning algorithms, but also obscure important information (e.g., intrinsic structure) behind data. Along with new and emerging techniques, data sets in many domains are becoming larger and larger and many irrelevant features are often prevailing in these data sets. This, however, poses great challenges to traditional learning algorithms, such as low efficiency and over-fitting. Thus, it becomes apparent that an efficient technique is needed to eliminate redundant or irrelevant features from the data sets. Currently, many endeavors to cope with this problem have been attempted and various outstanding feature selection methods have been proposed. Unlike other selection methods, in this paper we propose a general scheme of boosting feature selection method using information metric. The primary characteristic of our method is that it exploits weight of data to select salient features. Furthermore, the weight of data will be dynamically changed after each candidate feature has been selected. Thus, the information criteria used in feature selector can exactly represent the relevant degree between features and the class labels. As a result, the selected feature subset has maximal relevance to the class labels. Simulation studies carried out on UCI data sets show that the classification performance achieved by our proposed method is better than those of other selection methods in most cases. [Copyright &y& Elsevier]