Search

In this paper we propose three novel feature ranking methods for supervised feature selection in the context of classification which are based on possibility theory. All three methods – nonspecificity, strife and total uncertainty – are tested on eight artificial data sets and ten medical real-world data sets and benchmarked against ReliefF, the Fisher score, the fuzzy entropy and similarity (FES), the Fuzzy similarity and entropy (FSAE) filter, symmetrical uncertainty as well as using no feature selection. The feature ranking methods were applied following two approaches: (1) using a fixed threshold for the number of highest-ranking features selected and (2) using a hybrid feature selection approach with a classifier (k-nearest neighbor classifier, decision tree, similarity classifier, SVM) to select the optimal number of features to select. The results indicate that strife and the Fisher score are the two feature ranking methods that for both approaches are on average ranked the highest in terms of the test set accuracy on the real-world data sets. Besides that, for the hybrid approach, strife uses most of the time a considerably smaller number of features than nonspecificity and total uncertainty. In terms of stability, which was measured with the adjusted stability measure (ASM), the Fisher score and strife were among the most stable feature ranking methods in this study. Additionally, strife's feature subsets were diverse compared to those of the remaining feature selection methods, making it a good candidate to be included in a feature selection ensemble. [ABSTRACT FROM AUTHOR]

• Feature selection improves performance of machine learning algorithms. • Feature selection with more n-tier techniques is simpler and more stable. • A feature selection model that is not specific to any data set is widely applied. Feature Selection and classification have previously been widely applied in various areas like business, medical and media fields. High dimensionality in datasets is one of the main challenges that has been experienced in classifying data, data mining and sentiment analysis. Irrelevant and redundant attributes have also had a negative impact on the complexity and operation of algorithms for classifying data. Consequently, the algorithms record poor results or performance. Some existing work use all attributes for classification, some of which are insignificant for the task, thereby leading to poor performance. This paper therefore develops a hybrid filter model for feature selection based on principal component analysis and information gain. The hybrid model is then applied to support classification using machine learning techniques e.g. the Naïve Bayes technique. Experimental results demonstrate that the hybrid filter model reduces data dimensions, selects appropriate feature sets, and reduces training time, hence providing better classification performance as measured by accuracy, precision and recall.. [ABSTRACT FROM AUTHOR]