Search

Feature selection aims to select crucial features to improve classification accuracy in machine learning and data mining. Existing methods concentrate on the classification information from candidate features while seldom considering the changing information supported by selected features. In this paper, we construct a dynamic support ratio (DSR), which employs the new information of selected features to support classification. DSR explicitly describes the dynamic interactions between selected features and candidate features. Simultaneously, the feature relevance and feature redundancy are treated adaptively. Thus, distinctive features can be noticed sensitively. Afterward, a novel feature selection method based on a dynamic support ratio (DSRFS) is proposed. The proposed method is established on 18 benchmark data sets with four different classifiers. Classification accuracy, standard deviation, recall and statistical validations are employed to measure the classification performance. Extensive experiments demonstrate that DSRFS not only reduces the dimension of the feature space effectively, but also obtains the best average classification accuracy. [ABSTRACT FROM AUTHOR]

Feature selection, which aims to screen out redundant and irrelevant features from datasets, is integral to machine learning and data mining. Grey Wolf Optimization (GWO) is a recent meta-heuristic algorithm based on swarm intelligence and has wide applicability to various optimization problems due to its fast convergence and few parameters. However, since the wolf pack is always dominated by the three leading wolves (i.e. , α , β and δ), the GWO algorithm suffers from weak exploration throughout the whole optimization process and easily stagnates into local optima. In this paper, an Adaptively Balanced Grey Wolf Optimization (ABGWO) algorithm is proposed to seek out the optimal feature subset for high-dimensional classification. Specifically, to improve the exploration ability of GWO, a random wolf is introduced to cooperate with α , β and δ. A novel level-based strategy is further adopted to select the random wolf. Besides, to dynamically modulate the exploration and exploitation ability in different optimization stages, an adaptive coefficient is introduced to regulate the leadership of α , β , δ and the randomly-selected wolf. Finally, the improvement of exploration and exploitation is validated on 12 high-dimensional datasets provided by Arizona State University and University of California Irvine, and the superiority of ABGWO is further verified by comparing it with seven state-of-the-art feature selection approaches on the aspect of classification accuracy, size of feature subset and computational time. • Introducing a random individual to prevent sharp decline in population diversity. • Semi-random selection prevents the rapid decrease of the convergence speed. • A coefficient factor linearly increases to adaptively adjust the exploration and exploitation. • The improved GWO algorithm for high-dimensional feature selection significantly boosts the classification performance. [ABSTRACT FROM AUTHOR]

There is a growing interest in developing feature subset selection schemes for high-dimensional datasets by filter, wrapper, embedded, and hybrid manners. In this paper, we propose a new hybrid (filter-wrapper) feature selection approach. At first, in the filter step, we rank input features according to their relevance with the class label. Afterwards, we apply different clustering methods for the classification of the selected features. We perform an inner and outer cluster ranking based on the primary feature ranking in the next step. Then, different search strategies are performed on the best cluster of features in the wrapper phase. Moreover, we add some of them to the feature set based on the classifiers (nearest neighbor, decision tree, support vector machine, and random forests) feedback. Then, the algorithm goes to the next cluster, and this process is continued till all clusters are met. Finally, we compare the results of the proposed method to the state-of-the-art schemes. Comparison results imply the superiority of the proposed method to the counterparts on eight high-dimensional datasets in terms of accuracy and computational complexity. • A new hybrid feature selection framework named the HyCluster is proposed. • Our algorithm significantly improves the time complexity using feature clustering. • It selects relevant and irredundant features and significantly improves accuracy rates. • Comparison results imply the superiority of the proposed method to the counterparts. [ABSTRACT FROM AUTHOR]

In recent years, brain storm optimization (BSO) algorithm has received much attention in solving classical optimization problems and is used to implement evolutionary classification models. However, in practical applications, large-scale datasets complicate the structure of the classification model, which can have a great impact on the classification performance. In the optimization process, the traditional single-strategy BSO cannot preserve the information of dominant solution well, and its generation strategy is inefficient in solving various complex practical problems. To solve this problem, we introduce feature selection to improve the optimization model structure. Meanwhile, in order to enhance the search capability of BSO, three new generation strategy are embedded in the BSO algorithm in this paper. With the three generation methods of global optimal, local optimal and nearest neighbor, the information of the dominant solution can be better preserved and the search efficiency can be improved. The performance of the proposed generation strategy in solving classification problems is demonstrated on ten datasets with different sizes and dimensions. The experimental results reveal that the new generation strategy can enhance the performance of BSO algorithm for solving classification problems. [ABSTRACT FROM AUTHOR]

In this paper we propose three novel feature ranking methods for supervised feature selection in the context of classification which are based on possibility theory. All three methods – nonspecificity, strife and total uncertainty – are tested on eight artificial data sets and ten medical real-world data sets and benchmarked against ReliefF, the Fisher score, the fuzzy entropy and similarity (FES), the Fuzzy similarity and entropy (FSAE) filter, symmetrical uncertainty as well as using no feature selection. The feature ranking methods were applied following two approaches: (1) using a fixed threshold for the number of highest-ranking features selected and (2) using a hybrid feature selection approach with a classifier (k-nearest neighbor classifier, decision tree, similarity classifier, SVM) to select the optimal number of features to select. The results indicate that strife and the Fisher score are the two feature ranking methods that for both approaches are on average ranked the highest in terms of the test set accuracy on the real-world data sets. Besides that, for the hybrid approach, strife uses most of the time a considerably smaller number of features than nonspecificity and total uncertainty. In terms of stability, which was measured with the adjusted stability measure (ASM), the Fisher score and strife were among the most stable feature ranking methods in this study. Additionally, strife's feature subsets were diverse compared to those of the remaining feature selection methods, making it a good candidate to be included in a feature selection ensemble. [ABSTRACT FROM AUTHOR]

Sine Cosine Algorithm (SCA) that is one of the population-based metaheuristic optimization algorithms basically consists of the updating mechanism based on sine and cosine functions. In this algorithm, a few random and adaptive variables are also utilized for more effective motions of the candidate solutions. SCA has some drawbacks like other some metaheuristic algorithms. SCA tends to be stuck into the local regions in the search space and this affects negatively on the computational effort required to find the best solution point in the search space. This paper presents four different improved versions of SCA. The proposed improvements on original SCA are the innovations on the updating mechanism of SCA. To evaluate the performances of Improved Sine Cosine Algorithms (ImpSCAs), well-known numerical optimization problems including CEC 2014 test suite are used. Firstly, different analyses of the proposed ImpSCAs are dealt with such as the convergence analysis, search history analysis, trajectory analysis, average distance analysis, and computational complexity analysis. Secondly, the proposed four versions of ImpSCAs are compared with the original SCA for CEC 2014 benchmark problems with dimension sizes of 10D, 30D and 50D. Finally, original SCA and ImpSCAs are adapted to select optimal feature combination and they are tested for 10 feature selection datasets taken from the UCI machine learning repository. The benchmark results show that the performances of the ImpSCA 1 , ImpSCA 2 , and ImpSCA 4 are better than that of the original SCA. From the feature selection results, it is observed that three versions of ImpSCAs (except ImpSCA 3) outperform the original SCA in 80% of the datasets. Source codes of ImpSCAs are publicly available at https://github.com/uguryuzgec/ImpSCAs. • We focused to improve the exploration and exploitation ability of the original SCA. • The basic idea is based on the innovations on the update mechanism. • The ISCAs were evaluated using CEC2014 benchmark test suite. • The proposed ISCAs were tested for 10 popular UCI datasets. • Three versions of ISCAs (except ISCA 3) outperform the SCA in 80% of the datasets. [ABSTRACT FROM AUTHOR]