Search

Feature selection and construction are important preprocessing techniques in data mining. They allow not only dimensionality reduction but also classification accuracy and efficiency improvement. While feature selection consists in selecting a subset of relevant feature from the original feature set, feature construction corresponds to the generation of new high-level features, called constructed features, where each one of them is a combination of a subset of original features. Based on these definitions, feature construction could be seen as a bi-level optimization problem where the feature subset should be defined first and then the corresponding (near) optimal combination of the selected features should be found. Motivated by this observation, we propose, in this paper, a bi-level evolutionary approach for feature construction. The basic idea of our algorithm, named bi-level feature construction genetic algorithm (BFC-GA), is to evolve an upper-level population for the task of feature selection, while optimizing the feature combinations at the lower level by evolving a follower population. It is worth noting that for each upper-level individual (feature subset), a whole lower-level population is optimized to find the corresponding (near) optimal feature combination (constructed feature). In this way, BFC-GA would be able to output a set of optimized constructed features that could be very informative to the considered classifier. A detailed experimental study has been conducted on a set of commonly used datasets with varying dimensions. The statistical analysis of the obtained results shows the competitiveness and the outperformance of our bi-level feature construction approach with respect to many state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

Feature selection is one of the hottest machine learning topics in recent years. The main purposes of it are to simplify the original model, improve the readability of the model, and prevent over-fitting by searching for a suitable subset of features. There are many methods for this problem, including evolutionary algorithms and particle swarm optimization. Among them, the competitive swarm optimizer is a new optimization algorithm proposed in recent years, which is based on particle swarm optimization algorithm, and has achieved good results in high-dimensional feature selection problems, but it also has the problems of high computation time cost and easily being premature. Aiming at these problems, this paper proposes to add the crossover operator and mutation operator in the genetic algorithm to the competitive swarm optimization, so as to improve the generation speed of new individuals in the algorithm and prevent premature population. After testing on UC Irvine Machine Learning Repository, the new algorithm not only improves the computational efficiency, but also avoids the problem that the competitive swarm optimization algorithm is easy to fall into the local optimum, which greatly improves the calculation effect. [ABSTRACT FROM AUTHOR]