Search

Feature selection is one of the most important tasks in many machine learning and data mining problems. Due to the increasing size of the problems, removing useless, erroneous or noisy features is frequently an initial step that is performed before other data mining algorithms are applied. The aim is to reproduce, or even improve, the performance of the data mining algorithm when all the features are used. Furthermore, the selection of the most relevant features may offer the expert valuable information about the problem to be solved. Over the past few decades, many different feature selection algorithms have been proposed, each with its own strengths and weaknesses. However, as in the case of classification, it is unlikely that a single feature selection algorithm would be able to achieve good results across many different datasets and application fields. Furthermore, when we are dealing with thousands of features, the most powerful feature selection methods are frequently too time consuming to be applied. In classification, one of the most successful ways of consistently improving the performance of a single weak learner is to construct ensembles using boosting methods. In this paper, we propose a general framework for feature selection boosting in the same way boosting is applied to classification. The proposed approach opens a new field of research in which to apply the many techniques developed for boosting classifiers. Using 120 datasets, the experiments reported show a clear improvement in several state-of-the-art feature selection algorithms using the proposed methodology. [ABSTRACT FROM AUTHOR]

In this paper, we propose two cooperative ensemble learning algorithms, i.e., NegBagg and NegBoost, for designing neural network (NN) ensembles. The proposed algorithms incrementally train different individual NNs in an ensemble using the negative correlation learning algorithm. Bagging and boosting algorithms are used in NegBagg and NegBoost, respectively, to create different training sets for different NNs in the ensemble. The idea behind using negative correlation learning in conjunction with the bagging/boosting algorithm is to facilitate interaction and cooperation among NNs during their training. Both NegBagg and NegBoost use a constructive approach to automatically determine the number of hidden neurons for NNs. NegBoost also uses the constructive approach to automatically determine the number of NNs for the ensemble. The two algorithms have been tested on a number of benchmark problems in machine learning and NNs, including Australian credit card assessment, breast cancer, diabetes, glass, heart disease, letter recognition, satellite, soybean, and waveform problems. The experimental results show that NegBagg and NegBoost require a small number of training epochs to produce compact NN ensembles with good generalization. [ABSTRACT FROM AUTHOR]