Search

This paper presents a workbench to get simple neural classification models based on product evolutionary networks via a prior data preparation at attribute level by means of filter-based feature selection. Therefore, the computation to build the classifier is shorter, compared to a full model without data pre-processing, which is of utmost importance since the evolutionary neural models are stochastic and different classifiers with different seeds are required to get reliable results. Feature selection is one of the most common techniques for pre-processing the data within any kind of learning task. Six filters have been tested to assess the proposal. Fourteen (binary and multi-class) difficult classification data sets from the University of California repository at Irvine have been established as the test bed. An empirical study between the evolutionary neural network models obtained with and without feature selection has been included. The results have been contrasted with nonparametric statistical tests and show that the current proposal improves the test accuracy of the previous models significantly. Moreover, the current proposal is much more efficient than the previous methodology; the time reduction percentage is above 40%, on average. Our approach has also been compared with several classifiers both with and without feature selection in order to illustrate the performance of the different filters considered. Lastly, a statistical analysis for each feature selector has been performed providing a pairwise comparison between machine learning algorithms. [ABSTRACT FROM AUTHOR]

• Subset-based feature selection + multi-layer perceptron. • Subset-based feature selection + radial basis function neural networks. • Semi-wrapper feature subset selection approach based on Naïve Bayes. • Extensive experiments on 34 data sets (22 from University of California at Irvine –UCI- and 12 synthetic data sets). Data preparation via feature selection + neural networks. • Feature selection: 7 approaches, which are 5 filters and 2 semi-wrappers (based on Correlation, consistency and Naïve Bayes; other with SOAP or SU), have been tested on difficult classification problems. This paper explores widely the data preparation stage within the process of knowledge discovery and data mining via feature subset selection in the context of two very well-known neural models: radial basis function neural networks and multi-layer perceptron. It is known the best performance of wrapper attribute selection methods based on the evaluation measure provided by a classifier, although the temporal complexity of learning neural networks practically precludes the use of wrapper techniques, especially in complex databases with high dimensionality and a large number of labels. In this paper, we propose the use of the Naïve Bayes classifier as a fitness function within a semi-wrapper feature selection approach. The Naïve Bayes classifier is a good fast approach to a neural network and utilising it as a measure of goodness in a backward search on a ranking provides a specific attribute selection method for neural networks in complex data. The test-bed consists of 34 binary and multi-class classification problems and 7 feature selectors. Of these, there are 6 data sets with upwards of 5 classes. According to the reported accuracy results that have been supported by non-parametric statistical tests in different scenarios, our method has been shown to be very suitable for both kinds of neural networks. Moreover, the reduced feature-space is around 20% of the full attribute space. The speedup with the aforementioned semi-wrapper is very outstanding and its value fluctuates, on average, from about 1.5 with radial basis function neural networks to around 30 with multi-layer perceptron. [ABSTRACT FROM AUTHOR]

Data splitting is an important step in artificial neural network (ANN) models, which is found in the form of training and testing subsets. In general, a random data splitting method is favored to divide a pool of samples into subsets, without considering the quality of data for the training step of a neural network. The drawback of poor data splitting methods is that they poses ill effects to the performance of the neural network when the data involves complex matrices or multivariate modeling. In order to overcome this drawback, the current paper presents our proposed M-SPXY method. It is based on a modified version of Sample Set Partitioning, which relies on a joint X-y distances (SPXY) method. The proposed method has resulted in better performance, compared to the modified Kennard-Stone (KS) method, using Mahalanobis distances (MDKS). In our experiments, the proposed approach was employed to compare various data splitting methods using data sets from the repository of the University of California in Irvine (UCI), processed through an Extreme Learning Machine (ELM) neural network. Performance was measured in terms of classification accuracy. The results indicate that the classification accuracy of the proposed M-SPXY process is superior to that of the MDKS data splitting method. [ABSTRACT FROM AUTHOR]

Trained radial basis function networks are well-suited for use in extracting rules and explanations because they contain a set of locally tuned units. However, for rule extraction to be useful, these networks must first be pruned to eliminate unnecessary weights. The pruning algorithm cannot search the network exhaustively because of the computational effort involved. It is shown that using multiple pruning methods with smart ordering of the pruning candidates, the number of weights in a radial basis function network can be reduced to a small fraction of the original number. The complexity of the pruning algorithm is quadratic (instead of exponential) in the number of network weights. Pruning performance is shown using a variety of benchmark problems from the University of California, Irvine machine learning database. [ABSTRACT FROM AUTHOR]