Search

Dimension reduction is a preprocessing step in machine learning for eliminating undesirable features and increasing learning accuracy. In order to reduce the redundant features, there are data representation methods, each of which has its own advantages. On the other hand, big data with imbalanced classes is one of the most important issues in pattern recognition and machine learning. In this paper, a method is proposed in the form of a cost-sensitive optimization problem which implements the process of selecting and extracting the features simultaneously. The feature extraction phase is based on reducing error and maintaining geometric relationships between data by solving a manifold learning optimization problem. In the feature selection phase, the cost-sensitive optimization problem is adopted based on minimizing the upper limit of the generalization error. Finally, the optimization problem which is constituted from the above two problems is solved by adding a cost-sensitive term to create a balance between classes without manipulating the data. To evaluate the results of the feature reduction, the multi-class linear SVM classifier is used on the reduced data. The proposed method is compared with some other approaches on 21 datasets from the UCI learning repository, microarrays and high-dimensional datasets, as well as imbalanced datasets from the KEEL repository. The results indicate the significant efficiency of the proposed method compared to some similar approaches. [ABSTRACT FROM AUTHOR]

Feature selection is one of the main steps in preprocessing data in machine learning, and its goal is to reduce features by removing additional and noisy features. Feature selection methods and feature reduction in a dataset must consider the accuracy of the classifying algorithms. Meta-heuristic algorithms serve as the most successful and promising methods to solve this problem. Symbiotic Organisms Search (SOS) is one of the most successful meta-heuristic algorithms inspired by organisms' interaction in nature called mutualism, commensalism, and parasitism. In this paper, three SOS-based binary approaches are offered to solve the feature selection problem. In the first and second approaches, several S-shaped transfer functions and several Chaotic Tent Function-based V-shaped transfer functions called BSOSST and BSOSVT are used to make the binary SOS (BSOS). In the third approach, an advanced BSOS based on changing SOS and the chaotic Tent function operators called EBCSOS is provided. The EBCSOS algorithm uses the chaotic Tent function and the Gaussian mutation to increase usefulness and exploration. Moreover, two new operators, i.e., BMPT and BCPT, are suggested to make the commensalism and mutualism stage binary based on a chaotic function to solve the feature selection problem. Finally, the proposed BSOSST and BSOSVT methods and the advanced version of EBCSOS were implemented on 25 datasets than the basic algorithm's binary meta-heuristic algorithms. Various experiments demonstrated that the proposed EBCSOS algorithm outperformed other methods in terms of several features and accuracy. To further confirm the proposed EBCSOS algorithm, the problem of detecting spam E-mails was applied, with the results of this experiment indicating that the proposed EBCSOS algorithm significantly improved the accuracy and speed of all categories in detecting spam E-mails. [ABSTRACT FROM AUTHOR]

It is an enormous challenge for imbalanced data learning in the field of machine learning. To construct balanced datasets, oversampling techniques have been studied extensively. However, many oversampling methods suffer from introducing noisy samples and blurring classification boundaries, leading to overfitting. To solve this problem, this paper proposes a new oversampling method, namely CS-SMOTE, for synthesizing minority class samples by three-point interpolation. CS-SMOTE is mainly based on the center offset factor and a synthesis strategy. First, the CS-SMOTE method removes noise samples, calculates the center offset factor, and selects sparsely distributed minority class samples by using the K-distance graph technique. Next, new samples are generated based on sparse minority samples, random minority samples, and sub-cluster centers located in the same sub-cluster samples. Finally, multiple comparative experiments on 18 well-known datasets demonstrate the effectiveness and general applicability of the proposed CS-SMOTE method for the imbalanced data classification. The experiments show that CS-SMOTE outperforms other competitors in terms of classification accuracy, while avoiding the issue of overfitting. [ABSTRACT FROM AUTHOR]

Machine learning algorithms perform differently in settings with varying levels of training set mislabeling noise. Therefore, the choice of the right algorithm for a particular learning problem is crucial. The contribution of this paper is towards two, dual problems: first, comparing algorithm behavior; and second, choosing learning algorithms for noisy settings. We present the 'sigmoid rule' framework, which can be used to choose the most appropriate learning algorithm depending on the properties of noise in a classification problem. The framework uses an existing model of the expected performance of learning algorithms as a sigmoid function of the signal-to-noise ratio in the training instances. We study the characteristics of the sigmoid function using five representative non-sequential classifiers, namely, Naïve Bayes, kNN, SVM, a decision tree classifier, and a rule-based classifier, and three widely used sequential classifiers based on hidden Markov models, conditional random fields and recursive neural networks. Based on the sigmoid parameters we define a set of intuitive criteria that are useful for comparing the behavior of learning algorithms in the presence of noise. Furthermore, we show that there is a connection between these parameters and the characteristics of the underlying dataset, showing that we can estimate an expected performance over a dataset regardless of the underlying algorithm. The framework is applicable to concept drift scenarios, including modeling user behavior over time, and mining of noisy time series of evolving nature. [ABSTRACT FROM AUTHOR]