Search

Multi-label classification is a machine learning task to construct a model for assigning an entity in the dataset to two or more class labels. In order to improve the performance of multi-label classification, a multi-objective feature selection algorithm has been proposed in this paper. Feature selection as a preprocessing task for Multi-label classification problems aims to choose a subset of relevant features. Selecting a small number of high-quality features decreases the computational cost and at the same time maximizes the classification performance. However extreme decreasing the number of features causes the failure of classification. As a result, feature selection has two conflicting objectives, namely, minimizing the classification error and minimizing the number of selected features. This paper proposes a multi-objective optimization algorithm to tackle the multi-label feature selection. The task is to find a set of solutions (a subset of features) in a sophisticated large-scale search space using a reference-based multi-objective optimization method. The proposed algorithm utilizes an opposition-based binary operator to generate more diverse solutions. Injection of extreme point of the Pareto-front is another component of the algorithm which aims to find feature subsets with less classification error. The proposed method is compared with two other existing methods on eight multi-label benchmark datasets. The experimental results show that the proposed method outperforms existing algorithms in terms of various multi-objective evaluation measures, such as Hyper-volume indicator, Pure diversity, Two-set coverage, and Pareto-front proportional contribution. The proposed method leads to get a set of well-distributed trade-off solutions which reach less classification error in comparing with competitors, even with the fewer number of features. [ABSTRACT FROM AUTHOR]

Considering that most popular models solving aspect-level sentiment classification problems mainly focus on designing complicated neural networks to scale the importance of each word in the sentence, this paper addresses this problem from the view of semantic space. Motivated by the fact that the senses of a word can be sophisticatedly embedded into the semantic space using a distributed representation, this paper hypothesizes that each sense of a word can be represented by one or more specific dimensions, and thus the target of aspect-level sentiment classification can be simplified into searching the related dimensions for the aspects and sentiments concerned. Particularly, an Attention Vector (ATV) based on attention mechanism is designed for each aspect in terms of a specific task, which involves two sub-vectors, i.e., a Dimension Attention Vector (DATV) and a Sentiment Attention Vector (SATV). The DATV determines the significances of different dimensions based on their correlations with an aspect ; and the SATV allocates weights for the attributes of words, which are decided by sentiment polarities and part-of-speech (PoS) tagging. Given a sub-dataset related to an aspect , the ATV will be optimized by an Artificial Bee Colony (ABC) algorithm with a Support Vector Machine (SVM) classifier, the objective of which is to maximize classification accuracy. Intrinsically, the DATV can reduce the ambiguity existing in polysemy, meanwhile, the SATV is an auxiliary means for the optimization of the DATV , which can help eliminate the misunderstandings caused by antonyms. Then, the optimized DATV will be applied on a Convolutional Neural Network (CNN) model via simply scaling the pretrained word embeddings as inputs (named as ATV-CNN model). Experimental results show that the ATV-CNN model can have substantial advantages when compared with state-of-the-art models. [ABSTRACT FROM AUTHOR]

More training instances could lead to better classification accuracy. However, accuracy could also degrade if more training instances mean further noises and outliers. Additional training instances arguably need additional computational resources in future data mining operations. Instance selection algorithms identify subsets of training instances that could desirably increase accuracy or at least do not decrease accuracy significantly. There exist many instance selection algorithms, but no single algorithm, in general, dominates the others. Moreover, existing instance selection algorithms do not allow direct controlling of the instance selection rate. In this paper, we present a simple and generic cluster-oriented instance selection algorithm for classification problems. Our proposed algorithm runs an unsupervised K Means Clustering algorithm on the training instances and with a given selection rate, selects instances from the centers and the borders of the clusters. On 24 benchmark classification problems, when very similar percentages of instances are selected by various instance selection algorithms, K Nearest Neighbours classifiers achieve more than 2%–3% better accuracy when using instances selected by our proposed method than when using those selected by other state-of-the-art generic instance selection algorithms. [ABSTRACT FROM AUTHOR]

Algorithm selection technology, aimed at choosing the most suitable algorithm for a given machine learning task, has achieved significant success in the domain of 2D vision. However, few studies have explored its application to the 3D point cloud domain. The rapid proliferation and development of 3D point cloud classification algorithms underscore the urgent need for exploration in selecting these algorithms. In this paper, we propose a novel meta-learning-based 3D classification approach, termed 3D meta-classification , to address this gap. The approach operates at both the base-level and meta-level phases. At the base level, candidate 3D classification algorithms constantly classify various 3D datasets, recording their classification performance as empirical knowledge. As for the meta-level phase, it specifically tailors the 3D meta-knowledge generator, 3D meta-feature extractor, and 3D meta-database constructor to establish the 3D meta-database, capturing the relationship between the 3D meta-features and empirical knowledge. Leveraging the 3D meta-database, the 3D classification meta-learner trains the meta-model and predicts suitable algorithms for new incoming 3D datasets. Extensive experiments are conducted to select the best-performing classification algorithm for specific 3D datasets from ModelNet40. The results demonstrate the effectiveness of the proposed 3D meta-classification model; the accuracies of one-from-two and one-from-three algorithm selection tasks reach over 98% and 80%, respectively. [ABSTRACT FROM AUTHOR]

In this study, we focus on semi-supervised data stream classification tasks. With the advent of applications that generate vast streams of data, data stream mining algorithms are required to handle the intrinsic issues of learning patterns from this type of non-stationary data. The main challenges are fast arriving, infinite length, limited labeled data, concept drift, and concept evolution. Although many algorithms have been used to overcome these issues, there are still many challenges that need to be addressed and discussed in practical domains. This paper mainly focuses on the limited labeled data and concept drift detection issues. Therefore, a semi-supervised framework called CPSSDS is proposed, which uses an incremental classifier as a base learner and a self-training framework to handle the scarcity of labeled samples. The proposed approach utilizes a form of Conformal Predictors to discover a set of informative unlabeled data instances to add to the original training set in each training procedure which is the main challenge in the standard self-training approach. In terms of this framework, the Kolmogorov-Smirnov test is adopted to detect concept drift by comparing the outputs of conformal prediction for two sequences of data chunks. Finally, experimental results on several synthetic and real datasets demonstrate that the proposed learning framework is comparable to the state-of-the-art semi-supervised algorithms for data streams. The implementation source code of the CPSSDS is available at https://github.com/neginsmd/CPSSDS.git. [ABSTRACT FROM AUTHOR]

• We consider an incremental learning mechanism based on progressive fuzzy three-way concept. • We design a novel classification mechanism and develop its corresponding algorithm for individual recognition. • For the changing individual, a dynamic update mechanism is studied for progressive concepts. • Effectiveness of the proposed mechanism is verified by comparing with classification mechanism. Fuzzy concept has been an important methodology for data analysis, especially in the classification research. Particularly, fuzzy concept could directly process the continuous data through contrasting the numerical data into the membership degree of object to attribute. However, the classical fuzzy concept only focuses on the positive information, that is, the information about membership degree, while ignoring non-membership degree. Meanwhile, since the limitations of individual cognition and cognitive environment, the concept learning is progressive. Inspired by these thoughts, we design an incremental learning mechanism based on progressive fuzzy three-way concept for object classification in dynamic environment. In this paper, the object and attribute learning operators are first defined to obtain fuzzy three-way concept. Then, a progressive fuzzy three-way concept and its corresponding concept space are learned considering the progressive process of concept learning. Moreover, the object classify mechanism and dynamic update mechanism based on the progressive concept space are proposed, and their effectiveness is verified by numerical experiments. Finally, an incremental learning mechanism is further designed for dynamic increased data and compared with other fuzzy classify methods. All the experimental results carried on ten datasets from UCI and KEEL illustrate the proposed learning mechanism is an excellent object classify algorithm. [ABSTRACT FROM AUTHOR]

Classification is one of the fundamental supervised learning tasks which learns classifiers from the given training data and related labels. The quality of labels is important in classification tasks. However, in many real-world scenarios, data annotation is often corrupted, especially when the annotation process is done by humans. Vaguely labeled data is one of the common problems caused by limited domain knowledge or partial data observation. In this paper, a novel method is proposed to classify vaguely labeled data based on evidential fusion. Vaguely labeled data are divided into several small data groups by the proposed valid label-set cover assignment algorithm. Evidence theory is applied to vaguely labeled data classification by regarding each base classifier on a small data group as one piece of evidence. This gives the chance of classifying unseen precise labeled data from related vague labels. Note that our approach is not restricted to any specific classifiers. It can be generalized to any off-the-shelf classification methods with probabilistic outputs. Finally, experiments are conducted on both synthetic data and real-world data with different base classifiers. Experimental results show that the proposed method achieves superior performance against compared methods. [ABSTRACT FROM AUTHOR]

• A supervised multi-view model is proposed to realize a block-based feature selection. • The proposed model is composed of all sharing sub-models in each class. • The sparse regularizer can enhance the sparsity of blocks from features and views. • The proposed algorithm can realize the block separation and independent solution. • Numerical experiments show the effectiveness of our method on large-scale datasets. With the increasing scale of obtained multi-view data, how to deal with large-scale multi-view data quickly and efficiently is a significant problem. In this paper, a novel supervised multi-view feature selection method based on locally sparse regularization and block computing is proposed to solve the problem. Specifically, the multi-view dataset is firstly divided into sub-blocks according to classes and views. Then with the aid of the Alternating Direction Method of Multipliers (ADMM), a sharing sub-model is proposed to perform feature selection on each class by integrating each view's locally sparse regularizers and shared loss that makes all views share a common penalty and regresses samples to their labels. Finally, all the sharing sub-models are fused to form the final general additive feature selection model, in which each sub-block adjusts its corresponding variables to perform block-based feature selection. In the optimization process, the proposed model can be decomposed into multiple separate subproblems, and an efficient optimization algorithm is proposed to solve them quickly. The comparison experiments with several state-of-the-art feature selection methods show that the proposed method is superior in classification accuracy and training speed. [ABSTRACT FROM AUTHOR]

Attribute reduction (AR) plays an important role in reducing irrelevant and redundant domain attributes, while maintaining the underlying semantics of retained ones. Based on Earth Mover's Distance (EMD), this paper presents a robust AR algorithm from the perspective of minimising the inconsistency between the discernibility of the reduct and the entire original attribute set. Due to the susceptibility of the inconsistency gauger to noisy information, a strategy for instance denoising is also proposed by detecting abnormal local class distributions with regard to the global class distribution. With such a pretreatment process for AR, the robustness of the reduct found is significantly improved, as testified by systematic experimental investigations. The experimental results demonstrate that the reduct gained by the proposed approach generally outperforms those attained by the application of popular, state-of-the-art AR techniques, in terms of both the size of attribute reduction and the classification results using the reduced attributes. [ABSTRACT FROM AUTHOR]

Imbalanced classification problems are attracting the attention of the research community because they are prevalent in real-world problems and they impose extra difficulties for learning methods. Fuzzy rule-based classification systems have been applied to cope with these problems, mostly together with sampling techniques. In this paper, we define a new fuzzy association rule-based classifier, named FARCI, to tackle directly imbalanced classification problems. Our new proposal belongs to the algorithm modification category, since it is constructed on the basis of the state-of-the-art fuzzy classifier FARC–HD. Specifically, we modify its three learning stages, aiming at boosting the number of fuzzy rules of the minority class as well as simplifying them and, for the sake of handling unequal fuzzy rule lengths, we also change the matching degree computation, which is a key step of the inference process and it is also involved in the learning process. In the experimental study, we analyze the effectiveness of each one of the new components in terms of performance, F - score , and rule base size. Moreover, we also show the superiority of the new method when compared versus FARC–HD alongside sampling techniques, another algorithm modification approach, two cost-sensitive methods and an ensemble. [ABSTRACT FROM AUTHOR]

In this paper, an adaptive semi-supervised rotation forest (SSRoF) algorithm is proposed for the classification of hyperspectral images with limited training data. Our proposition is based on Rotation Forest (RoF), a classifying technique that has proved to be remarkably accurate in the context of high-dimensional data. It is adapted to the semi-supervised context, by increasing the number of training instances in the learning stage, with high-quality unlabeled samples mined using ensemble margin. SMOTE is adopted to overcome the class imbalance problem. Out-Of-Bag (OOB) instances are used in a second phase to figure out the optimal number of samples to be added to the training set. Five ensemble methods and five semi-supervised methods are employed as comparisons. The results on three real hyperspectral remote sensing datasets demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Semi-supervised learning is a new challenge that exploits the information of unlabeled instances. In this paper, we propose a novel Laplacian pair-weight vector projection (LapPVP) algorithm for semi-supervised classification. LapPVP consists of two stages: projection and classification. In the projection stage, LapPVP integrates a within-class scatter, a between-class scatter and a Laplacian regularization together and formulates a pair of optimization problems. The goal of LapPVP is to maximize the between-class scatter and minimize the within-class scatter on the basis of labeled data, and simultaneously maintain the geometric structure of labeled and unlabeled data in the projected subspace. The optimization problems of LapPVP are identical to generalized eigenvalue ones. Thus, it is easy to obtain the optimal solutions to LapPVP. In the classification stage, LapPVP adopts a minimal distance classifier to implement tasks in the projected subspace. The proposed LapPVP can find a better separability using the geometric information embedded in labeled and unlabeled instances and the discriminative information embedded in labeled instance. Experiments on artificial datasets and UCI datasets validate the feasibility and effectiveness of the proposed algorithm. [ABSTRACT FROM AUTHOR]

The foremost requirement for a decision forest to achieve better ensemble accuracy is building a set of accurate and diverse individual decision trees as base classifiers. Existing decision forest algorithms mainly differ from each other on how they induce diversity among the decision trees. At the same time, most of the drawbacks of existing algorithms originate from their induction processes of diversity. In this paper, we propose a new decision forest algorithm that is more balanced through effective synchronization between different sources of diversity. The proposed algorithm is balanced theoretically and empirically. We carried out experiments on 25 well-known data sets that are publicly available from the UCI Machine Learning Repository, to perform an extensive empirical evaluation. The experimental results indicate that the proposed algorithm has the best average ensemble accuracy rank of 1.8 compared to its closest competitor at 3.5. Using the Friedman and Bonferroni-Dunn tests, we also show that such an improvement is indeed statistically significant. In addition, the proposed algorithm is found to be competitive in terms of complexity and other relevant parameters. [ABSTRACT FROM AUTHOR]

• Two new mutation strategies to enhance the DE performance are proposed. • A new adaptive approach to key control parameters are defined. • The proposed approach has superior classiffication accuracy. • The proposed algorithm's performance is acceptable in feature subset selection. In classification problems, normally there exists a large number of features, but not all of them contributing to the improvement of classification performance. These redundant features make the classification problem time consuming and often result in poor performance. Feature selection methods have been proposed to reduce the number of features, minimize computational cost, and maximize classification accuracy. As a wrapper-based approach, evolutionary algorithms have been widely applied in feature subset selection tasks. However, some of them trap into local minima, especially when the number of features increases, while others are not efficient in computational time. This paper proposes a Modified Differential Evolution (DE) approach to Feature Selection (MDEFS) by utilizing two new mutation strategies to create a feasible balance between exploration and exploitation and maintain the classification performance in an acceptable range concerning both the number of features and accuracy. Some modifications are made to the standard DE crossover and its key control parameters to enhance the proposed algorithm's capabilities further. The proposed method has been compared to several state-of-the-art methods utilizing standard datasets from the UCI repository and results of the experiments demonstrate the superiority of the proposed approach. [ABSTRACT FROM AUTHOR]

• Two improved AdaBoost algorithms are proposed for the imbalanced data classification problem. • Adjust the weighted parameters of the weak classifiers and consider the imbalanced index to improve the algorithm. • Theoretic proofs of the powers of the proposed algorithms are presented. • Simulated and real datasets are applied to show the recognition of our proposed algorithms. Class imbalance is one of the most popular and important issues in the domain of classification. The AdaBoost algorithm is an effective solution for classification, but it still needs improvement in the imbalanced data problem. This paper proposes a method to improve the AdaBoost algorithm using the new weighted vote parameters for the weak classifiers. Our proposed weighted vote parameters are determined not only by the global error rate but also by the classification accuracy rate of the positive class, which is our primary interest. The imbalanced index of the data is also a factor in constructing our algorithms. Our proposed algorithms outperform the traditional ones, especially regarding the evaluation criterion of F - 1 Measure. Theoretic proofs of the advantages of our proposed algorithms are presented. Two kinds of simulated datasets and four real datasets are applied in the experiment as the specific support to our proposed algorithms. [ABSTRACT FROM AUTHOR]

This paper presents an interpretable neural system—termed Evolving Long-term Cognitive Network—for pattern classification. The proposed model was inspired by Fuzzy Cognitive Maps, which are interpretable recurrent neural networks for modeling and simulation. The network architecture is comprised of two neural blocks: a recurrent input layer and an output layer. The input layer is a Long-term Cognitive Network that gets unfolded in the same way as other recurrent neural networks, thus producing a sort of abstract hidden layers. In our model, we can attach meaningful linguistic labels to each neuron since the input neurons correspond to features in a given classification problem and the output neurons correspond to class labels. Moreover, we propose a variant of the backpropagation learning algorithm to compute the required parameters. This algorithm includes two new regularization components that are aimed at obtaining more interpretable knowledge representations. The numerical simulations using 58 datasets show that our model achieves higher prediction rates when compared with traditional white boxes while remaining competitive with the black boxes. Finally, we elaborate on the interpretability of our neural system using a proof of concept. [ABSTRACT FROM AUTHOR]

• Supervised feature selection of high-dimensional data is formulated as an MOP. • We developed a problem-specific non-dominated sorting genetic algorithm to solve the MOP. • We made a systematical comparison between our method and some state-of-the-art FS approaches. Feature selection (FS), which plays an important role in classification tasks, has been recently studied as a multi-objective optimization problem (MOP). In this paper, we consider minimizing three objectives of FS and propose a problem-specific non-dominated sorting genetic algorithm (PS-NSGA). In PS-NSGA, an accuracy-preferred domination operator is applied, which makes the individual with higher classification accuracy in the population more likely to survive. And a quick bit mutation is used, which breaks through the limitation of traditional bit string mutation and increases the efficiency. In addition, a mutation-retry operator and a combination operator are designed to make our algorithm converge faster and better. At last, a solution selection strategy is developed to determine the most proper feature subset from the obtained Pareto solutions. Experimental results on 15 real-world high-dimensional datasets demonstrate that our proposed algorithm can achieve competitive classification accuracy while obtaining a smaller size of feature subset compared with some state-of-the-art evolutionary and traditional FS algorithms. [ABSTRACT FROM AUTHOR]

Sparse representation has been in a widespread use in hyperspectral image (HSI) classification task. The samples to be classified can be linearly represented with a few samples from the same class. However, when samples from different classes are highly correlated with each other, it makes the classification task challenging. To solve this problem, we take the Euclidean distance information between the training samples and testing samples into consideration to construct a new dictionary for sparse representation. That is, we propose a locality-constrained sparse representation classifier (LSRC) in this paper. First, the K-nearest neighbour (KNN) algorithm is applied to the training data set to form a locality-constrained dictionary by excluding the samples separated from testing samples in the Euclidean space. Then, the sparse coding is applied to the testing sample with the formed dictionary via class dependent orthogonal matching pursuit (OMP) algorithm which utilizes the class label information. Finally, by using the minimal residual rule within all catergories, we can obtain class label of the testing sample. Experiments based on the chosen three hyperspectral datasets prove that our proposed LSRC outperforms other popular classifiers. [ABSTRACT FROM AUTHOR]

Nonparallel support vector machine is a binary classifier which has better sparsity and classification performance than other support vector machine models. However, it works slowly. Although sample selection is an effective acceleration approach, it will disturb the accuracy. Fortunately, another recently emerging screening method could speedup the solving process and keeps the accuracy unchanged. Motivated by the above research, a novel two-stage hybrid screening rule based on variational inequality and duality gap is proposed in this paper. It can reduce the scale of dual problem by deleting more redundant samples. Then, a better acceleration effect will be obtained. Meanwhile, it is still a safe method which means that the optimal solution remains unchanged. Moreover, Shrinking technique is embedded into the fast iterative algorithm DCDM to get further speedup. Thus the hybrid method can reduce the size of the problem, and accelerate the solving process greatly. Numerical experiments on ten benchmark data sets and a real Chinese red wine data set are conducted to verify the effectiveness of our hybrid acceleration strategy. [ABSTRACT FROM AUTHOR]

This paper proposes a new algorithm for an automatic feature selection procedure in High Dimensional Graphical Models. The algorithm, called Best-Path Algorithm (BPA), rests on a filter method and performs feature selection based on mutual information. Over the last years, filter methods have been successfully employed to reduce the size of the input dataset and retain, at the same time, the relevant feature information for modeling and classification problems. However, the extant filter algorithms are mostly heuristic or require high computational effort. The BPA overcomes these drawbacks by taking advantage of the links between variables brought to the fore by the Edwards's algorithm. Once the High Dimensional Graphical Model, depicting the probabilistic structure of the variables, is determined, the BPA selects the best subset of features by analyzing its path-steps. The path-step that includes the variables with the most predictive power for the target one is then determined via the computation of the entropy correlation coefficient. This index, being based on the notion of (symmetric) Kullback-Leibler divergence, is closely connected to the mutual information that the path-step variables share with that of interest. The BPA application to simulated and real-word benchmark datasets highlights its potential and greater effectiveness compared to alternative extant methods. [ABSTRACT FROM AUTHOR]

Data stream mining has gained increasing attention in recent years due to its wide range of applications. In this paper, we propose a new selective prototype-based learning (SPL) method on evolving data streams, which dynamically maintains representative instances to capture the time-changing concepts, and make predictions in a local fashion. As an instance-based learning model, SPL only maintains some important prototypes (i.e., ISet) via error-driven representativeness learning. The fast condensed nearest neighbor (FCNN) rule, is further introduced to compress these prototypes, making the algorithm also applicable under memory constraints. To better distinguish noises from the instances associated with the new emerging concept, a potential concept instance set (i.e., PSet) is used to store all misclassified instances. Relying on the potential concept instance set, a local-aware distribution-based concept drift detection approach is proposed. SPL has several attractive benefits: (a) it can fit the evolving data streams very well by maintaining a small size of instance set; (b) it is capable of capturing both gradual and sudden concept drifts effectively; (c) it has great capabilities to distinguish noise/outliers from drifting instances. Experimental results show that the SPL has better classification performance than many other state-of-the-art algorithms. [ABSTRACT FROM AUTHOR]

Recently, the research on semi-supervised support vector machine (S 3 VM) has received much attention, and many S 3 VM algorithms have been proposed. Existing studies have shown that S 3 VM is effective especially in the situations where labeled data is scarce. Nevertheless, most of existing S 3 VM algorithms belong to centralized learning, that is, all the data is stored and processed at a fusion center. In many real-world applications, data may be horizontally or vertically distributed over multiple nodes (parties). Besides, from the concerns of privacy and security, each node would not like to share its original data with the others. On the other hand, considering that the data is usually sequentially generated, online processing is preferred. In this paper, we propose two online distributed S 3 VM (dS 3 VM) algorithms, which are respectively used for horizontally and vertically partitioned data classification. In these two algorithms, to get a fully decentralized implementation, we propose a new form of manifold regularization defined on some anchor points that are adaptively selected by an online strategy. Besides, we use the sparse random feature map to approximate the kernel feature map. In this manner, the model parameters can be collaboratively estimated without transmitting the original data between neighbors. The convergence performances of the proposed algorithms are analyzed. Simulations on several data sets are performed. Results show that the proposed dS 3 VM algorithms achieve good classification performance even when there is only a small portion of labeled data. [ABSTRACT FROM AUTHOR]

The detection of gravitational waves with ground-based laser-interferometric detectors requires sensitivity to changes in distance much smaller than the diameter of atomic nuclei. Though sophisticated machinery and techniques have been developed over the past few decades to isolate such instruments from non-astrophysical noise, the detectors are still susceptible to instrumental and environmental noise transients known as “glitches,” which hinder searches for transient gravitational waves. The Gravity Spy project is an effort to comprehensively classify the glitches that afflict gravitational wave detectors into morphological families by combining the strengths of machine learning algorithms and citizen scientists. This paper presents the initial Gravity Spy dataset used for citizen scientist and machine learning classification – a static, accessible, documented dataset for testing machine learning supervised classification. Previous versions of this dataset used in [8, 53] did not include all current classes and also for some of the classes, some samples were pruned and added. This set consists of time–frequency images of LIGO glitches and their associated metadata. These glitches are organized by time–frequency morphology into 22 classes for which descriptions and representative images are presented. Results from the application of state-of-the-art supervised classification methods to this dataset are presented in order to provide baselines for future glitch classification work. Standard splitting for training, validation, and testing sets are also presented to facilitate the comparison between different machine learning methods. The baseline methods are selected from both traditional and more recent deep learning approaches. An ensemble framework is developed that demonstrates that combining various classifiers can yield a more accurate model for classification. The ensemble classifier, trained with the standard training set, achieves 98.21% accuracy on the standard test set. [ABSTRACT FROM AUTHOR]

In real world data set the underlying data distribution may be highly skewed. Building accurate classifiers for predicting group membership is made difficult because the classifier has a tendency to be biased towards the over represented or majority group as a result. This problem is referred to as a class imbalance problem. Re-sampling techniques that produce new samples by means of over-sampling aim to combat class imbalance by increasing the number of members that belong to the minority group. This paper introduces a new over-sampling technique that focuses on noise reduction and selective sampling of the minority group which results in improvement for prediction of minority group membership. Experiments are conducted across a wide range of data sets, learners and over sampling methods. The results for this new method show improvement for Sensitivity and Gmean measures over the compared approaches. [ABSTRACT FROM AUTHOR]

Abstract: This paper presents an approach to aircraft target recognition using Inverse Synthetic Aperture Radar (ISAR) images. The goal of this work is to develop a robust algorithm to add Automated Target Recognition (ATR) capabilities to extract efficient feature vectors and ensure a comprehensive recognition process. In the first part on this paper, we used a database of ISAR images reconstructed from anechoic chamber simulations in order to extract efficient feature vectors. In the second part of this work, we proposed a recognition architecture to perform recognition tasks and provide a human operator with useful information for target recognition tasks. Several kinds of descriptor can be used to acquire information about target characteristics from radar signals and images. Indeed, a number of methods are currently used in automatic target recognition; notably based on ISAR imaging. However, target characteristic extraction from radar echoes remains a difficult task, and the methods are generally specific to either aircraft or ship recognition. In this work, we describe our approach to designing faster, more effective retrieval systems and a comprehensive architecture. Our approach uses global feature vectors for both ship and aircraft recognition. Firstly, the global feature vectors are described by two types of descriptor which have shown to be efficient in radar target recognition. The first type defines target shape obtained by watershed transformation, facilitating interpretation for the human operator. The second type of vector descriptor is based on so-called polar signatures, obtained using the polar mapping procedure. The latter are highly discriminative and thus significantly improve and facilitate target recognition, leading to greater precision and accuracy. Secondly, we describe the retrieval architecture suitable for the second type of vector descriptor, which checks invariance in relation to target rotation and scale. It is, in itself, more efficient and processing time at the retrieval step is reduced and controlled by the human operator. Finally, in order to validate our proposed feature vectors and architecture, the Support Vector Machine (SVM) classifier will be implemented; the results of which will be tested and presented in the last section of this paper. [Copyright &y& Elsevier]

Abstract: This paper proposes a classification method that is based on easily interpretable fuzzy rules and fully capitalizes on the two key technologies, namely pruning the outliers in the training data by SVMs (support vector machines), i.e., eliminating the influence of outliers on the learning process; finding a fuzzy set with sound linguistic interpretation to describe each class based on AFS (axiomatic fuzzy set) theory. Compared with other fuzzy rule-based methods, the proposed models are usually more compact and easily understandable for the users since each class is described by much fewer rules. The proposed method also comes with two other advantages, namely, each rule obtained from the proposed algorithm is simply a conjunction of some linguistic terms, there are no parameters that are required to be tuned. The proposed classification method is compared with the previously published fuzzy rule-based classifiers by testing them on 16 UCI data sets. The results show that the fuzzy rule-based classifier presented in this paper, offers a compact, understandable and accurate classification scheme. A balance is achieved between the interpretability and the accuracy. [Copyright &y& Elsevier]

This paper presents a new approach to solving the optimization task that arises when L2-SVM in its primal form is considered. In particular, we propose the application of a Barzilai–Borwein (BB) update step in five variants for the classic Stochastic Gradient Descent (SGD) algorithm. The evaluation is designed to check the effectiveness of the proposed methods in large scale scenarios in terms of execution time, convergence and sensitivity to the choice of initial parameters. The obtained results are compared with those obtained for well-known linear SVM algorithms and they indicate that the level of convergence of the proposed methods is very similar to that found in the other studies. Moreover, our approach shows much lower sensitivity to the choice of initial parameters, which allows for a substantial reduction of pre-processing. [ABSTRACT FROM AUTHOR]

Abstract: This paper formulates a twin-hypersphere support vector machine (THSVM) classifier for binary recognition. Similar to the twin support vector machine (TWSVM) classifier, this THSVM determines two hyperspheres by solving two related support vector machine (SVM)-type problems, each one is smaller than the classical SVM, which makes the THSVM be more efficient than the classical SVM. In addition, the THSVM avoids the matrix inversions in its two dual quadratic programming problems (QPPs) compared with the TWSVM. By considering the characteristics of the dual QPPs of THSVM, an efficient Gilbert’s algorithm for the THSVM based on the reduced convex hull (RCH) instead of directly optimizing its pair of QPPs is further presented. Computational results on several synthetic as well as benchmark datasets indicate the significant advantages of the THSVM classifier in the computational time and test accuracy. [Copyright &y& Elsevier]

Abstract: Classification is one of the key issues in the fields of decision sciences and knowledge discovery. In this paper, we present a new classification method based on gravitational potential energy between particles. The basic principle of gravitation based classification (GBC) algorithm is to find the equilibrium condition of the classifier, which is modeled as a classifier line between two groups of fixed particles. In the proposed approach, the input data is a collection of masses, which interact with each other based on Newton’s universal law of gravitation and the laws of motion. We present a convex formulation for this problem that always converges to a global optimum solution. The proposed method has been compared with some well-known classification approaches, and the results confirm the high performance of the proposed method. [Copyright &y& Elsevier]

Abstract: Fuzzy weighting, which is designed to reduce the effects of outliers for batch classification problems, might generate unreasonable membership grades especially for the samples following an input outlier, when incorporated into online classification algorithms directly. In this paper, a generalized framework for online fuzzy weighting is presented, which incrementally calculates the membership of each incoming sample by taking into account the membership grades of previous samples in a pairwise manner. The advocated pairwise-distance based scheme can not only identify possible outliers, but also show good adaptation to the sequentially received samples in the online setting. We apply it to online Passive–Aggressive (PA) algorithm in a direct way. The resulting Fuzzy Passive–Aggressive (FPA) algorithm achieves comparable classification accuracy with benchmark incremental SVM, while still enjoying the time efficiency of simple PA, which is a Perceptron-like algorithm. Besides, FPA exhibits the best performance among PA family, which makes it a robust and efficient alternative to PA, in order to deal with unavoidable outliers in large-scale or high-dimensional real datasets. The study is supported by a series of experiments with IDA benchmark repository, as well as two real-world problems namely place recognition and radar emitter recognition. [Copyright &y& Elsevier]

Abstract: The machine learning community has traditionally used correct classification rates or accuracy (C) values to measure classifier performance and has generally avoided presenting classification levels of each class in the results, especially for problems with more than two classes. C values alone are insufficient because they cannot capture the myriad of contributing factors that differentiate the performance of two different classifiers. Receiver Operating Characteristic (ROC) analysis is an alternative to solve these difficulties, but it can only be used for two-class problems. For this reason, this paper proposes a new approach for analysing classifiers based on two measures: C and sensitivity (S) (i.e., the minimum of accuracies obtained for each class). These measures are optimised through a two-stage evolutionary process. It was conducted by applying two sequential fitness functions in the evolutionary process, including entropy (E) for the first stage and a new fitness function, area (A), for the second stage. By using these fitness functions, the C level was optimised in the first stage, and the S value of the classifier was generally improved without significantly reducing C in the second stage. This two-stage approach improved S values in the generalisation set (whereas an evolutionary algorithm (EA) based only on the S measure obtains worse S levels) and obtained both high C values and good classification levels for each class. The methodology was applied to solve 16 benchmark classification problems and two complex real-world problems in analytical chemistry and predictive microbiology. It obtained promising results when compared to other competitive multi-class classification algorithms and a multi-objective alternative based on E and S. [Copyright &y& Elsevier]

Abstract: This paper presents a new feature discovery approach called FEADIS that strengthens learning algorithms with discovered features. The discovered features are formed by various mathematical functions including ceil, mod, sin, and similar. These features are constructed in an iterative manner to improve gradually its learning performance. We demonstrate FEADIS capabilities by testing different types of datasets including periodical datasets. From the results, we conclude that FEADIS increases the performance of learning algorithms in a wide range of datasets for nominal or numeric target feature. Furthermore, most of the well known classifiers without FEADIS strengthening have severe difficulty in handling datasets that have periodical functional relations between input features and target feature – a difficulty circumvented by their potential use of FEADIS. [Copyright &y& Elsevier]

Abstract: In this paper, the problems of fuzzy binary relations on fuzzy n-cell number space and their applications are investigated. Firstly, we have defined some fuzzy approximation relations on fuzzy n-cell number space, and studied their properties. Secondly, as application, we have developed an algorithmic version of classification in an imprecise or uncertain environment by using the fuzzy approximation relations. Practical examples are provided to show the application and rationality of the proposed techniques. [Copyright &y& Elsevier]

Abstract: This paper proposes a filter-based algorithm for feature selection. The filter is based on the partitioning of the set of features into clusters. The number of clusters, and consequently the cardinality of the subset of selected features, is automatically estimated from data. The computational complexity of the proposed algorithm is also investigated. A variant of this filter that considers feature-class correlations is also proposed for classification problems. Empirical results involving ten datasets illustrate the performance of the developed algorithm, which in general has obtained competitive results in terms of classification accuracy when compared to state of the art algorithms that find clusters of features. We show that, if computational efficiency is an important issue, then the proposed filter may be preferred over their counterparts, thus becoming eligible to join a pool of feature selection algorithms to be used in practice. As an additional contribution of this work, a theoretical framework is used to formally analyze some properties of feature selection methods that rely on finding clusters of features. [Copyright &y& Elsevier]

Abstract: In this paper, we make a comparative study of the effectiveness of ensemble technique for sentiment classification. The ensemble framework is applied to sentiment classification tasks, with the aim of efficiently integrating different feature sets and classification algorithms to synthesize a more accurate classification procedure. First, two types of feature sets are designed for sentiment classification, namely the part-of-speech based feature sets and the word-relation based feature sets. Second, three well-known text classification algorithms, namely naı¨ve Bayes, maximum entropy and support vector machines, are employed as base-classifiers for each of the feature sets. Third, three types of ensemble methods, namely the fixed combination, weighted combination and meta-classifier combination, are evaluated for three ensemble strategies. A wide range of comparative experiments are conducted on five widely-used datasets in sentiment classification. Finally, some in-depth discussion is presented and conclusions are drawn about the effectiveness of ensemble technique for sentiment classification. [ABSTRACT FROM AUTHOR]

Abstract: In this paper a novel computation method is proposed to perform the common vector approach (CVA) faster than its conventional implementation in pattern recognition. While conventional CVA calculations perform the classification with respect to the distance between vectors, the new method performs the classification using scalars. A theoretical proof of the equivalence of the proposed method is provided. Next, in order to verify the numerical equivalence of the proposed computation method to the conventional (vector-based) method, numerical experiments are conducted over three different face databases, namely the AR Database, extended Yale Face Database B, and FERET Database. Since the computational gain may depend on (i) the dimension of the feature vectors, (ii) the number of feature vectors used in training, and (iii) the number of classes, the effects of these items are clearly verified via these databases. Our theoretically equivalent (but faster) method provided no difference in the classification rates despite its improved classification speed as compared to the classical implementation of CVA. The new method is found to be about 2.1–3.0 times faster than the conventional CVA implementation for the AR face database, 1.9–3.3 times faster for the extended Yale Face Database B, and 1.9–3.1 times faster for the FERET Database. [ABSTRACT FROM AUTHOR]

Abstract: Rule learning is one of the most common tasks in knowledge discovery. In this paper, we investigate the induction of fuzzy classification rules for data mining purposes, and propose a hybrid genetic algorithm for learning approximate fuzzy rules. A novel niching method is employed to promote coevolution within the population, which enables the algorithm to discover multiple rules by means of a coevolutionary scheme in a single run. In order to improve the quality of the learned rules, a local search method was devised to perform fine-tuning on the offspring generated by genetic operators in each generation. After the GA terminates, a fuzzy classifier is built by extracting a rule set from the final population. The proposed algorithm was tested on datasets from the UCI repository, and the experimental results verify its validity in learning rule sets and comparative advantage over conventional methods. [Copyright &y& Elsevier]

Abstract: Discretization algorithms have played an important role in data mining and knowledge discovery. They not only produce a concise summarization of continuous attributes to help the experts understand the data more easily, but also make learning more accurate and faster. In this paper, we propose a static, global, incremental, supervised and top-down discretization algorithm based on Class-Attribute Contingency Coefficient. Empirical evaluation of seven discretization algorithms on 13 real datasets and four artificial datasets showed that the proposed algorithm could generate a better discretization scheme that improved the accuracy of classification. As to the execution time of discretization, the number of generated rules, and the training time of C5.0, our approach also achieved promising results. [Copyright &y& Elsevier]

In this paper a modified algorithm is suggested for developing polynomial neural network (PN N) models. Optimal partial description (PD) modeling is introduced at each layer of the PNN expansion, a task accomplished using the orthogonal least squares (OLS) method. Based on the initial PD models determined by the polynomial order and the number of PD inputs, OLS selects the most significant regressor terms reducing the output error variance. The method produces PN N models exhibiting a high level of accuracy and superior generalization capabilities. Additionally, parsimonious models are obtained comprising a considerably smaller number of parameters compared to the ones generated by means of the conventional PNN algorithm. Three benchmark examples are elaborated, including modeling of the gas furnace process as well as the iris and wine classification problems. Extensive simulation results and comparison with other methods in the literature, demonstrate the effectiveness of the suggested modeling approach. [ABSTRACT FROM AUTHOR]

This paper addresses the well-known classification task of data mining, where the objective is to predict the class which an example belongs to. Discovered knowledge is expressed in the form of high-level, easy-to-interpret classification rules. In order to discover classification rules, we propose a hybrid decision tree/genetic algorithm method. The central idea of this hybrid method involves the concept of small disjuncts in data mining, as follows. In essence, a set of classification rules can be regarded as a logical disjunction of rules, so that each rule can be regarded as a disjunct. A small disjunct is a rule covering a small number of examples. Due to their nature, small disjuncts are error prone. However, although each small disjunct covers just a few examples, the set of all small disjuncts can cover a large number of examples, so that it is important to develop new approaches to cope with the problem of small disjuncts. In our hybrid approach, we have developed two genetic algorithms (GA) specifically designed for discovering rules covering examples belonging to small disjuncts, whereas a conventional decision tree algorithm is used to produce rules covering examples belonging to large disjuncts. We present results evaluating the performance of the hybrid method in 22 real-world data sets. [ABSTRACT FROM AUTHOR]