Search

• We proposed a new ensemble learning framework in the setting of granular computing. • We conducted data transformation by information granulation for in-depth training. • We developed multi-level fusion strategies for in-depth classifiers fusion. • The proposed framework outperforms popular ensemble learning methods. • The proposed framework outperforms popular standard learning methods. Classification is a special type of machine learning tasks, which is essentially achieved by training a classifier that can be used to classify new instances. In order to train a high performance classifier, it is crucial to extract representative features from raw data, such as text and images. In reality, instances could be highly diverse even if they belong to the same class, which indicates different instances of the same class could represent very different characteristics. For example, in a facial expression recognition task, some instances may be better described by Histogram of Oriented Gradients features, while others may be better presented by Local Binary Patterns features. From this point of view, it is necessary to adopt ensemble learning to train different classifiers on different feature sets and to fuse these classifiers towards more accurate classification of each instance. On the other hand, different algorithms are likely to show different suitability for training classifiers on different feature sets. It shows again the necessity to adopt ensemble learning towards advances in the classification performance. Furthermore, a multi-class classification task would become increasingly more complex when the number of classes is increased, i.e. it would lead to the increased difficulty in terms of discriminating different classes. In this paper, we propose an ensemble learning framework that involves transforming a multi-class classification task into a number of binary classification tasks and fusion of classifiers trained on different feature sets by using different learning algorithms. We report experimental studies on a UCI data set on Sonar and the CK+ data set on facial expression recognition. The results show that our proposed ensemble learning approach leads to considerable advances in classification performance, in comparison with popular learning approaches including decision tree ensembles and deep neural networks. In practice, the proposed approach can be used effectively to build an ensemble of ensembles acting as a group of expert systems, which show the capability to achieve more stable performance of pattern recognition, in comparison with building a single classifier that acts as a single expert system. [ABSTRACT FROM AUTHOR]

Abstract: Neuro-fuzzy classifiers are characterized by incorporation of the expert knowledge into their construction. The most popular neuro-fuzzy systems are Mamdani-type systems. The main groups of neuro-fuzzy systems are also Takagi–Sugeno and logical-type systems. The latter were very rarely studied in the literature, however it was shown that logical-type reasoning transpired to be better suited for classification tasks whereas Mamdani-type reasoning for approximation problems (). It is well known that an ensemble of several classifiers improves classification accuracy. Many ensembling methods are meta-learning techniques, thus they can be used to design ensembles of various member classifiers. In the paper an ensemble of logical-type neuro-fuzzy systems, based on S-implications, is proposed. The design of such an ensemble, and fuzzy ensemble in general, is a challenging task and encounters technical difficulties. The major problem is that such an ensemble contains separate rule bases which cannot be directly merged. As systems are separate, we cannot treat fuzzy rules coming from different systems as rules from the same (single) system. In the paper, the problem is addressed by a novel design of fuzzy systems constituting the ensemble, resulting in normalization of individual rule bases during learning. Several experiments illustrate the idea presented in the paper. [Copyright &y& Elsevier]

Abstract: This paper deals with the strategies for feature selection and multi-class classification in the emotion detection problem. The aim is two-fold: to increase the effectiveness of four feature selection algorithms and to improve accuracy of multi-class classifiers for emotion detection problem under different frameworks and strategies. Although, a large amount of research has been conducted to determine the most informative features in emotion detection, it is still an open problem to identify reliably discriminating features. As it is believed that highly informative features are more critical factor than classifier itself, recent studies have been focused on identifying the features that contribute more to the classification problem. In this paper, in order to improve the performance of multi-class SVMs in emotion detection, 58 features extracted from recorded speech samples are processed in two new frameworks to boost the feature selection algorithms. Evaluation of the final feature sets validates that the frameworks are able to select more informative subset of the features in terms of class-separability. Also it is found that among four feature selection algorithms, a recently proposed one, LSBOUND, significantly outperforms the others. The accuracy rate obtained in the proposed framework is the highest achievement reported so far in the literature for the same dataset. [Copyright &y& Elsevier]

Abstract: This paper proposed a new improved method for back propagation neural network, and used an efficient method to reduce the dimension and improve the performance. The traditional back propagation neural network (BPNN) has the drawbacks of slow learning and is easy to trap into a local minimum, and it will lead to a poor performance and efficiency. In this paper, we propose the learning phase evaluation back propagation neural network (LPEBP) to improve the traditional BPNN. We adopt a singular value decomposition (SVD) technique to reduce the dimension and construct the latent semantics between terms. Experimental results show that the LPEBP is much faster than the traditional BPNN. It also enhances the performance of the traditional BPNN. The SVD technique cannot only greatly reduce the high dimensionality but also enhance the performance. So SVD is to further improve the document classification systems precisely and efficiently. [Copyright &y& Elsevier]

Abstract: Determining the importance of different management areas in a company provides guidance about the needs of increasing the analysis and actions focuses in particular topic. To do it, it is necessary to decompose the management in a coherent set of specific management areas and provide a way that allows the company to determine the importance of these areas for them. This paper presents a novel system that guides companies to obtain a classification of important management areas for them. It is focused on the use of a case based reasoning system because the variability and the evolution of companies as time passes requires using techniques with learning capabilities. The proposed system provides an automatic self-assessment system that provides companies an ordered list of their most important management areas. This system was implemented a year ago for the evaluation of Spanish companies. Currently, it is in production providing relevant information about the management areas of these companies. [Copyright &y& Elsevier]

Abstract: Datasets with missing values are frequent in real-world classification problems. It seems obvious that imputation of missing values can be considered as a series of secondary tasks, while classification is the main purpose of any machine dealing with these datasets. Consequently, Multi-Task Learning (MTL) schemes offer an interesting alternative approach to solve missing data problems. In this paper, we propose an MTL-based method for training and operating a modified Multi-Layer Perceptron (MLP) architecture to work in incomplete data contexts. The proposed approach achieves a balance between both classification and imputation by exploiting the advantages of MTL. Extensive experimental comparisons with well-known imputation algorithms show that this approach provides excellent results. The method is never worse than the traditional algorithms – an important robustness property – and, also, it clearly outperforms them in several problems. [Copyright &y& Elsevier]

Abstract: In this paper, we consider the problem of generating concise but accurate naive Bayes classifiers using taxonomy of propositionalized attributes. For the problem, we introduce propositionalized attribute taxonomy guided naive Bayes Learner (PAT-NBL), a machine learning algorithm that effectively utilizes taxonomy to generate compact classifiers. We extend classical naive Bayes learner to the PAT-NBL algorithm that traverses over a propositionalized taxonomy to search for a locally optimal cut. PAT-NBL uses bottom-up search to find the locally optimal cut on a given taxonomy. For the evaluation of candidate cuts, we apply conditional log-likelihood, conditional minimum description length, and conditional Akaike information criterion. The detected cut enables PAT-NBL to construct an instance space which corresponds to the taxonomy and the data. That is, after PAT-NBL determines a cut according to its information-theoretic criteria, the algorithm generates a concise naive Bayes classifier based on the cut. Our experimental results on UCI Machine Learning benchmark data sets indicate that the proposed algorithm can generate naive Bayes classifiers that are more compact and often comparably accurate to those produced by standard naive Bayes learners. [Copyright &y& Elsevier]

Abstract: In this paper, a shell fitting space (SFS) is presented to map non-linearly separable data to linearly separable ones. A linear or quadratic transformation maps data into a new space for better classification, if the transformation method is properly guessed. This new SFS space can be of high or low dimensionality, and the number of dimensions is generally low and it is equal to the number of classes. The SFS method is based on fitting a hyper-plane or shell to the learning data or enclosing them into a hyper-surface. In the proposed method, the hyper-planes, curves, or cortex become the axis of the new space. In the new space a linear support vector machine (SVM) multi-class classifier is applied to classify the learn data. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, we propose a novel Hybrid Higher Order Neural Classifier (HHONC) which contains different high-order units. In contrast with conventional fully-connected higher order neural networks (HONN), our proposed method uses fewer learning parameters and allocates the best fitted model in dealing with different datasets by modifying the orders of different high-order units and updating the learning parameters. Structure, model selection and updating the learning parameters of HHONC is introduced and is applied in classification of the Iris data set, the breast cancer data set, the Wine recognition data set, the Glass identification data set, the Balance scale data set, and the Pima diabetes data set. Acquired results are compared with the methods presented in . It is observed that the fewer features the dataset contains, the more accurate the HHONC performs, however the accuracy of datasets with more features are acceptable. Experimental results show about 3.5% and 0.6% improvements compared to the best accuracy obtained in previously methods for classifying the Pima diabetes and Iris datasets, respectively. In addition, by using a same method for reducing the feature number, it’s shown the proposed method perform more accurate than methods presented in . In this case, improvements compared to the best acquired accuracy of mentioned methods are about 1.7%, 1.3% and 0.2% in classification of Pima, Iris and Breast cancer datasets, respectively. [Copyright &y& Elsevier]

Abstract: Multi-valued and multi-labeled learning is concerned with samples associated with a set of values both with label and attribute. This paper proposes a new learning framework, which combines multi-valued attribute decomposition with multi-label learning. To deal with multi-valued attribute, we present five methods which differ in strategies with the correlations of multi values. After data transformation, three classic multi-label algorithms are employed for learning. Experimental results demonstrate that most combined methods significantly outperform the existing decision tree based algorithms. Furthermore, exploring the advantages and limitations of each combined method, we find the optimal combination corresponding to different types of datasets. [ABSTRACT FROM AUTHOR]

Machine learning can extract desired knowledge and ease the development bottleneck in building expert systems. Among the proposed approaches, deriving classification rules from training examples is the most common. Given a set of examples, a learning program tries to induce rules that describe each class. The rough-set theory has served as a good mathematical tool for dealing with data classification problems. It adopts the concept of equivalence classes to partition training instances according to some criteria. In the past, we thus proposed a fuzzy-rough approach to produce a set of certain and possible rules from quantitative data. Attributes are, however, usually organized into hierarchy in real applications. This paper thus extends our previous approach to deal with the problem of producing a set of cross-level maximally general fuzzy certain and possible rules from examples with hierarchical and quantitative attributes. The proposed approach combines the rough-set theory and the fuzzy-set theory to learn. It is more complex than learning from single-level values, but may derive more general knowledge from data. Fuzzy boundary approximations, instead of upper approximations, are used to find possible rules, thus reducing some subsumption checking. Some pruning heuristics are adopted in the proposed algorithm to avoid unnecessary search. A simple example is also given to illustrate the proposed approach. [Copyright &y& Elsevier]

Ordinal (i.e., ordered) classifiers are used to make judgments that we make on a regular basis, both at work and at home. Perhaps surprisingly, there have been no comprehensive studies in the scientific literature comparing the various ordinal classifiers. This paper compares the accuracy of five ordinal and three non-ordinal classifiers on a benchmark of fifteen real-world datasets. The results show that the ordinal classifiers that were tested had no meaningful statistical advantage over the corresponding non-ordinal classifiers. Furthermore, the ordinal classifiers that guaranteed monotonic classifications showed no meaningful statistical advantage over a majority-based classifier. We suggest that the tested ordinal classifiers did not properly utilize the order information in the presence of non-monotonic noise. [Copyright &y& Elsevier]

Abstract: Ensemble classification – combining the results of a set of base learners – has received much attention in the machine learning community and has demonstrated promising capabilities in improving classification accuracy. Compared with neural network or decision tree ensembles, there is no comprehensive empirical research in support vector machine (SVM) ensembles. To fill this void, this paper analyses and compares SVM ensembles with four different ensemble constructing techniques, namely bagging, AdaBoost, Arc-X4 and a modified AdaBoost. Twenty real-world data sets from the UCI repository are used as benchmarks to evaluate and compare the performance of these SVM ensemble classifiers by their classification accuracy. Different kernel functions and different numbers of base SVM learners are tested in the ensembles. The experimental results show that although SVM ensembles are not always better than a single SVM, the SVM bagged ensemble performs as well or better than other methods with a relatively higher generality, particularly SVMs with a polynomial kernel function. Finally, an industrial case study of gear defect detection is conducted to validate the empirical analysis results. [Copyright &y& Elsevier]

Abstract: Data mining usually means the approaches and appliances for the valid new knowledge discovery from databases. A response model can be built as a decision model for prediction or classification of a domain problem potential like expert systems. In this paper, a hybrid meta-evolutionary rule mining based approach to assess numerical data pattern in the classification problems is proposed for extracting the decision rules including the predictors, the corresponding inequalities and parameters simultaneously so as to building a decision-making model with maximum classification accuracy. In real world, problems are highly nonlinear in nature so that it’s hard to develop a comprehensive model taking into account all the independent variables through the conventional statistical methods. Recently, nonlinear and complex machine learning approaches such as neural networks and support vector machines have been demonstrated to be with more reliable than the conventional statistical approaches. Although the usefulness of using neural networks and support machines has been reported in literatures, the most obstacles are in model building and use of model in which the classification rules are hard to be realized. Through two numerical experiments, we compared our results against the commercial data mining software and other methods in literature, and then we show experimentally that the proposed approach is promising for improving prediction accuracy and enhancing the modeling simplicity. [Copyright &y& Elsevier]