Search

Classification is an important data mining task that discovers hidden knowledge from the labeled datasets. Most approaches to pruning assume that all dataset are equally uniform and equally important, so they apply equal pruning to all the datasets. However, in real-world classification problems, all the datasets are not equal and considering equal pruning rate during pruning tends to generate a decision tree with large size and high misclassification rate. We approach the problem by first investigating the properties of each dataset and then deriving data-specific pruning value using expert knowledge which is used to design pruning techniques to prune decision trees close to perfection. An efficient pruning algorithm dubbed EKBP is proposed and is very general as we are free to use any learning algorithm as the base classifier. We have implemented our proposed solution and experimentally verified its effectiveness with forty real world benchmark dataset from UCI machine learning repository. In all these experiments, the proposed approach shows it can dramatically reduce the tree size while enhancing or retaining the level of accuracy.

Decision trees induction is among powerful and commonly encountered architecture for extracting of classification knowledge from datasets of labeled instances. However, learning decision trees from large irrelevant datasets is quite different from learning small and moderate sized datasets. In this paper, we propose a simple yet effective composite splitting criterion equal to a random sampling approach and gain ratio. Our random sampling method depends on small random subset of attributes and it is computationally cheap to act on such a set in a reasonable time. The superiority of the composite splitting criterion can persist when used for high dimensional datasets with irrelevant attributes. The empirical and theoretical prospective are validated by using 40 UCI datasets. The experimental results indicate that the proposed new heuristic function can result in much more simpler trees with almost unaffected or improved accuracy.

The ever growing presence of data led to a large number of proposed algorithms for classification and especially decision trees over the last years. Recently, it has been shown that decision trees outperform traditional approaches also on limited data. Therefore, increasing the decision tree classification accuracy yields better performance on both huge and moderate sized datasets. This paper proposes a practical algorithm to deal with the data specific classification problem when there are datasets with different properties. Another key motivation of the data specific pruning in the paper is "trading accuracy and size". A novel algorithm called Expert Knowledge Based Pruning, EKBP is proposed to solve this dilemma. We proposed to integrate error rate, missing values and expert judgment as factors for determining data specific pruning for each dataset. In experimental evaluation against three existing techniques on 40 datasets we showed that our best approach outperforms all competitors and yields significant improvement over previous results in terms of accuracy and tree size.

Data mining tasks results are usually improved by reducing the dimensionality of data. This improvement however is achieved harder in the case that data size is moderate or huge. Although numerous algorithms for accuracy improvement have been proposed, all assume that inducing a compact and highly generalized model is difficult. In order to address above said issue, we introduce Randomized Gini Index (RGI), a novel heuristic function for dimensionality reduction, particularly applicable in large scale databases. Apart from removing irrelevant attributes, our algorithm is capable of minimizing the level of noise in the data to a greater extend which is a very attractive feature for data mining problems. We extensively evaluate its performance through experiments on both artificial and real world datasets. The outcome of the study shows the suitability and viability of our approach for knowledge discovery in moderate and large datasets.

In this paper, we present a practical algorithm to deal with the data specific classification problem when there are datasets with different properties. We proposed to integrate error rate, missing values and expert judgment as factors for determining data specific pruning to form Expert Knowledge Based Pruning (EKBP). We conduct an extensive experimental study on openly available 40 real world datasets from UCI repository. In all these experiments, the proposed approach shows considerably reduction of tree size and achieves equal or better accuracy compared to several bench mark decision tree methods. We have also conducted a case study of heart disease dataset by using our improved algorithm. This study suggests that (Thal), type of defect in heart is the most important predictor for confirming heart disease presence, Number of major vessels colored by fluoroscopy (MV) and type of chest pain (Chest) as biomarkers of heart disease.

In this paper, we propose a new pruning method which is a combination of pre-pruning and post-pruning, aiming on both classification accuracy and tree size. Based upon this method, we induce a decision tree. The experimental results are computed by using 18 benchmark datasets from UCI Machine Learning Repository. The results, when compared to benchmark algorithms, indicate that our new tree pruning method considerably reduces the tree size and increases the accuracy in general. We have also conducted a case study of heart disease dataset by using our improved algorithm. This study suggests that (Thal), type of defect in heart is the most important predictor for confirming the presence of heart disease. Number of major vessels colored by fluoroscopy (MV) and type of chest pain (Chest) as biomarkers of heart disease.

Many traditional pruning methods assume that all the datasets are equally probable and equally important. Thus, they apply equal pruning to all the datasets. However, in real-world classification problems, all the datasets are not equal. Consequently, considering equal pruning rate tends to generate inefficient and large size decision trees. Therefore, we propose a practical algorithm to deal with the data specific classification problem when there are datasets with different properties. In this paper, First, we computed the data specific pruning values for each dataset. Then, we used expert knowledge to find inexact pruning value. Finally, we integrated those values in a well established pruning technique to form Expert Knowledge based Pruning (EKBP). We empirically validated the analysis with publicly available 40 datasets from UCI on four existing techniques. Both the analytical and experimental results have shown that our proposed method achieves reduction of tree size and retains equal or better accuracy.

In this work we investigate several issues in order to improve the performance of decision trees. Firstly, we introduced or adopt a new composite splitting criterion aimed to improve classification accuracy. Secondly, we derive a new pruning technique using expert knowledge, which is able to significantly reduce the size of tree without degrading the classification accuracy. Finally, we implemented our new splitting criterion and pruning technique to form a new decision tree model; Classification Using Randomization and Expert knowledge (CURE). Carried out experiments using 40 UCI datasets on four existing algorithms showed empirical effectiveness of the devised approach.

The ever growing presence of data lead to a large number of proposed algorithms for classification and especially decision trees over the last few years. However, learning decision trees from large irrelevant datasets is quite different from learning small and moderate sized datasets. In practice, use of only small and moderate sized datasets is rare. Unfortunately, the most popular heuristic function gain ratio has a serious disadvantage towards dealing with large and irrelevant datasets. To tackle these issues, we design a new composite splitting criterion with random sampling approach. Our random sampling method depends on small random subset of attributes and it is computationally cheap to act on such a set in a reasonable time. The empirical and theoretical properties are validated by using 40 UCI datasets. The experimental result supports the efficacy of the proposed method in terms of tree size and accuracy.

Many traditional pruning methods assume that all the datasets are equally probable and equally important, so they apply equal pruning to all the datasets. However, in real-world classification problems, all the datasets are not equal and considering equal pruning rate during pruning tends to generate a decision tree with a large size and high misclassification rate.In this paper, we present a practical algorithm to deal with the data specific classification problem when there are datasets with different properties. Another key motivation of the data specific pruning in the paper is "trading accuracy and size". A new algorithm called Expert Knowledge Based Pruning (EKBP) is proposed to solve this dilemma. We proposed to integrate error rate, missing values and expert judgment as factors for determining data specific pruning for each dataset. We show by analysis and experiments that using this pruning, we can scale both accuracy and generalisation for the tree that is generated. Moreover, the method can be very effective for high dimensional datasets. We conduct an extensive experimental study on openly available 40 real world datasets from UCI repository. In all these experiments, the proposed approach shows considerably reduction of tree size having equal or better accuracy compared to several benchmark decision tree methods that are proposed in literature.