Search

Class noise is a common issue that affects the performance of classification techniques on real-world data sets. Class noise appears when a class variable in data sets has incorrect class labels. In the case of noisy data, the robustness of classification techniques against noise could be more important than the performance results on noise-free data sets. The decision tree method is one of the most popular techniques for classification tasks. The C4.5, CART, and random forest (RF) algorithms are considered to be three of the most used algorithms in decision trees. The aim of this paper is to reach conclusions on which decision tree algorithm is better to use for building decision trees in terms of its performance and robustness against class noise. In order to achieve this aim, we study and compare the performance of the models when applied to class variables with noise. The results obtained indicate that the RF algorithm is more robust to data sets with noisy class variable than other algorithms. [ABSTRACT FROM AUTHOR]

In the contemporary world, data pervades every facet of human life, and the information contained in this data plays a pivotal role in shaping decision-making and advancing technology. Among the plethora of techniques available, classification methods are highly effective tools for extracting valuable insights from vast volumes of data. The R ule A ggregation C lassifi ER (RACER) is a novel rule-based classification algorithm known for its exceptional performance. A notable limitation of RACER lies in its inability to handle continuous features. In this paper, we address the aforementioned limitation by employing various supervised discretization methods, including CAIM, MDLP, Decision Tree (CART), and ChiMerge. The impact of these methods on RACER's accuracy and understandability is evaluated across nine datasets from the UCI repository. Additionally, the paper conducts a comparative analysis of RACER's accuracy against well-known classifiers such as Naive Bayes, Logistic Regression, SVM, LightGBM, and Decision Tree. The findings indicate that RACER achieves the highest average accuracy when we utilize MDLP as the discretization method, surpassing its initial average accuracy. Moreover, RACER demonstrates superior understandability by generating the lowest number of rules when employing ChiMerge and Decision Tree for discretizing numerical features. Furthermore, RACER outperforms the other five classifiers when employing MDLP. • The discretization unit enables the RACER algorithm to handle continuous data. • The discretization unit improves the classification accuracy of the RACER. • The discretization unit increases the understandability of the RACER rules. [ABSTRACT FROM AUTHOR]

The deep decision tree migration learning boosting method ( DtrBoost) can effectively realize the migration learning from a single source domain to a target domain under supervision, but can not realize the unsupervised migration scenario under multiple source domains. To solve this problem, this paper proposed an unsupervised transfer learning boosting method for optimizing the weight under multi-source domain distribution. The main idea was to calculate the corresponding KL value according to the distribution of different source domains and target domains, selected an appropriate number of samples from different source domains to train the classifier and pseudo label the samples from the target domain. Finally, the algorithm assigned different learning weights according to the KL distance of each different source domain, and the labeled source domain samples integrated to the pseudo labeled target domain to obtain the final result. Comparative experiments show that the proposed algorithm achieves better classification accuracy and adaptive effect on different data sets. The average classification error rate decreases by 2. 4% and more than 6% on the best marketing data set. [ABSTRACT FROM AUTHOR]

The project aims to generate a model to predict the likelihood of an individual being affected by heart disease in the next 10 years. Many hospitals generate data about patients that can identify early symptoms by identifying the patterns and facts present in them. This huge data makes it difficult for the Doctors to identify patterns. As the old saying goes "Prevention is better than Cure", early detection and continuous management by clinicians will bring down the mortality rate. All that said, it's impractical to observe patients daily to accurately predict and provide 24x7 consultations, since it requires heaps of knowledge and time. During this paper we have developed and researched models to predict heart disease through multifarious heart attributes of patients and identified machine learning techniques just round the corner to effectively predict heart diseases. Data Science, with its vast applied capabilities, has a key role in processing mammoth amounts of data to extract meaningful insights from it in every walk of life, with specific mention of healthcare services. In this paper the Naive Bayes Algorithm is used and it is built on the Framingham dataset. And this data is divided as training of data and testing of data in ratio of 80:20. The training set is used to train the classifier and test data is used to generate a confusion matrix to calculate accuracy. Several techniques can be used to solve this problem. The proposed Naive Bayes algorithm has achieved an accuracy of approximately 81.5% on the testing data, which is greater than Decision Tree accuracy. [ABSTRACT FROM AUTHOR]

Support vector machines (SVMs) are designed to solve the binary classification problems at the beginning, but in the real world, there are a lot of multiclassification cases. The multiclassification methods based on SVM are mainly divided into the direct methods and the indirect methods, in which the indirect methods, which consist of multiple binary classifiers integrated in accordance with certain rules to form the multiclassification model, are the most commonly used multiclassification methods at present. In this paper, an improved multiclassification algorithm based on the balanced binary decision tree is proposed, which is called the IBDT-SVM algorithm. In this algorithm, it considers not only the influence of "between-classes distance" and "class variance" in traditional measures of between-classes separability but also takes "between-classes variance" into consideration and proposes a new improved "between-classes separability measure." Based on the new "between-classes separability measure," it finds out the two classes with the largest between-classes separability measure and uses them as the positive and negative samples to train and learn the classifier. After that, according to the principle of the class-grouping-by-majority, the remaining classes are close to these two classes and merged into the positive samples and the negative samples to train SVM classifier again. For the samples with uneven distribution or sparse distribution, this method can avoid the error caused by the shortest canter distance classification method and overcome the "error accumulation" problem existing in traditional binary decision tree to the greatest extent so as to obtain a better classifier. According to the above algorithm, each layer node of the decision tree is traversed until the output classification result is a single-class label. The experimental results show that the IBDT-SVM algorithm proposed in this paper can achieve better classification accuracy and effectiveness for multiple classification problems. [ABSTRACT FROM AUTHOR]

An intrusion detection system is a security system that aims to detect sabotage and intrusions on networks to inform experts of the attack and abuse of the network. Different classification methods have been used in the intrusion detection systems such as fuzzy, genetic algorithms, decision trees, artificial neural networks, and support vector machines. Moreover, ensemble classifiers have shown more robust and effective performance for various tasks in the field. In this paper, we adopt ensemble models in order to improve the performance of intrusion detection and, at the same time, decrease the false alarm rate. We use kNN for multi-class classification, as well as SVM to approach the classification problem in normal-based detection. In order to combine multiple outputs, we use the Dempster–Shafer method in which there is the possibility of explicit retrieval of uncertainty. Moreover, we utilize deep learning for extracting features to train the samples, selected by the sample selection algorithm based on ensemble margin. We compare our results with state-of-the-art methods on benchmarking datasets such as UNSW-NB15, CICIDS2017, and NSL-KDD. Our proposed method indicates the superiority in terms of prominent metrics Accuracy, Precision, Recall, and F-measure. [ABSTRACT FROM AUTHOR]

The diagnosis of heart disease is found to be a serious concern, so the diagnosis has to be done remotely and regularly to take the prior action. In the present years, the diagnosis of heart disease has become a key research area for researchers and many models have been proposed in recent years. The diagnosis of heart disease can be done using optimization algorithms, and it provides results with good efficiency. The main objective of this paper is to propose a hybrid fuzzy-based decision tree algorithm for the process of prediction of heart disease at an early stage through the continuous and remote patient monitoring system. The results obtained from the proposed algorithm are compared with the various number of classifier algorithms like decision tree J48, naïve Bayes, GA with FCM, KNN with NB, ANN, SVM with fuzzy in which the proposed HFDT algorithm provides better accuracy of 98.30%. From the above-obtained results, the proposed hybrid fuzzy-based decision tree algorithm efficiently predicts heart disease compared to the other classifier algorithms in the literature. The proposed work is implemented in the MATLAB environment using the heart disease dataset. [ABSTRACT FROM AUTHOR]

In classification, a decision tree is a common model due to its simple structure and easy understanding. Most of decision tree algorithms assume all instances in a dataset have the same degree of confidence, so they use the same generation and pruning strategies for all training instances. In fact, the instances with greater degree of confidence are more useful than the ones with lower degree of confidence in the same dataset. Therefore, the instances should be treated discriminately according to their corresponding confidence degrees when training classifiers. In this paper, we investigate the impact and significance of degree of confidence of instances on the classification performance of decision tree algorithms, taking the classification and regression tree (CART) algorithm as an example. First, the degree of confidence of instances is quantified from a statistical perspective. Then, a developed CART algorithm named C_CART is proposed by introducing the confidence of instances into the generation and pruning processes of CART algorithm. Finally, we conduct experiments to evaluate the performance of C_CART algorithm. The experimental results show that our C_CART algorithm can significantly improve the generalization performance as well as avoiding the over-fitting problem to a certain extend. [ABSTRACT FROM AUTHOR]

Highlights • A novel modeling strategy for Decision Trees is proposed. • Interactive basis functions induce oblique and non-linear decision boundaries. • Benchmarking shows potential gains in accuracy and/or parsimony of models. • Empirical examples illustrate the application of the modeling strategy. • Decision charts are introduced to improve the interpretability of models. Abstract Decision Trees (DTs) are a machine learning technique widely used for regression and classification purposes. Conventionally, the decision boundaries of Decision Trees are orthogonal to the features under consideration. A well-known limitation of this is that the algorithm may fail to find optimal partitions, or in some cases any partitions at all, depending on the underlying distribution of the data. To remedy this limitation, several modifications have been proposed that allow for oblique decision boundaries. The objective of this paper is to propose a new strategy for generating flexible decision boundaries by means of interactive basis functions (IBFs). We show how oblique decision boundaries can be obtained as a particular case of IBFs, and in addition how non-linear decision boundaries can be induced. One attractive aspect of the strategy proposed in this paper is that training Decision Trees with IBFs does not require custom software, since the functions can be precalculated for use in any existing implementation of the algorithm. Since the underlying mechanisms remain unchanged there is no substantial computational overhead compared to conventional trees. Furthermore, this also means that IBFs can be used in any extensions of the Decision Tree algorithm, such as evolutionary trees, boosting, and bagging. We conduct a benchmarking exercise to understand under which conditions the use of IBFs can improve model the performance. In addition, we present three empirical applications that illustrate the approach in classification and regression. As part of discussing the empirical applications, we introduce a device called decision charts to facilitate the interpretation of DTs with IBFs. Finally, we conclude the paper by outlining some directions for future research. [ABSTRACT FROM AUTHOR]

Due to the high-speed requirement of high-end network devices, hardware using TCAMs has been the dominant implementation of packet classification in industry. Despite its capability for line-speed queries, TCAM is not only power hungry but also capacity inefficient. By making use of a pre-classifier to activate TCAM blocks selectively, many research efforts significantly reduce the power consumption of TCAM. However, these bottom-up based pre-classifiers achieve power savings at the expense of poor utilization of TCAM capacity, and the potential of power reduction is not fully exploited in many cases. This paper proposed a power-saving pre-classifier for TCAM based packet classification, which constructed based on decision trees. By grouping rules with respect to their small fields, rules could be recursively mapped into decision trees without the trouble of rule replications, so that a top-down traversal algorithm could be well applied for obtaining index items. Experimental results show that for rule sets up to one hundred thousand entries, the proposed design achieves 98. 2% power reduction with a TCAM storage overhead of 1. 3% on average. [ABSTRACT FROM AUTHOR]

In order to overcome the problems of poor accuracy and high complexity of current classification algorithm for non-equilibrium data set, this paper proposes a decision tree classification algorithm for non-equilibrium data set based on random forest. Wavelet packet decomposition is used to denoise non-equilibrium data, and SNM algorithm and RFID are combined to remove redundant data from data sets. Based on the results of data processing, the non-equilibrium data sets are classified by random forest method. According to Bootstrap resampling method with certain constraints, the majority and minority samples of each sample subset are sampled, CART is used to train the data set, and a decision tree is constructed. Obtain the final classification results by voting on the CART decision tree classification. Experimental results show that the proposed algorithm has the characteristics of high classification accuracy and low complexity, and it is a feasible classification algorithm for non-equilibrium data set. [ABSTRACT FROM AUTHOR]

This paper proposes a new content recommendation system which combines the newly proposed embedded feature selection method and the new Fuzzy Temporal Logic based Decision Tree incorporated Convolutional Neural Network classifier. The newly proposed embedded feature selection called Fuzzy Decision Tree and Weighted Gini-Index based Feature Selection Algorithm (FDTWGI-FSA) that contains the existing incorporated the Fuzzy Decision Tree (FDT) and the Weighted Gini-index based Feature Selection Algorithm (WGIFSA) for getting optimized feature subset. Moreover, an enhanced CNN and Fuzzy Temporal Decision Tree for performing the deep learning process which is able to identify the exact e-content from the huge volume of data with the help of the recommended features by the proposed embedded feature selection method. The exact e-content can be identified after performing the five-layer network structure for extracting the relevant features and it also can be classified by applying the Fuzzy Temporal Decision Tree for the e-learners. Finally, the proposed content recommendation system provides exact content to the e-learners according to their level of understanding and it also satisfies them by providing the exact high level contents. The experiments have been conducted for evaluating the proposed content recommendation system and compared with the existing classifier including the standard CNN. [ABSTRACT FROM AUTHOR]

Summary: This paper presents a protection method for microgrids by data mining of voltage disturbances. The occurrence of a fault on the system is associated with a sudden voltage depression, which is fast detected by the adaptive cumulative sum (ACUSUM) algorithm. There are other operational (no‐fault) events causing voltage depression such as motor starting, transformer energizing, and capacitor or heavy load switching. In order to discriminate between the fault and no‐fault events, one cycle of the voltage waveform is preprocessed by the short‐time Fourier transform (STFT) to extract and construct effective features of the disturbance. The features are then used in the decision trees (DTs) for the discrimination. The proposed protection method is tested for fault or no‐fault conditions of grid‐connected or islanded mode of the microgrid operation, as well as radial or meshed topology. The proposed method also identifies the fault type and faulted phase(s) for selective phase tripping. The immunity of the method against different noise levels is investigated. It is shown by the simulation study that by using only two features for symmetrical events and six features for asymmetrical events, any fault can be detected accurately. [ABSTRACT FROM AUTHOR]

Expert and intelligent systems understand the underlying information behind the data by relying on a wide range of machine learning techniques. The interpretation of machine learning models is often the key to success in research areas such as business, finance, medical and health science, and bioinformatics; such research areas demand human understanding of the obtained model. The logistic model tree (LMT) algorithm is a popular classification method that combines a decision tree and logistic regression models. The combination of two complementary algorithms produces an accurate and interpretable classifier by combining the advantages of both logistic regression and tree induction. However, LMT has the disadvantage of high computational cost, which makes the algorithm undesirable in practice. In this paper, we propose an efficient method to learn the logistic regression models in the tree. We employ least angle regression to update the regression model in LogitBoost so that the algorithm efficiently learns sparse logistic regression models composed of relevant input variables. We compare the performance of our proposed method with the original LMT algorithm using 14 benchmark datasets and show that the training time dramatically decreases while the accuracy is preserved. Our proposed algorithm is not only accurate and intuitively interpretable but also computationally efficient. It helps users in making the best possible use of the data that are included in expert and intelligent systems. [ABSTRACT FROM AUTHOR]

Aim This paper presents a discussion of classification and regression tree analysis and its utility in nursing research. Background Classification and regression tree analysis is an exploratory research method used to illustrate associations between variables not suited to traditional regression analysis. Complex interactions are demonstrated between covariates and variables of interest in inverted tree diagrams. Design Discussion paper. Data sources English language literature was sourced from eBooks, Medline Complete and CINAHL Plus databases, Google and Google Scholar, hard copy research texts and retrieved reference lists for terms including classification and regression tree* and derivatives and recursive partitioning from 1984-2013. Discussion Classification and regression tree analysis is an important method used to identify previously unknown patterns amongst data. Whilst there are several reasons to embrace this method as a means of exploratory quantitative research, issues regarding quality of data as well as the usefulness and validity of the findings should be considered. Implications for Nursing Research Classification and regression tree analysis is a valuable tool to guide nurses to reduce gaps in the application of evidence to practice. With the ever-expanding availability of data, it is important that nurses understand the utility and limitations of the research method. Conclusion Classification and regression tree analysis is an easily interpreted method for modelling interactions between health-related variables that would otherwise remain obscured. Knowledge is presented graphically, providing insightful understanding of complex and hierarchical relationships in an accessible and useful way to nursing and other health professions. [ABSTRACT FROM AUTHOR]

Fuzzy Decision Tree (FDT) classifiers combine decision trees with approximate reasoning offered by fuzzy representation to deal with language and measurement uncertainties. When a FDT induction algorithm utilizes stopping criteria for early stopping of the tree's growth, threshold values of stopping criteria will control the number of nodes. Finding a proper threshold value for a stopping criterion is one of the greatest challenges to be faced in FDT induction. In this paper, we propose a new method named Iterative Deepening Fuzzy ID3 (IDFID3) for FDT induction that has the ability of controlling the trees growth via dynamically setting the threshold value of stopping criterion in an iterative procedure. The final FDT induced by IDFID3 and the one obtained by common FID3 are the same when the numbers of nodes of induced FDTs are equal, but our main intention for introducing IDFID3 is the comparison of different stopping criteria through this algorithm. Therefore, a new stopping criterion named Normalized Maximum fuzzy information Gain multiplied by Number of Instances (NMGNI) is proposed and IDFID3 is used for comparing it against the other stopping criteria. Generally speaking, this paper presents a method to compare different stopping criteria independent of their threshold values utilizing IDFID3. The comparison results show that FDTs induced by the proposed stopping criterion in most situations are superior to the others and number of instances stopping criterion performs better than fuzzy information gain stopping criterion in terms of complexity (i.e. number of nodes) and classification accuracy. Also, both tree depth and fuzzy information gain stopping criteria, outperform fuzzy entropy, accuracy and number of instances in terms of mean depth of generated FDTs. [ABSTRACT FROM AUTHOR]

Discretization is an essential preprocessing technique used in many knowledge discovery and data mining tasks. Its main goal is to transform a set of continuous attributes into discrete ones, by associating categorical values to intervals and thus transforming quantitative data into qualitative data. In this manner, symbolic data mining algorithms can be applied over continuous data and the representation of information is simplified, making it more concise and specific. The literature provides numerous proposals of discretization and some attempts to categorize them into a taxonomy can be found. However, in previous papers, there is a lack of consensus in the definition of the properties and no formal categorization has been established yet, which may be confusing for practitioners. Furthermore, only a small set of discretizers have been widely considered, while many other methods have gone unnoticed. With the intention of alleviating these problems, this paper provides a survey of discretization methods proposed in the literature from a theoretical and empirical perspective. From the theoretical perspective, we develop a taxonomy based on the main properties pointed out in previous research, unifying the notation and including all the known methods up to date. Empirically, we conduct an experimental study in supervised classification involving the most representative and newest discretizers, different types of classifiers, and a large number of data sets. The results of their performances measured in terms of accuracy, number of intervals, and inconsistency have been verified by means of nonparametric statistical tests. Additionally, a set of discretizers are highlighted as the best performing ones. [ABSTRACT FROM AUTHOR]

The classification can be performed by using the decision tree approach. Previous researches on the classification using the decision tree have mostly been intended to classify text data. This paper was intended to introduce a classification application to the content-based image retrieval (CBIR) with multiattributes by using a decision tree. The attributes used were the visual features of the image, i.e. : color moments (order 1, 2 and 3), image entropy, energy and homogeneity. K-means cluster algorithm was used to categorize each attribute. The result of categorized data was then built into a decision tree by using C4.5. To show the concept in application, this research built an application with main features, i.e.: cases data input, cases list, training process and testing process to do classification. The resulting tests of 150 rontgen data showed the training data classification's truth value of 75.33% and testing data classification of 55.7%. [ABSTRACT FROM AUTHOR]

In many real-world applications, instances (data) arrive sequentially in the form of streams. Processing such data poses challenges to machine learning. While adhering to on-line learning strategies, in this paper we extend the Flexible Fuzzy Decision Tree (FlexDT) algorithm with multiple partitioning that makes it possible to carry out automatic on-line fuzzy data classification. The proposed method is aimed to balance accuracy and tree size in data stream mining. The objective of the classification problem is to predict the true class of each incoming instances in real time. In terms of evaluation of the method, accuracy, tree depth, and the learning time are significant factors influencing the performance. A series of experiments demonstrate that the proposed method produces optimal trees for both numeric and nominal features (variables). [ABSTRACT FROM AUTHOR]

This paper presents a survey of evolutionary algorithms that are designed for decision-tree induction. In this context, most of the paper focuses on approaches that evolve decision trees as an alternate heuristics to the traditional top-down divide-and-conquer approach. Additionally, we present some alternative methods that make use of evolutionary algorithms to improve particular components of decision-tree classifiers. The paper’s original contributions are the following. First, it provides an up-to-date overview that is fully focused on evolutionary algorithms and decision trees and does not concentrate on any specific evolutionary approach. Second, it provides a taxonomy, which addresses works that evolve decision trees and works that design decision-tree components by the use of evolutionary algorithms. Finally, a number of references are provided that describe applications of evolutionary algorithms for decision-tree induction in different domains. At the end of this paper, we address some important issues and open questions that can be the subject of future research. [ABSTRACT FROM PUBLISHER]

Abstract: The wavelet transform (WT) is used to represent all possible types of transients in vibration signals generated by faults in a gear box. It is shown that the transform provides a powerful tool for condition monitoring and fault diagnosis. The vibration signal of a spur bevel gear box in different conditions is used to demonstrate the application of various wavelets in feature extraction. In present work, a discrete wavelet, Daubechies wavelets (db1–db15) is used for feature extraction and their relative effectiveness in feature extraction is compared. The major steps in pattern classification are feature extraction and classification. This paper investigates the use of discrete wavelets for feature extraction and a Decision Tree for classification. J48 Decision Tree algorithm has been used for feature selection as well as for classification. This paper illustrates the powerfulness and flexibility of the discrete wavelet transform to decompose linear and non-linear processing of vibration signal. [Copyright &y& Elsevier]

Data mining (DM) can be defined as the non-trivial process of identifying valid, novel, potentially useful and ultimately understandable patterns in data. Modelling is the crucial step where DM algorithms are applied in order to extract data patterns. In order for domain experts, who play significant roles in DM process, to make the most efficient and effective use of DM tools, these tools must incorporate appropriate visualization to facilitate the process of modelling. Yet, unfortunately, study of how visualization should be designed, particularly what components should be included and how to present them, has been rather limited. This paper surveys the current state of art in application of visualization techniques to better comprehend and improve the decision trees modelling process in three modes: visualization of tree models, visualization of model evaluation and visual interactive tree construction. A number of issues that have been overlooked and areas that need to be improved are identified through reviewing a collection of related research and examining six current DM softwares in terms of their design of a few important features in each mode of the visualization support to decision trees classification modelling. Although this article focuses on decision trees classification modelling, guidelines derived from this study can be beneficial to other modelling techniques as well. At the end of the paper, a desirable design of visualization support to DM modelling is proposed with a conceptual model. [ABSTRACT FROM AUTHOR]

A shortcoming of univariate decision tree learners is that they do not learn intermediate concepts and select only one of the input features in the branching decision at each intermediate tree node. It has been empirically demonstrated that cascading other classification methods, which learn intermediate concepts, with decision tree learners can alleviate such representational bias of decision trees and potentially improve classification performance. However, a more complex model that fits training data belier may not necessarily perform better on unseen data, commonly referred to as the overfitting problem. To find the most appropriate degree of such cascade generalization, a decision forest (i.e., a set of decision trees with other classification models cascaded to different degrees) needs to be generated, from which the best decision tree can then be identified. In this paper, the authors pro- pose an efficient algorithm for generating such decision forests. The algorithm uses an extended decision tree data structure and constructs any node that is common to multiple decision trees only once. The authors have empirically evaluated the algorithm using 32 data sets for classification problems from the University of California, Irvine (UCI) machine learning repository and report on results demonstrating the efficiency of the algorithm in this paper. [ABSTRACT FROM AUTHOR]

In many data mining tasks, the goal is to classify entities into a set of pre-defined groups (classes). A second and equally important goal is the interpretation, i.e. understanding the nature of the population aggregated in each class. These tasks are rendered even more complex when there is no a-priori information regarding the right classification. The current paper is based on two concepts: (1) Bounded-Rationality theory which implements an S-shaped function that represents human logic as a saliency measure to determine the substantial features that characterize each potential group and (2) Classification by clustering (CBC) that applies Decision Tree-like classification in unsupervised clustering problems, where neither an a-priori classification nor target-attributes are known in advance. In the context of these two concepts, the current research contributes: (1) by expanding the saliency measure to all possible types of variables (nominal as well as numerical), (2) by evaluating, using five datasets, a composite model that combines the CBC method and the saliency concept. The findings show that by using clustering algorithms for classification tasks (CBC method) the results are as accurate as those obtained by conventional Decision Trees, but with a better saliency factor. [ABSTRACT FROM AUTHOR]

Cardiovascular Disease (CVD) is one of the most catastrophic and life threatening health issue nowadays. Early detection of CVD is an important solution to reduce its devastating effects on health. In this paper, an efficient CVD detection algorithm is identified. The algorithm uses patient demographic data as inputs, along with several ECG signal features extracted automatically through signal processing techniques. Cross-validation results show a 98.29 % accuracy for the decision tree classification algorithm. The algorithm has been integrated into a web based system that can be used at anytime by patients to check their heart health status. At one end of the system is the ECG sensor attached to the patient's body, while at the other end is the detection algorithm. Communication between the two ends is done through an Android application. [ABSTRACT FROM AUTHOR]

Background: This paper presents an automatic method for assessing temperature distribution on the patient's back using thermal imaging. It discusses the advantages and disadvantages of thermal imaging and presents an algorithm for image analysis and processing that runs automatically and reproducibly. Method: The proposed new method of image analysis relates to automatic and reproducible analysis of temperature distribution on the patient's back. It includes the following steps: (1) detection of the external contours of the human body, (2) identification and recognition of the position of the arms, hips and shoulders, (3) application of a mesh having a fixed number of fields (which automatically adapts to the size of the patient's back), (4) analysis of the paraspinal muscles. These steps are preceded by image pre-processing. Results: Based on a thermal image, this new fully automated algorithm enables to: (1) evaluate temperature distribution in the paraspinal area; (2) pre-assess the degree of lateral spinal curvature in screening, and (3) evaluate anomalies (deviations from the norm) in temperature distribution. The algorithm was practically implemented in Matlab Image Processing Toolbox. [ABSTRACT FROM AUTHOR]

• A classification criteria that enlarges and generalizes security boundary methods. • It classifies the operating space based on the nearest Saddle Node Bifurcation (SNB). • A sampling method that reduces computation burden by exploiting SNB conditions. • An algorithm that creates a database of operating points to train decision trees. • A testing method for decision trees accuracy prediction using time-domain simulation. This paper presents a method that performs classification of thousands of operating conditions w.r.t. power system voltage stability by using decision trees. The proposed method uses a new and flexible classification criterion that allows to identify operating conditions that are near or within the region for which the system is voltage unstable, and more importantly, that can consider operational requirements. The method creates both training and test data sets when building and validating the decision trees. To minimize computational burden, a sampling method is proposed, this method exploits the Saddle Node Bifurcation conditions to explore the operational space used to train the decision trees. Case studies were performed using the IEEE 9-bus system for several operating conditions and different network configurations. This paper also proposes the use of time domain simulations to assess the prediction accuracy of decision trees. Decision trees were created for network configurations involving outage of the line were tested on test sets and also using time domain simulations results from PSS/E. The ability to classify the degree of voltage stability of a multitude of operation conditions could be useful to aid operators in selecting and applying preventive measures to steer away the system from unstable conditions or conditions that are close to breaching operational requirements w.r.t. voltage stability. [ABSTRACT FROM AUTHOR]

• Discretization is an important process in terms of both machine learning and data mining. • The effect of discretization on ensemble methods was analyzed. • Four Chi-square based discretization algorithms were used. • The Ensemble methods performed better on discrete data sets. Classification is one of the important tasks in data mining and machine learning. Classification performance depends on many factors as well as data characteristics. Some algorithms are known to work better with discrete data. In contrast, most real-world data contain continuous variables. For algorithms working with discrete data, these continuous variables must be converted to discrete ones. In this process called discretization, continuous variables are converted to their corresponding discrete variables. In this paper, four Chi-square based supervised discretization algorithms ChiMerge(ChiM), Chi2, Extended Chi2(ExtChi2) and Modified Chi2(ModChi2) were used. In the literature, the performance of these algorithms is often tested with decision trees and Naïve Bayes classifiers. In this study, differently, four sets of data discretized by these algorithms were classified with ensemble methods. Classification accuracies for these data sets were obtained through using a stratified 10-fold cross-validation method. The classification performance of the original and discrete data sets of the methods is presented comparatively. According to the results, the performance of the discrete data is more successful than the original data. [ABSTRACT FROM AUTHOR]

Abstract: For high dimensional data a large portion of features are often not informative of the class of the objects. Random forest algorithms tend to use a simple random sampling of features in building their decision trees and consequently select many subspaces that contain few, if any, informative features. In this paper we propose a stratified sampling method to select the feature subspaces for random forests with high dimensional data. The key idea is to stratify features into two groups. One group will contain strong informative features and the other weak informative features. Then, for feature subspace selection, we randomly select features from each group proportionally. The advantage of stratified sampling is that we can ensure that each subspace contains enough informative features for classification in high dimensional data. Testing on both synthetic data and various real data sets in gene classification, image categorization and face recognition data sets consistently demonstrates the effectiveness of this new method. The performance is shown to better that of state-of-the-art algorithms including SVM, the four variants of random forests (RF, ERT, enrich-RF, and oblique-RF), and nearest neighbor (NN) algorithms. [Copyright &y& Elsevier]

Many decision tree algorithms were proposed over the last few decades. A lack of publishing standards for decision tree algorithm software produced a large time gap between algorithm proposals and their wider application in practice. Non-existence of common repository for storing algorithms and their parts led to a need to re-implement these algorithms from a scratch when they had to be implemented on a different platform. This makes the comparison between algorithms and their partial improvements vague. In addition, combinations and interactions between different algorithm parts haven't been analyzed thoroughly. Reusable component design of decision tree algorithms has been recently suggested as a potential solution to these problems. In this paper we describe an architecture for component-based (white-box) decision tree algorithm design, and we present an open-source framework which enables design and fair testing of decision tree algorithms and their parts. This architecture and developed platform can provide the research community with a common codebase for storing, designing, and evaluating decision tree algorithms (traditional, multivariate and hybrid) and their partial improvements. It is intended for data mining practitioners, algorithm and software developers, and as well for students, as a technology enhanced learning tool. [ABSTRACT FROM AUTHOR]

Abstract: Given the pace and scale of urban expansion in many parts of the globe, urban environments are playing an increasingly important role in daily quality-of-life issues, ecological processes, climate, material flows, and land transformations. Remote sensing has emerged as a powerful tool to monitor rates and patterns of urban expansion, but many early challenges – such as distinguishing new urban land from bare ground – remain unsolved. To deal with the high temporal and spatial variability as well as complex, multi-signature classes within settlements, this paper presents a new approach that exploits multi-seasonal information in dense time stacks of Landsat imagery using a multi-date composite change detection technique. The central premise of the approach is that lands within/near urban areas have distinct temporal trajectories both before and after change occurs, and that these lead to characteristic temporal signatures in several spectral regions. The method relies on a supervised classification that exploits training data of stable/changed areas interpreted from Google Earth images, and a ‘brute force’ approach of providing all available Landsat data as input, including scenes with data gaps due to the Scan Line Corrector (SLC) problem. Three classification algorithms (maximum likelihood, boosted decision trees, and support vector machines) were tested for their ability to monitor expansion across five time periods (1988–1995, 1996–2000, 2001–2003, 2004–2006, 2007–2009) in three study areas that differ in size, eco-climatic conditions, and rates/patterns of development. Both the decision trees and support vector machines outperformed the maximum likelihood classifier (overall accuracy of 90–93%, compared to 65%), but the decision trees were superior at handling missing data. Adding transformed features such as band metrics to the Landsat data stack increased accuracy 1–4%, while experiments with a reduced number of features (designed to mimic noisy or missing data) led to a drop in accuracy of 1–9%. The methodology also proved particularly effective for monitoring peri-urbanization outside the urban core, capturing >98% of village settlements. [Copyright &y& Elsevier]

Decision trees are commonly used in supervised classification. Currently, supervised classification problems with large training sets are very common, however many supervised classifiers cannot handle this amount of data. There are some decision tree induction algorithms that are capable to process large training sets, however almost all of them have memory restrictions because they need to keep in main memory the whole training set, or a big amount of it. Moreover, algorithms that do not have memory restrictions have to choose a subset of the training set, needing extra time for this selection; or they require to specify the values for some parameters that could be very difficult to determine by the user. In this paper, we present a new fast heuristic for building decision trees from large training sets, which overcomes some of the restrictions of the state of the art algorithms, using all the instances of the training set without storing all of them in main memory. Experimental results show that our algorithm is faster than the most recent algorithms for building decision trees from large training sets. [ABSTRACT FROM AUTHOR]

Abstract: Numerous algorithms have been proposed to improve Naive Bayes (NB) by weakening its conditional attribute independence assumption, among which Tree Augmented Naive Bayes (TAN) has demonstrated remarkable classification performance in terms of classification accuracy or error rate, while maintaining efficiency and simplicity. In many real-world applications, however, classification accuracy or error rate is not enough. For example, in direct marketing, we often need to deploy different promotion strategies to customers with different likelihood (class probability) of buying some products. Thus, accurate class probability estimation is often required to make optimal decisions. In this paper, we investigate the class probability estimation performance of TAN in terms of conditional log likelihood (CLL) and present a new algorithm to improve its class probability estimation performance by the spanning TAN classifiers. We call our improved algorithm Averaged Tree Augmented Naive Bayes (ATAN). The experimental results on a large number of UCI datasets published on the main web site of Weka platform show that ATAN significantly outperforms TAN and all the other algorithms used to compare in terms of CLL. [Copyright &y& Elsevier]

Abstract: This paper introduces the current machine learning approach to solving spatial modeling problems in the domain of landslide susceptibility assessment. The latter is introduced as a classification problem, having multiple (geological, morphological, environmental etc.) attributes and one referent landslide inventory map from which to devise the classification rules. Three different machine learning algorithms were compared: Support Vector Machines, Decision Trees and Logistic Regression. A specific area of the Fruška Gora Mountain (Serbia) was selected to perform the entire modeling procedure, from attribute and referent data preparation/processing, through the classifiers'' implementation to the evaluation, carried out in terms of the model''s performance and agreement with the referent data. The experiments showed that Support Vector Machines outperformed the other proposed methods, and hence this algorithm was selected as the model of choice to be compared with a common knowledge-driven method – the Analytical Hierarchy Process – to create a landslide susceptibility map of the relevant area. The SVM classifier outperformed the AHP approach in all evaluation metrics (κ index, area under ROC curve and false positive rate in stable ground class). [Copyright &y& Elsevier]

Abstract: The management of atmospheric pollution using video is not yet widespread. However it is an efficient way to evaluate the polluting rejects coming from large industrial facilities when traditional captors are not usable. This paper presents a comparison of different classifiers for a monitoring system of polluting smokes. The data used in this work are stemming from a system of video analysis and signal processing. The database includes the pollution level of puffs of smoke defined by an expert. Six machine learning techniques are tested and compared to classify the puffs of smoke: k-nearest neighbour, naïve Bayes classifier, artificial neural network, decision tree, support vector machine and a fuzzy model. The parameters of each type of classifier are split into three categories: learned parameters, parameters determined by a first step of the experimentation, and parameters set by the programmer. We compare the results of the best classifier of each type depending on the size of the learning set. A part of the discussion concerns the robustness of the classifier facing the case where classes of interest are under-represented, as the high level of pollution in our data. [Copyright &y& Elsevier]

Abstract: The paper presents results of application of a rule induction and pruning algorithm for classification of a microseismic hazard sate in coal mines. Due to imbalanced distribution of examples describing states “hazardous” and “safe”, the special algorithm was used for induction and rule pruning. The algorithm selects optimal parameters‘ values influencing rule induction and pruning based on training and tuning sets. A rule quality measure which decides about a form and classification abilities of rules that are induced is the basic parameter of the algorithm. The specificity and sensitivity of a classifier were used to evaluate its quality. Conducted tests show that the admitted method of rules induction and classifier’s quality evaluation enables to get better results of classification of microseismic hazards than by methods currently used in mining practice. Results obtained by the rules-based classifier were also compared with results got by a decision tree induction algorithm and by a neuro-fuzzy system. [Copyright &y& Elsevier]

Abstract: In this paper, we propose a hierarchical fuzzy clustering decision tree (HFCDT) for the classification problem with large number of classes and continuous attributes. The HFCDT combines a division-degree matrix based hierarchal clustering technique with the entropy-based C4.5 decision tree algorithm. A hierarchical clustering concept is introduced to achieve a finer fuzzy partition. The hierarchical clustering technique splits the data set into leaf clusters using splitting attributes based on a division-degree matrix and fuzzy rules. The leaf clusters consisting of the data of more than one class will be further classified using the C4.5 algorithm. We have successfully applied the HFCDT for classifying recipes of the working wafers in an ion implanter, and compared the classification results and the training time with the existing software See5 and CART. The comparison results show that the HFCDT not only performs better than See5 and CART in the aspect of 10-fold cross validation for the average of total classification error rates but also consumes less training time. Thus, HFCDT obtains a very successful classification result. This also demonstrates why the hierarchical clustering technique helps reduce the computational complexity of the C4.5 algorithm. [Copyright &y& Elsevier]

Abstract: Multi-valued and multi-labeled learning is concerned with samples associated with a set of values both with label and attribute. This paper proposes a new learning framework, which combines multi-valued attribute decomposition with multi-label learning. To deal with multi-valued attribute, we present five methods which differ in strategies with the correlations of multi values. After data transformation, three classic multi-label algorithms are employed for learning. Experimental results demonstrate that most combined methods significantly outperform the existing decision tree based algorithms. Furthermore, exploring the advantages and limitations of each combined method, we find the optimal combination corresponding to different types of datasets. [ABSTRACT FROM AUTHOR]

Abstract: In this paper, we introduce an efficient algorithm for mining discriminative regularities on databases with mixed and incomplete data. Unlike previous methods, our algorithm does not apply an a priori discretization on numerical features; it extracts regularities from a set of diverse decision trees, induced with a special procedure. Experimental results show that a classifier based on the regularities obtained by our algorithm attains higher classification accuracy, using fewer discriminative regularities than those obtained by previous pattern-based classifiers. Additionally, we show that our classifier is competitive with traditional and state-of-the-art classifiers. [Copyright &y& Elsevier]

Abstract: Extant multiple-cost-sensitive learning algorithms are usually designed for dealing with misclassification cost (MC) and test cost (TC) together. This paper outlines a new learning algorithm, called cost-time sensitive classification, designed for minimizing tangible costs (which includes TC and waiting cost (WC)) as well as maximizing the decrease of the intangible costs (also called MC). The proposed algorithm induces decision trees from training datasets with missing data, in which the costs are measured in different units. Firstly, a split criterion is proposed for building cost-time sensitive decision trees, aiming at possibly reducing the intangible cost. Then a hybrid test strategy, which can handle missing values in test datasets, is designed for combining the sequential test with the batch test strategy. To evaluate the efficiency of the proposed method, extensive experiments were conducted on the UCI datasets at different missing rates. The experimental results show that the proposed algorithm achieves better than the existing ones in terms of reducing the intangible costs when taking into account waiting costs. [Copyright &y& Elsevier]

Abstract: This paper reports the development of a decision tree algorithm to classify the surface soil freeze/thaw states. The algorithm uses SSM/I brightness temperatures recorded in the early morning. Three critical indices are used as classification criteria—the scattering index (SI), the 37 GHz vertical polarization brightness temperature (T 37V), and the 19 GHz polarization difference (PD19). The thresholds of these criteria were obtained from samples of frozen soil, thawed soil, desert, and snow. The algorithm is capable of distinguishing between frozen soil, thawed soil, desert, and precipitation. In-situ 4-cm deep soil temperatures on the Qinghai–Tibetan Plateau were used to validate the classification results, and the average classification accuracy was found to be 87%. Regarding the misclassified pixels, about 40% and 73% of them appeared when the surface soil temperature ranged from −0.5 °C to 0.5 °C and from −2.0 °C to 2.0 °C, respectively, which means that most misclassifications occurred near the soil freezing point. In addition, misclassifications mainly occurred from April to May and September to October, the transition periods between warm and cold seasons. A grid-to-grid Kappa analysis was also conducted to evaluate the consistency between the map of the actual number of frozen days obtained using the decision tree classification algorithm and the reference map of geocryological regionalization and classification in China. The overall classification accuracy was 91.7%, and the Kappa index was 80.5%. The boundary between the frozen and thawed soil was consistent with the southern limit of seasonally frozen ground from the reference map. The statistics show that the maximum area of frozen soil is about 6.82×106 km2 in late January, accounting for 69% of total Chinese land area. [Copyright &y& Elsevier]

Decision trees are one of the most popular data-mining techniques for knowledge discovery. Many approaches for induction of decision trees often deal with the continuous data and missing values in information systems. However, they do not perform well in real situations. This paper presents a new algorithm, decision tree construction based on the Cloud transform and Rough set theory under the characteristic relation (CR), for mining classification knowledge from a given data set. The continuous data is transformed into discrete qualitative concepts via the cloud transformation and then the attribute with the smallest weighted mean roughness under the characteristic relation is selected as the current splitting node. Experimental evaluation shows the decision trees constructed by the CR algorithm tend to have a simpler structure, much higher classification accuracy and more understandable rules than those by C5.0 in most cases. [ABSTRACT FROM AUTHOR]

Abstract: Crisp decision tree algorithms face the problem of having sharp decision boundaries which may not be found in all real life classification problems. A fuzzy decision tree algorithm Gini Index based (G-FDT) is proposed in this paper to fuzzify the decision boundary without converting the numeric attributes into fuzzy linguistic terms. Gini Index is used as split measure for choosing the most appropriate splitting attribute at each node. The performance of G-FDT algorithm is compared with Gini Index based crisp decision tree algorithm (SLIQ) using several real life datasets taken from the UCI machine learning repository. G-FDT algorithm outperforms its crisp counterpart in terms of classification accuracy. The size of the G-FDT is significantly less compared to SLIQ. [Copyright &y& Elsevier]

Most decision tree classifiers are designed to classify the data with categorical or Boolean class labels. Unfortunately, many practical classification problems concern data with class labels that are naturally organized as a hierarchical structure, such as test scores. In the hierarchy, the ranges in the upper levels are less specific but easier to predict, while the ranges in the lower levels are more specific but harder to predict. To build a decision tree from this kind of data, we must consider how to classify data so that the class label can be as specific as possible while also ensuring the highest possible accuracy of the prediction. To the best of our knowledge, no previous research has considered the induction of decision trees from data with hierarchical class labels. This paper proposes a novel classification algorithm for learning decision tree classifiers from data with hierarchical class labels. Empirical results show that the proposed method is efficient and effective in both prediction accuracy and prediction specificity. [Copyright &y& Elsevier]

Traditional decision tree algorithms face the problem of having sharp decision boundaries which are hardly found in any real-life classification problems. A fuzzy supervised learning in Quest (SLIQ) decision tree (FS-DT) algorithm is proposed in this paper. It is aimed at constructing a fuzzy decision boundary instead of a crisp decision boundary. Size of the decision tree constructed is another very important parameter in decision tree algorithms. Large and deeper decision tree results in incomprehensible induction rules. The proposed FS-DT algorithm modifies the SLIQ decision tree algorithm to construct a fuzzy binary decision tree of significantly reduced size. The performance of the FS-DT algorithm is compared with SLIQ using several real-life datasets taken from the UCI Machine Learning Repository. The FS-DT algorithm outperforms its crisp counterpart in terms of classification accuracy. FS-DT also results in more than 70% reduction in size of the decision tree compared to SLIQ. [ABSTRACT FROM AUTHOR]

This paper introduces a new method for supervised discretization based on interval distances by using a novel concept of neighborhood in the target's space. The proposed method takes into consideration the order of the class attribute, when this exists, so that it can be used with ordinal discrete classes as well as continuous classes, in the case of regression problems. The method has proved to be very efficient in terms of accuracy and faster than the most commonly supervised discretization methods used in the literature. It is illustrated through several examples, and a comparison with other standard discretization methods is performed for three public data sets by using two different learning tasks: a decision tree algorithm and SVM for regression. [ABSTRACT FROM AUTHOR]

Classification is a quite relevant task within data analysis field. This task is not a trivial task and different difficulties can arise depending on the nature of the problem. All these difficulties can become worse when the datasets are too large or when new information can arrive at any time. Incremental learning is an approach that can be used to deal with the classification task in these cases. It must alleviate, or solve, the problem of limited time and memory resources. One emergent approach uses concentration bounds to ensure that decisions are made when enough information supports them. IADEM is one of the most recent algorithms that use this approach. The aim of this paper is to improve the performance of this algorithm in different ways: simplifying the complexity of the induced models, adding the ability to deal with continuous data, improving the detection of noise, selecting new criteria for evolutionating the model, including the use of more powerful prediction techniques, etc. Besides these new properties, the new system, IADEM-2, preserves the ability to obtain a performance similar to standard learning algorithms independently of the datasets size and it can incorporate new information as the basic algorithm does: using short time per example. [ABSTRACT FROM AUTHOR]

This paper explores the potential of an artificial immune-based supervised classification algorithm for land-cover classification. This classifier is inspired by the human immune system and possesses properties similar to nonlinear classification, self/non-self identification, and negative selection. Landsat ETM+ data of an area lying in Eastern England near the town of Littleport are used to study the performance of the artificial immune-based classifier. A univariate decision tree and maximum likelihood classifier were used to compare its performance in terms of classification accuracy and computational cost. Results suggest that the artificial immune-based classifier works well in comparison with the maximum likelihood and the decision-tree classifiers in terms of classification accuracy. The computational cost using artificial immune based classifier is more than the decision tree but less than the maximum likelihood classifier. Another data set from an area in Spain is also used to compare the performance of immune based supervised classifier with maximum likelihood and decision-tree classification algorithms. Results suggest an improved performance with the immune-based classifier in terms of classification accuracy with this data set, too. The design of an artificial immune-based supervised classifier requires several user-defined parameters to be set, so this work is extended to study the effect of varying the values of six parameters on classification accuracy. Finally, a comparison with a backpropagation neural network suggests that the neural network classifier provides higher classification accuracies with both data sets, but the results are not statistically significant. [ABSTRACT FROM AUTHOR]

Inducing a classification function from a set of examples in the form of labeled instances is a standard problem in supervised machine learning. In this paper, we are concerned with ambiguous label classification (ALC), an extension of this setting in which several candidate labels may be assigned to a single example. By extending three concrete classification methods to the ALC setting (nearest neighbor classification, decision tree learning, and rule induction) and evaluating their performance on benchmark data sets, we show that appropriately designed learning algorithms can successfully exploit the information contained in ambiguously labeled examples. Our results indicate that the fundamental idea of the extended methods, namely to disambiguate the label information by means of the inductive bias underlying (heuristic) machine learning methods, works well in practice. [ABSTRACT FROM AUTHOR]

A decision tree is a classification algorithm that automatically derives a hierarchy of partition rules with respect to a target attribute of a large dataset. However, spatial autocorrelation makes conventional decision trees underperform for geographical datasets as the spatial distribution is not taken into account. The research presented in this paper introduces the concept of a spatial decision tree based on a spatial diversity coefficient that measures the spatial entropy of a geo-referenced dataset. The principle of this solution is to take into account the spatial autocorrelation phenomena in the classification process, within a notion of spatial entropy that extends the conventional notion of entropy. Such a spatial entropy-based decision tree integrates the spatial autocorrelation component and generates a classification process adapted to geographical data. A case study oriented to the classification of an agriculture dataset in China illustrates the potential of the proposed approach. [ABSTRACT FROM AUTHOR]