Search

Web spam is an illegal and immoral way to increase the ranking of web pages by deceiving search engine algorithms. Therefore, different methods have been proposed to detect and improve the quality of results. Since a web page can be viewed from two aspects of the content and the link, the number of extracting features is high. Thus, selection of features with high separating ability can be considered as a preprocessing step in order to decrease computational time and cost. In this study, a new backward elimination approach is proposed for feature selection. The main idea of this method is measuring the impact of eliminating a set of features on the performance of a classifier instead of a single feature which is similar to the sequential backward selection. This method seeks for the largest feature subset that their omission from whole set features not only reduces the efficiency of the classifier but also improves it. Implementations on WEBSPAM-UK2007 dataset with Naive Bayes classifier show that the proposed method selects fewer features in comparison with other methods and improves the performance of the classifier in the IBA index about 7%.

The Naive Credal Classifier (NCC) was the first method proposed for Imprecise Classification. It starts from the known Naive Bayes algorithm (NB), which assumes that the attributes are independent given the class variable. Despite this unrealistic assumption, NB and NCC have been successfully used in practical applications. In this work, we propose a new version of NCC, called Extreme Prior Naive Credal Classifier (EP-NCC). Unlike NCC, EP-NCC takes into consideration the lower and upper prior probabilities of the class variable in the estimation of the lower and upper conditional probabilities. We demonstrate that, with our proposed EP-NCC, the predictions are more informative than with NCC without increasing the risk of making erroneous predictions. An experimental analysis carried out in this work shows that EP-NCC significantly outperforms NCC and obtains statistically equivalent results to the algorithm proposed so far for Imprecise Classification based on decision trees, even though EP-NCC is computationally simpler. Therefore, EP-NCC is more suitable to be applied to large datasets for Imprecise Classification than the methods proposed so far in this field. This is an important issue in favor of our proposal due to the increasing amount of data in every area.

Software defect prediction plays a significant part in identifying the most defect-prone modules before software testing. Quite a number of researchers have made great efforts to improve prediction accuracy. However, the problem of insufficient historical data available for within- or cross- project still remains unresolved. Further, it is common practice to use the probability density function for a normal distribution in Naive Bayes (NB) classifier. Nevertheless, after performing a Kolmogorov–Smirnov test, we find that the 21 main software metrics are not normally distributed at the 5% significance level. Therefore, this paper proposes a new Bayes classifier, which evolves NB classifier with non-normal information diffusion function, to help solve the problem of lacking appropriate training data for new projects. We conduct three experiments on 34 data sets obtained from 10 open source projects, using only 10%, 6.67%, 5%, 3.33% and 2% of the total data for training, respectively. Four well-known classification algorithms are also included for comparison, namely Logistic Regression, Naive Bayes, Random Tree and Support Vector Machine. All experimental results demonstrate the efficiency and practicability of the new classifier.

Knowledge based systems have accomplished remarkable achievements in assisting evidence based decision making for complex problems. However, machine learning-driven, intelligent systems of today are dependent on the underlying knowledge model, which is acquired from domain experts, or the available datasets in a structured or unstructured format. Most of the existing literature utilized a single modal, while very few have combined multi-modalities (mainly two) for knowledge acquisition. In order to achieve a strong Artificial Intelligence, multi-domain and multi-modal knowledge acquisition, and consolidation is required. This paper presents the research work, driving the realization of such a comprehensive framework, in the field of healthcare. Using area specific, state-of-the-art machine learning techniques, we first extract knowledge from structured and unstructured data, which is consolidated with expert knowledge and managed through ripple down rules. Our presented technique shows an accuracy of 92.05%, which is much higher than single modal deep learning at 78.20%, naive bayes at 69.70%, logistic regression at 61.20%, expert driven knowledge at 86.02%, and naive knowledge combination at 70.86%. Thus, through the application of our proposed technique, we provide the foundations for an accurate and evolvable knowledge-base, that can greatly enhance decision making in the healthcare domain.

Activity recognition is one of the emerging trends in the domain of mining ubiquitous environments. It assumes that we can recognize the current action undertaken by the monitored subject on the basis of outputs of a set of associated sensors. Often different combinations of smart devices are being used, thus creating an Internet of Things. Such data will arrive continuously during the operation time of sensors and require an online processing in order to keep a real-time track of the current activity being undertaken. This forms a natural data stream problem with the potential presence of changes in the arriving data. Therefore, we require an efficient online machine learning system that can offer high recognition rates and adapt to drifts and shifts in the stream. In this paper we propose an efficient and lightweight adaptive ensemble learning system for real-time activity recognition. We use a weighted modification of Naive Bayes classifier that can swiftly adapt itself to the current state of the stream without a need for an external concept drift detector. To tackle the multi-class nature of activity recognition problem we propose to use an one-vs-one decomposition to form a committee of simpler and diverse learners. We introduce a novel weighted combination for one-vs-one decomposition that can adapt itself over time. Additionally, to limit the cost of supervision we propose to enhance our classification system with active learning paradigm to select only the most important objects for labeling and work under constrained budget. Experiments carried out on six data streams gathered from ubiquitous environments show that the proposed active and adaptive ensemble offer excellent classification accuracy with low requirement for access to true class labels.

Email authorship analysis is a challenging task involving the detection of an author’s style to help determine their identity. Emails represent a widespread application of big data, and email authorship analysis is widely performed in the forensic linguistics field. However, the high-dimensional feature space encountered in authorship analysis affects the classification performance. Moreover, the Arabic language is highly inflected and involves certain unique characteristics, which pose critical challenges in identifying the context. Therefore, the selection of prominent features is a critical step in realizing authorship analysis. Swarm intelligence (SI) algorithms are widely adopted to address such feature selection problems. In this study, an efficient hybrid feature selection algorithm based on binary particle swarm optimization (BPSO) and chi-square BPSO (CS-BPSO) was developed to enhance the performance of Arabic email authorship analysis. Static and dynamic features were considered. Experiments were conducted on Arabic email messages collected from a sample population to test the algorithm performance using three popular classifiers: support vector machine (SVM), K-nearest neighbour (KNN), and naive Bayes (NB) classifiers. Different metrics, specifically, the accuracy, precision, recall, and f1-score, were considered as performance measures. The results showed that the CS-BPSO method achieves impressive results using dynamic features. The findings were quite satisfactory in terms of solving multiple types of difficulties, e.g., imbalanced dataset, small dataset, and short text.

In this contribution we propose a hybrid recommendation framework based on classification algorithms such as Random Forests and Naive Bayes, which are fed with several heterogeneous groups of features. We split our features into two classes: classic features, as popularity-based, collaborative and content-based ones, and extended features gathered from the Linked Open Data (LOD) cloud, as basic ones (i.e. genre of a movie or the writer of a book) and graph-based features calculated on the ground of the different topological characteristics of the tripartite representation connecting users, items and properties in the LOD cloud. In the experimental session we evaluate the effectiveness of our framework on varying of different groups of features, and results show that both LOD-based and graph-based features positively affect the overall performance of the algorithm, especially in highly sparse recommendation scenarios. Our approach also outperforms several state-of-the-art recommendation techniques, thus confirming the insights behind this research. This extended abstract summarizes the content of the journal paper published on Knowledge-based Systems.

A recent trend in healthcare is to motivate patients to self-manage their health conditions in home-based settings. Self-management programs guide and motivate patients to achieve self-efficacy in the self-management of their disease through a regime of educational and behavioural modification strategies. To improve self-management programs effectiveness and efficacy, we must consider Ambient Assisted Living (AAL) technologies (smart environments, activity recognition, aid acts planning), since they alleviate issues related to unreliable self-reported data by monitoring self-management activities. To improve self-management programs in smart environments, it is necessary to recognize the occupant behaviour from observed data. Observed data/attributes generated from various sources (sensors, questionnaires, low-level activity recognition) are certain to uncertain (imprecise, incomplete, missing), where several values are plausible instead of only one. Thus, activity recognition must consider heterogeneous observations (sources’ types) and uncertainty in the activity recognition inputs (observations). To address this challenge, we propose an activity recognition approach based on possibilistic network classifiers with uncertain observations. We believe that this is the first work to consider possibilistic network classifiers for the recognition of activities in smart environments using uncertain observations. We have validated the approach on 780 synthetic scenarios illustrating behaviours related to medication adherence. The activity classifiers, based on knowledge and beliefs about the activities related to medication adherence, can correctly recognize 79% of an activity current state, which is comparable with approaches based on data-driven naive Bayesian classifiers. Furthermore, the classification performance only decreases when we have highly partial to complete ignorance about the observations values. Hence, the validations results show the interest of activity recognition based on possibilistic network classifiers for handling uncertain observations.

Back-propagation neural network (BPNN).Variable Adaptive Momentum.Supervised Learning Algorithms.Deep Learning. In this paper, we propose a novel machine learning classifier by deriving a new adaptive momentum back-propagation (BP) artificial neural networks algorithm. The proposed algorithm is a modified version of the BP algorithm to improve its convergence behavior in both sides, accelerate the convergence process for accessing the optimum steady-state and minimizing the error misadjustment to improve the recognized patterns superiorly. This algorithm is controlled by the learning rate parameter which is dependent on the eigenvalues of the autocorrelation matrix of the input. It provides low error performance for the weights update. To discuss the performance measures of this proposed algorithm and the other supervised learning algorithms such as k-nearest neighbours (k-NN), Naive Bayes (NB), linear discriminant analysis (LDA), support vector machines (SVM), BP, and BP with adaptive momentum (PBPAM) have been compared in term of speed of convergence, Sum of Squared Error (SSE), and accuracy by implementing benchmark problem - XOR and seven datasets from UCI repository.

Naive Bayes (NB) classifier ease of use along with its remarkable performance has led many researchers to extend the scope of its applications to real-world domains by relaxing the conditional independence assumption of features given the information about the class variable. However, fulfilling this objective, most of the generalizations, cut their own way through compromising the model’s simplicity, make more complex classifiers with a substantial deviation from the original one. Multi Independent Latent Component Naive Bayes Classifier (MILC-NB) leverages a set of latent variables to preserve the overall structure of naive Bayes classifier while rectifying its major restriction. Each latent variable is responsible for keeping a subset of conditionally dependent features d-connected within a component, and the set of features is divided into non-overlapping partitions across different components accordingly. We prove that components are conditionally independent given the information about the class variable which allows us to devise novel mathematical methods with a substantial reduction in the complexities of classification and learning. Experiments on 34 datasets obtained from the OpenML repository indicate that MILC-NB outperforms state-of-the-art classifiers in terms of area under the ROC curve (AUC) and classification accuracy (ACC).

Bayesian networks are powerful tools for knowledge representation and inference under conditions of uncertainty. However, learning an optimal Bayesian network classifier (BNC) is an NP-hard problem since its topology complexity increases exponentially with the number of attributes. Researchers proposed to apply information-theoretic criteria to measure conditional dependence, and independence assumptions are introduced implicitly or explicitly to simplify the network topology of BNC. In this paper, we clarify the mapping relationship between conditional mutual information and local topology, and then illustrate that informational independence does not correspond to probabilistic independence, the criterion of probabilistic independence does not necessarily hold for the independence topology. A novel framework of semi-naive Bayesian operation, called Hierarchical Independence Thresholding (HIT), is presented to efficiently identify informational conditional independence and probabilistic conditional independence by applying an adaptive thresholding method, redundant edges will be filtered out and the learned topology will fit the data better. Extensive experimental evaluation on 58 publicly available datasets reveals that when HIT is applied to BNCs (such as tree augmented Naive Bayes or k-dependence Bayesian classifier), the final BNCs achieve competitive classification performance compared to state-of-the-art learners such as Random Forest and Logistic regression.

Feature subset selection (FSS) has been an active area of research in machine learning. A number of techniques have been developed for selecting an optimal or sub-optimal subset of features, because it is a major factor to determine the performance of a machine-learning technique. In this paper, we propose and develop a novel optimization technique, namely, a binary coordinate ascent (BCA) algorithm that is an iterative deterministic local optimization that can be coupled with wrapper or filter FSS. The algorithm searches throughout the space of binary coded input variables by iteratively optimizing the objective function in each dimension at a time. We investigated our BCA approach in wrapper-based FSS under area under the receiver-operating-characteristic (ROC) curve (AUC) criterion for the best subset of features in classification. We evaluated our BCA-based FSS in optimization of features for support vector machine, multilayer perceptron, and Naive Bayes classifiers with 12 datasets. Our experimental datasets are distinct in terms of the number of attributes (ranging from 18 to 11,340), and the number of classes (binary or multi-class classification). The efficiency in terms of the number of subset evaluations was improved substantially (by factors of 5-37) compared with two popular FSS meta-heuristics, i.e., sequential forward selection (SFS) and sequential floating forward selection (SFFS), while the classification performance for unseen data was maintained.

This paper works on feature weighting approaches for naive Bayes text classifiers. Almost all existing feature weighting approaches for naive Bayes text classifiers have some defects: limited improvement to classification performance of naive Bayes text classifiers or sacrificing the simplicity and execution time of the final models. In fact, feature weighting is not new for machine learning community, and many researchers have made fruitful efforts in the field of feature weighting. This paper reviews some simple and efficient feature weighting approaches designed for standard naive Bayes classifiers, and adapts them for naive Bayes text classifiers. As a result, this paper proposes two adaptive feature weighting approaches for naive Bayes text classifiers. Experimental results based on benchmark and real-world data show that, compared to their competitors, our feature weighting approaches show higher classification accuracy, yet at the same time maintain the simplicity and lower execution time of the final models.

We propose a novel approach to the problem of the classification with test costs understood as costs of obtaining attribute values of classified examples. Many existing approaches construct classifiers in order to control the tradeoff between test costs and the prediction accuracy (or misclassification costs). The aim of the proposed method is to reduce test costs while maintaining of the prediction accuracy of a classifier. We assume that attribute values are represented at different levels of abstraction and model domain background knowledge. Our approach sequentially explores these levels during classification – in each iteration it selects and conducts a test that precises the representation of a classified example (i.e., acquires an attribute value), invokes a naive Bayes classifier for this new representation and checks the classifier’s outcome to decide whether this iterative process can be stopped. The selection of the test in each iteration takes into account the possible improvement of the prediction accuracy and the cost of this test. We show that the prediction accuracy obtained for classified examples represented precisely (i.e., when all the tests have been conducted and all specific attribute values have been acquired) can be achieved for a much smaller number of tests (i.e., when not all specific attribute values have been acquired). Moreover, we show that without levels of abstraction and with uniform test costs our method can be used for selecting features and it is competitive to popular feature selection schemes: filter and wrapper.

Use of software product metrics in defect prediction studies highlights the utility of these metrics. Public availability of software defect data based on the product metrics has resulted in the development of defect prediction models. These models experience a limitation in learning Defect-prone (D) modules because the available datasets are imbalanced. Most of the datasets are dominated by Not Defect-prone (ND) modules as compared to D modules. This affects the ability of classification models to learn the D modules more accurately. This paper presents an association mining based approach that allows the defect prediction models to learn D modules in imbalanced datasets. The proposed algorithm preprocesses data by setting specific metric values as missing and improves the prediction of D modules. The proposed algorithm has been evaluated using 5 public datasets. A Naive Bayes (NB) classifier has been developed before and after the proposed preprocessing. It has been shown that Recall of the classifier after the proposed preprocessing has improved. Stability of the approach has been tested by experimenting the algorithm with different number of bins. The results show that the algorithm has resulted in up to 40% performance gain.

Bayesian network classifiers (BNCs) are powerful tools in knowledge representation and inference under conditions of uncertainty. In contrast to eager learning, lazy learning seeks to improve the classification accuracy of BNCs by forming a decision theory that is especially tailored for the testing instance, whereas it has received less attention due to the high computational cost at classification time. This study introduces the conditionally independently and identically distributed (c.i.i.d.) assumption to BNCs by assuming that all instances of the same class are conditionally independent of each other and stem from the same probability distribution. Based on this premise, we propose a novel lazy BNC, semi-lazy Bayesian network classifier (SLB), which transforms each unlabeled testing instance to a series of complete instances with discriminative supposed class labels, and then builds class-specific local BNCs for each of them. Our experimental comparison on 25 UCI datasets shows that SLB has modest training time overheads and less classification time overheads. The Friedman and Nemenyi tests show that SLB has significant zero–one loss and bias advantages over some state-of-the-art BNCs, such as selective k -dependence Bayesian classifier, k -nearest neighbor, lazy Bayesian rule and average n -dependence estimators with lazy subsumption resolution.

Naive Bayes is one of the most popular data mining algorithms. Its efficiency comes from the assumption of attribute independence, although this might be violated in many real-world data sets. Many efforts have been done to mitigate the assumption, among which attribute selection is an important approach. However, conventional efforts to perform attribute selection in naive Bayes suffer from heavy computational overhead. This paper proposes an efficient selective naive Bayes algorithm, which adopts only some of the attributes to construct selective naive Bayes models. These models are built in such a way that each one is a trivial extension of another. The most predictive selective naive Bayes model can be selected by the measures of incremental leave-one-out cross validation. As a result, attributes can be selected by efficient model selection. Empirical results demonstrate that the selective naive Bayes shows superior classification accuracy, yet at the same time maintains the simplicity and efficiency.

With the continued development of social networks, the spreading of information has become faster than ever. Consequently, this has resulted in a problem with the reliability of the information, where any user can publish whatever he/she wants. Automated systems capable of detecting fake contents with similar striking speed as the information being disseminated are urgently required. Detecting rumors in Arabic language social networks has lagged behind the work on other languages, particularly in English. In this paper, we address the problem of detecting rumors in Arabic tweets. We used a set of features extracted from the user and the content. These features were analyzed to determine their significance. Semi-supervised expectation–maximization (E–M) was used to train the proposed system with topics of newsworthy tweets. A comparison with supervised Gaussian Naive Bayes (NB) showed that our semi-supervised system, using a small base of labeled data, outperforms Gaussian NB achieving an accuracy of 78.6%. The performance of the unsupervised E–M depends on the initial values, and we achieved an F 1 score of 80% in one of our experiments.

Propose a new weighting method in the context of naive Bayes classification learning.Assign different weights for each feature value.A gradient approach for automatically calculating the weights of each feature value.Its performance is compared with that of other state-of-the-art methods.Experiments show the method could improve the performance of naive Bayes. Feature weighting has been an important topic in classification learning algorithms. In this paper, we propose a new paradigm of assigning weights in classification learning, called value weighting method. While the current weighting methods assign a weight to each feature, we assign a different weight to the values of each feature. The proposed method is implemented in the context of naive Bayesian learning, and optimal weights of feature values are calculated using a gradient approach. The performance of naive Bayes learning with value weighting method is compared with that of other state-of-the-art methods for a number of datasets. The experimental results show that the value weighting method could improve the performance of naive Bayes significantly.

Automated identification of normal and diabetic subjects using HRV signals.Features are extracted using DWT method.Ranked features are subjected to various classifiers.Proposed method classifies diabetes and normal subjects with 92.02% accuracy. Diabetes Mellitus (DM), a chronic lifelong condition, is characterized by increased blood sugar levels. As there is no cure for DM, the major focus lies on controlling the disease. Therefore, DM diagnosis and treatment is of great importance. The most common complications of DM include retinopathy, neuropathy, nephropathy and cardiomyopathy. Diabetes causes cardiovascular autonomic neuropathy that affects the Heart Rate Variability (HRV). Hence, in the absence of other causes, the HRV analysis can be used to diagnose diabetes. The present work aims at developing an automated system for classification of normal and diabetes classes by using the heart rate (HR) information extracted from the Electrocardiogram (ECG) signals. The spectral analysis of HRV recognizes patients with autonomic diabetic neuropathy, and gives an earlier diagnosis of impairment of the Autonomic Nervous System (ANS). Significant correlations with the impaired ANS are observed of the HRV spectral indices obtained by using the Discrete Wavelet Transform (DWT) method. Herein, in order to diagnose and detect DM automatically, we have performed DWT decomposition up to 5 levels, and extracted the energy, sample entropy, approximation entropy, kurtosis and skewness features at various detailed coefficient levels of the DWT. We have extracted relative wavelet energy and entropy features up to the 5th level of DWT coefficients extracted from HR signals. These features are ranked by using various ranking methods, namely, Bhattacharyya space algorithm, t-test, Wilcoxon test, Receiver Operating Curve (ROC) and entropy.The ranked features are then fed into different classifiers, that include Decision Tree (DT), K-Nearest Neighbor (KNN), Naive Bayes (NBC) and Support Vector Machine (SVM). Our results have shown maximum diagnostic differentiation performance by using a minimum number of features. With our system, we have obtained an average accuracy of 92.02%, sensitivity of 92.59% and specificity of 91.46%, by using DT classifier with ten-fold cross validation.

Recommender systems (RSs) have proven to be valuable means for online users to cope with the information overload and have become one of the most powerful and popular tools in electronic commerce. RSs are software tools providing suggestions for items of interest to users; hence, they typically apply techniques and methodologies from Data Mining. The most frequently used technique is the classification as it matches the aims of RSs that basically classify items based on user's preferences. The main contribution of this paper is in the area of applying classification techniques to enhance the performance of RSs. In this paper, an Intelligent Adaptive Vertical Recommendation (IAVR) system will be introduced. IAVR recommends text documents related to a specific domain. Basically, the paper concentrates on the first phase of IAVR, which contains two modules; the first is a distiller, while the second is a multi-class classifier. The proposed distiller is employed as a binary classifier that elects documents related to the domain of interest. It is built upon a novel neuro-fuzzy system as well as a modified K Nearest Neighbors (KNN) classifier. On the other hand, the proposed multi-class classifier merges a new instance of Naive Bayes (NB) classifier, that depends on a proposed learning technique called "accumulative learning", with association rules. Experimental results have proven the effectiveness of the proposed classifiers, which accordingly promote the overall system's recommendation accuracy.

This paper proposes an approach called TESC (TExt classification using Semi-supervised Clustering) to improve text classification. The basic idea is to regard one category of texts from one or more than one components. Thus, we use clustering to identify the components in text collection. In clustering process, TESC makes use of labeled texts to capture silhouettes of text clusters and unlabeled texts to adapt its centroids. The category of each text cluster is labeled by the label of texts in it. When a new unlabeled text is incoming, we measure its similarity with the text clusters and give its label with that of the nearest text clusters. Experiments on Reuters-21578 and TanCorp V1.0 text collection demonstrate that, in text classification, TESC outperforms Support Vector Machines (SVMs) and back propagation neural network (BPNN), and produces comparable performance to naive Bayes with EM (Expectation Maximization) however with lower computation complexity.

Feature selection is often used in email classification to reduce the dimensionality of the feature space. In this study, a new document frequency and term frequency combined feature selection method (DTFS) is proposed to improve the performance of email classification. Firstly, an existing optimal document frequency based feature selection method (ODFFS) and a predetermined threshold are applied to select the most discriminative features. Secondly, an existing optimal term frequency based feature selection (OTFFS) method and another predetermined threshold are applied to select more discriminative features. Finally, ODFFS and OTFFS are combined to select the remaining features. In order to improve the convergence rate of parameter optimization, a metaheuristic method, namely global best harmony oriented harmony search (GBHS), is proposed to search these optimal predetermined thresholds. Experiments with fuzzy Support Vector Machine (FSVM) and Naive Bayesian (NB) classifiers are applied on six corpuses: PU2, CSDMC2010, PU3, Lingspam, Enron-spam and Trec2007. Experimental results show that, DTFS outperforms other methods: such as Chi-squre, comprehensively measure feature selection, t-test based feature selection, term frequency based information gain, two-step based hybrid feature selection method and improved term frequency inverse document frequency method on six corpuses.

Emotional illiteracy exists in current e-learning environment, which will decay learning enthusiasm and productivity, and now gets more attentions in recent researches. Inspired by affective computing and active listening strategy, in this paper, a research and application framework of recognizing emotion based on textual interaction is presented first. Second, an emotion category model for e-learners is defined. Third, many Chinese metaphors are abstracted from the corpus according to the sentence semantics and syntax. Fourth, as the strategy of active learning, topic detection is used to detect the first turn in dialogs and recognize the type of emotion in the turn, which is different from the traditional emotion recognition approaches that try to classify every turn into an emotion category. Fifth, compared with Support Vector Machines (SVM), Naive Bayes, LogitBoost, Bagging, MultiClass Classifier, RBFnetwork, J48 algorithms and their corresponding cost-sensitive approaches, Random Forest and its corresponding cost-sensitive approaches achieve better results in our initial experiment of classifying the e-learners' emotions. Finally, a case-based reasoning for emotion regulation instance recommendation is proposed to guide the listener to regulate the negative emotion of a speaker, in which a weighted sum method of Chinese sentence similarity computation is adopted. The experimental result shows that the ratio of effective cases is 68%.

This paper deals with the problem of wrapper feature subset selection (FSS) in classification-oriented datasets with a (very) large number of attributes. In high-dimensional datasets with thousands of variables, wrapper FSS becomes a laborious computational process because of the amount of CPU time it requires. In this paper we study how under certain circumstances the wrapper FSS process can be speeded up by embedding the classifier into the wrapper algorithm, instead of dealing with it as a black-box. Our proposal is based on the combination of the NB classifier (which is known to be largely beneficial for FSS) with incremental wrapper FSS algorithms. The merit of this approach is analyzed both theoretically and experimentally, and the results show an impressive speed-up for the embedded FSS process.

Electrocardiogram (ECG) is widely used as a diagnostic tool to identify atrial tachyarrhythmias such as atrial fibrillation. The ECG signal is a P-QRS-T wave representing the cardiac function. The minute variations in the durations and amplitude of these waves cannot be easily deciphered by the naked eye. Hence, there is a need for computer aided diagnosis (CAD) of cardiac healthcare. The current paper presents a methodology for ECG based pattern analysis of normal sinus rhythm and atrial fibrillation (AF) beats. The denoised and registered ECG beats were subjected to independent component analysis (ICA) for data reduction. The weights of ICA were used as features for classification using Naive Bayes and Gaussian mixture model (GMM) classifiers. The performance and the upper bound on probability of error in classification were analyzed using Chernoff and Bhattacharyya bounds. The Naive Bayes classifier provided an average sensitivity of 99.32%, specificity of 99.33% and accuracy of 99.33%, while the GMM provided an average sensitivity of 100%, specificity of 99% and accuracy of 99.42%. The probability of error during classification was less for GMM compared to Naive Bayes classifier (NBC) as GMM provided higher performance than the NBC.

Rapid growth of digital information requires automated handling and organization of documents. The two main stages in automated document categorization are (i) term reduction and (ii) classification. In this paper, we present a novel two-stage term reduction strategy based on Information Gain (IG) theory and Geometric Particle Swarm Optimization (GPSO) search. We evaluate performance of the proposed term reduction approach with use of a new classifier, fuzzy unordered rule induction algorithm (FURIA) to categorize multi-label texts. In order to evaluate the performance of FURIA quantitatively, we compared it against two widely used algorithms, Naive Bayes and Support Vector Machine (SVM). Text Categorization (TC) performance of the proposed term reduction strategy is validated with use of Reuters-21578 and OHSUMED text collection datasets. The experimental results show that performance of the proposed term reduction method is efficient for document organization tasks.

Training methods for machine learning are often characterized as being generative or discriminative. We present a new co-training style algorithm which employs a generative classifier (Naive Bayes) and a discriminative classifier (Support Vector Machine) as base classifiers, to take advantage of both methods. Furthermore, we introduce a pair of weight parameters to balance the impact of labeled and pseudo-labeled data, and define a hybrid objective function to tune their values during co-training. The final prediction is given by the combination of base classifiers, and we define a pseudo-validation set to regulate their weight. Additionally, we present a strategy of pseudo-labeled data selecting to deal with the class imbalance problem. Experimental results on six datasets show that our method performs much better in practice, especially when the amount of labeled data is small.

Web proxy caching is one of the most successful solutions for improving the performance of Web-based systems. In Web proxy caching, the popular Web objects that are likely to be revisited in the near future are stored on the proxy server, which plays the key roles between users and Web sites in reducing the response time of user requests and saving the network bandwidth. However, the difficulty in determining the ideal Web objects that will be re-visited in the future is still a problem faced by existing conventional Web proxy caching techniques. In this paper, a Naive Bayes (NB) classifier is used to enhance the performance of conventional Web proxy caching approaches such as Least-Recently-Used (LRU) and Greedy-Dual-Size (GDS). NB is intelligently incorporated with conventional Web proxy caching techniques to form intelligent and effective caching approaches known as NB-GDS, NB-LRU and NB-DA. Experimental results have revealed that the proposed NB-GDS, NB-LRU and NB-DA significantly improve the performances of the existing Web proxy caching approaches across several proxy datasets.

Numerous algorithms have been proposed to improve Naive Bayes (NB) by weakening its conditional attribute independence assumption, among which Tree Augmented Naive Bayes (TAN) has demonstrated remarkable classification performance in terms of classification accuracy or error rate, while maintaining efficiency and simplicity. In many real-world applications, however, classification accuracy or error rate is not enough. For example, in direct marketing, we often need to deploy different promotion strategies to customers with different likelihood (class probability) of buying some products. Thus, accurate class probability estimation is often required to make optimal decisions. In this paper, we investigate the class probability estimation performance of TAN in terms of conditional log likelihood (CLL) and present a new algorithm to improve its class probability estimation performance by the spanning TAN classifiers. We call our improved algorithm Averaged Tree Augmented Naive Bayes (ATAN). The experimental results on a large number of UCI datasets published on the main web site of Weka platform show that ATAN significantly outperforms TAN and all the other algorithms used to compare in terms of CLL.

Data uncertainty can be caused by numerous factors such as measurement precision limitations, network latency, data staleness and sampling errors. When mining knowledge from emerging applications such as sensor networks or location based services, data uncertainty should be handled cautiously to avoid erroneous results. In this paper, we apply probabilistic and statistical theory on uncertain data and develop a novel method to calculate conditional probabilities of Bayes theorem. Based on that, we propose a novel Bayesian classification algorithm for uncertain data. The experimental results show that the proposed method classifies uncertain data with potentially higher accuracies than the Naive Bayesian approach. It also has a more stable performance than the existing extended Naive Bayesian method.

Content-based spam filtering is a binary text categorization problem. To improve the performance of the spam filtering, feature selection, as an important and indispensable means of text categorization, also plays an important role in spam filtering. We proposed a new method, named Bi-Test, which utilizes binomial hypothesis testing to estimate whether the probability of a feature belonging to the spam satisfies a given threshold or not. We have evaluated Bi-Test on six benchmark spam corpora (pu1, pu2, pu3, pua, lingspam and CSDMC2010), using two classification algorithms, Naive Bayes (NB) and Support Vector Machines (SVM), and compared it with four famous feature selection algorithms (information gain, @g^2-statistic, improved Gini index and Poisson distribution). The experiments show that Bi-Test performs significantly better than @g^2-statistic and Poisson distribution, and produces comparable performance with information gain and improved Gini index in terms of F1 measure when Naive Bayes classifier is used; it achieves comparable performance with the other methods when SVM classifier is used. Moreover, Bi-Test executes faster than the other four algorithms.

A novel algorithm named NB^+ which is an extended version of the traditional Naive Bayesian algorithm has been presented in this paper. An exception occurs when there is an equal probability for the class label value in the Naive Bayesian algorithm. The approach aims to suggest a solution with the help of a partial matching method. Consequently, the classification accuracy has drastically improved. Experimental evaluation has been done on various databases to show that NB^+ algorithm outperforms the traditional Naive Bayesian algorithm.

Many distance-related algorithms depend upon a good distance metric to be successful. The Value Difference Metric, simply VDM, is proposed to find reasonable distance metric between each pair of instances with nominal attribute values only. In VDM, all of the attributes are assumed to be fully independent, and the difference between two values of an attribute is only considered to be closer if they have more similar correlation with the output classes. It is obvious that the attribute independence assumption in VDM is rarely true in reality, which would harm its performance in the applications with complex attribute dependencies. In this paper, we single out an improved Value Difference Metric by relaxing its unrealistic attribute independence assumption. We call it One Dependence Value Difference Metric, simply ODVDM. In ODVDM, the structure learning algorithms for Bayesian network classifiers, such as tree augmented naive Bayes, are used to find the dependence relationships among the attributes. Our experimental results validate its effectiveness in terms of classification accuracy.

The ability to predict a student's performance could be useful in a great number of different ways associated with university-level distance learning. Students' marks in a few written assignments can constitute the training set for a supervised machine learning algorithm. Along with the explosive increase of data and information, incremental learning ability has become more and more important for machine learning approaches. The online algorithms try to forget irrelevant information instead of synthesizing all available information (as opposed to classic batch learning algorithms). Nowadays, combining classifiers is proposed as a new direction for the improvement of the classification accuracy. However, most ensemble algorithms operate in batch mode. Therefore a better proposal is an online ensemble of classifiers that combines an incremental version of Naive Bayes, the 1-NN and the WINNOW algorithms using the voting methodology. Among other significant conclusions it was found that the proposed algorithm is the most appropriate to be used for the construction of a software support tool.

Each type of classifier has its own advantages as well as certain shortcomings. In this paper, we take the advantages of the associative classifier and the Naive Bayes Classifier to make up the shortcomings of each other, thus improving the accuracy of text classification. We will classify the training cases with the Naive Bayes Classifier and set different confidence threshold values for different class association rules (CARs) to different classes by the obtained classification accuracy rate of the Naive Bayes Classifier to the classes. Since the accuracy rates of all selected CARs of the class are higher than that obtained by the Naive Bayes Classifier, we could further optimize the classification result through these selected CARs. Moreover, for those unclassified cases, we will classify them with the Naive Bayes Classifier. The experimental results show that combining the advantages of these two different classifiers better classification result can be obtained than with a single classifier.

This paper introduces group method of data handing (GMDH) theory to Bayesian classification, and proposes GMBC algorithm for structure identification of Bayesian classifiers. The algorithm combines two structure identification ideas of search & scoring and dependence analysis, and is able to accomplish the process of adaptive structure identification. We experimentally test two versions of Bayesian classifiers (GMBC-BDe and GMBC-BIC) over 25 data sets. The results show that, the structure identification of the two Bayesian classifiers especially GMBC-BDe is very effective. And when the data sets contain lots of noise, the superiority of Bayesian classifiers learned by GMBC is more obvious. Finally, giving a classification domain without any prior information about the noise, we recommend adopting GMBC-BDe rather than GMBC-BIC.

This paper presents a novel algorithm named ID6NB for extending decision tree induced by Quinlan's non-incremental ID3 algorithm. The presented approach is aimed at suggesting the solutions for few unhandled exceptions of the Decision tree induction algorithms such as (i) the situation in which the majority voting makes incorrect decision (generating two different types of rules for same data), and (ii) in case of dimensionality reduction by decision tree induction algorithms, the determination of appropriate attribute at a node where two or more attributes have equal highest information gain. Exception due to majority voting is handled with the help of Naive Bayes algorithm and also novel solutions are given for dimensionality reduction. As a result, the classification accuracy has drastically improved. An extensive experimental evaluation on a number of real and synthetic databases shows that ID6NB is a state-of-the-art classification algorithm that outperforms well than other methods of decision tree learning.

The growth of email users has resulted in the dramatic increasing of the spam emails during the past few years. In this paper, four machine learning algorithms, which are Naive Bayesian (NB), neural network (NN), support vector machine (SVM) and relevance vector machine (RVM), are proposed for spam classification. An empirical evaluation for them on the benchmark spam filtering corpora is presented. The experiments are performed based on different training set size and extracted feature size. Experimental results show that NN classifier is unsuitable for using alone as a spam rejection tool. Generally, the performances of SVM and RVM classifiers are obviously superior to NB classifier. Compared with SVM, RVM is shown to provide the similar classification result with less relevance vectors and much faster testing time. Despite the slower learning procedure, RVM is more suitable than SVM for spam classification in terms of the applications that require low complexity.

In many applications, an enormous amount of unlabeled data is available with little cost. Therefore, it is natural to ask whether we can take advantage of these unlabeled data in classification learning. In this paper, we analyzed the role of unlabeled data in the context of naive Bayesian learning. Experimental results show that including unlabeled data as part of training data can significantly improve the performance of classification accuracy.

The naive Bayes classifier continues to be a popular learning algorithm for data mining applications due to its simplicity and linear run-time. Many enhancements to the basic algorithm have been proposed to help mitigate its primary weakness - the assumption that attributes are independent given the class. All of them improve the performance of naive Bayes at the expense (to a greater or lesser degree) of execution time and/or simplicity of the final model. In this paper we present a simple filter method for setting attribute weights for use with naive Bayes. Experimental results show that naive Bayes with attribute weights rarely degrades the quality of the model compared to standard naive Bayes and, in many cases, improves it dramatically. The main advantages of this method compared to other approaches for improving naive Bayes is its run-time complexity and the fact that it maintains the simplicity of the final model.

Bayesian networks (BNs) provide a means for representing, displaying, and making available in a usable form the knowledge of experts in a given field. In this paper, we look at the performance of an expert constructed BN compared with other machine learning (ML) techniques for predicting the outcome (win, lose, or draw) of matches played by Tottenham Hotspur Football Club. The period under study was 1995-1997 - the expert BN was constructed at the start of that period, based almost exclusively on subjective judgement. Our objective was to determine retrospectively the comparative accuracy of the expert BN compared to some alternative ML models that were built using data from the two-year period. The additional ML techniques considered were: MC4, a decision tree learner; Naive Bayesian learner; Data Driven Bayesian (a BN whose structure and node probability tables are learnt entirely from data); and a K-nearest neighbour learner. The results show that the expert BN is generally superior to the other techniques for this domain in predictive accuracy. The results are even more impressive for BNs given that, in a number of key respects, the study assumptions place them at a disadvantage. For example, we have assumed that the BN prediction is 'incorrect' if a BN predicts more than one outcome as equally most likely (whereas, in fact, such a prediction would prove valuable to somebody who could place an 'each way' bet on the outcome). Although the expert BN has now long been irrelevant (since it contains variables relating to key players who have retired or left the club) the results here tend to confirm the excellent potential of BNs when they are built by a reliable domain expert. The ability to provide accurate predictions without requiring much learning data are an obvious bonus in any domain where data are scarce. Moreover, the BN was relatively simple for the expert to build and its structure could be used again in this and similar types of problems.

Decision tree is useful to obtain a proper set of rules from a large amount of instances. However, it has difficulty in obtaining the relationship between continuous-valued data points. We propose in this paper a novel algorithm, Self-adaptive NBTree, which induces a hybrid of decision tree and Naive Bayes. The Bayes measure, which is used to construct decision tree, can directly handle continuous attributes and automatically find the most appropriate boundaries for discretization and the number of intervals. The Naive Bayes node helps to solve overgeneralization and overspecialization problems which are often seen in decision tree. Experimental results on a variety of natural domains indicate that Self-adaptive NBTree has clear advantages with respect to the generalization ability.

This paper proposed a decision tree based classification method to detect e-mails that contain terrorism information. The proposed classification method is an incremental and user-feedback based extension of a decision tree induction algorithm named Ad Infinitum. We show that Ad Infinitum algorithm is a good choice for threatening e-mail detection as it runs fast on large and high dimensional databases, is easy to tune and is highly accurate, outperforming popular algorithms such as Decision Trees, Support Vector Machines and Naive Bayes. In particular, we are interested in detecting fraudulent and possibly criminal activities from such e-mails.