Search

Recognizing and classifying human actions in video clips is a powerful technology for surveillance applications. However, most of the state-of-the-art approaches for this task lack the possibility of being implemented in real-time applications without causing a critical delay. Thus, we propose a fast method to human action recognition for visual question answering, based on a novel feature extractor developed by us, 2D pose estimation, and machine learning techniques. Our extractor obtains features based on distances, angles, and positions of detected anatomical keypoints. We used the UCF101 dataset that corresponds to 13,320 videos with realistic human actions, collected from YouTube, for our work evaluation. The proposed feature extractor, combined with the Complement Naive Bayes classifier, reached a mean Average Precision ( m A P ) of 62.03 % and processed 5.26 frames per second, proving to be faster than most methods while achieving a decent m A P .

Epilepsy is generally considered as a collection of neurological disorders manifested by epileptic seizures. It is often confirmed with an electroencephalogram (EEG). EEG signals are non stationary, nonlinear and non Gaussian. In this paper, in order to tackle this problem, we have used three different methods for feature extraction namely wavelet based entropy (approximation entropy, sample entropy, permutation entropy), nonlinear features (Hurst exponent, Higuchi Fractal Dimension) and higher order spectra (mean, normalized entropy-1 and normalized entropy-2). Further multiclass classification using indirect approach with One vs One method is employed using heterogenous ensemble learning approach. Entropy features are used for classifying normal and interictal class using k-Nearest Neighbor. Higher Order Spectra features are used for classifying normal and ictal class using Support Vector Machine with Radial basis function as kernel and non linear features are used for classifying interictal and ictal class using Naive Bayes. Final verdict is taken by meta classifier with meta learning algorithm Stacking Correspondence Analysis and Nearest Neighbor (SCANN). The proposed method surpasses the existing methods in literature in terms of sensitivity and specificity.

In this study we introduce the k-Strongest Strengths (kSS) Classification Algorithm, a novel approach for classification problems based on the well-known k-Nearest Neighbor (kNN) classifier. The proposed kSS method is motivated by an analogy to the Law of Universal Gravitation. The novelty of kSS resides in that instead of only using the neighbors’ labels to classify an unseen object it uses gravitation forces or strengths exerted by training objects. To incorporate this Newtonian concept into kSS, mass to training objects needs to be assigned. Following this idea we propose novel mass functions that exploit object’s topology properties within 11 data sets, which comprise binary and multi-class problems that present high and low imbalance ratio. Experiments show that kSS obtains an average f1 score of 0.87, outperforming other popular Machine Learning methods such as Artificial Neural Networks (0.80), Decision Trees (0.75), other kNN variants (0.79), naive Bayes (0.72) and Support Vector Machines (0.82). All these differences being statistically significant.

[EN] Alpha integration is a family of integrators that encompasses many classic fusion operators (e.g., mean, product, minimum, maximum) as particular cases. This paper proposes vector score integration (VSI), a new alpha integration method for late fusion of multiple classifiers considering the joint effect of all the classes of the multi-class problem. Theoretical derivations to optimize the parameters of VSI for achieving the minimum probability of error are provided. VSI was applied to two classification tasks using electroencephalographic signals. The first task was the automatic stage classification of a neuropsychological test performed by epileptic subjects and the second one was the classification of sleep stages from apnea patients. Four single classifiers (linear and quadratic discriminant analysis, naive Bayes, and random forest) and three competitive fusion methods were estimated for comparison: mean, majority voting, and separated score integration (SSI). SSI is based on alpha integration, but unlike the proposed method, it considers the scores from each class in isolation, not accounting for possible dependencies among scores corresponding to different classes. VSI was able to optimally combine the results from all the single classifiers, in terms of accuracy and kappa coefficient, and outperformed the results of the other fusion methods in both applications., This work was supported by the Spanish Administration and European Union under grant TEC2017-84743-P.

Multilabel classification is the task of assigning relevant labels to an instance, and it has received considerable attention in recent years. This task can be performed by extending a single-label classifier, such as the naive Bayes classifier, to utilize the useful relations among labels for achieving better multilabel classification accuracy. However, the conventional multilabel naive Bayes classifier treats each label independently and hence neglects the relations among labels, resulting in degenerated accuracy. We propose a new multilabel naive Bayes classifier that considers the relations or dependence among labels. Experimental results show that the proposed method outperforms conventional multilabel classifiers.

We discuss a more powerful probabilistic graphical model for discovering semantic patterns from sequential text data, such as sentences. It is developed based on the idea that each word (or each symbol) in a sentence itself might carry lexical, semantic, or syntactic information, which can be used to replace conditional dependences in existing methods. Hence, our method has fewer conditional independence assumptions in contrast to these existing probabilistic graphical methods, such as CRFs, HMMs, MEMMs and Naive Bayes. Moreover, our method does not need to employ dynamic programming and therefore the on-line time complexity and memory complexity are reduced. We test the method on discovering noun phrases, the meaning of an ambiguous word, and semantic arguments of a verb in a sentence. We find that the misclassification rate is smaller compared to previously published results on the same data sets. For example, the method achieves an average f-measure of 98.25% for recognizing noun phrases on WSJ data from Penn Treebank; an average accuracy of 81.12% for recognizing the six sense word line; an average f-measure of 93.61% for classifying semantic argument boundaries of a verb in a sentence on WSJ data from Penn Treebank and PropBank.

A sequence of apparently ad-hoc hand postures can generate meaningful dynamic gestures which can be utilized in interface controls for computer, television, or games. In order to develop deployable systems with these gestures, selected descriptors should be fast enough to meet the live recognition requirements. This paper proposes framework for a practical system capable of recognizing continuous dynamic gestures characterized by short-duration posture sequences. A depth-based modification to the shape matrix is devised to describe hand silhouettes, which gives a faster alternative to region-based descriptors. Postures are recognized using depth matrix and 1-nearest neighbor strategy. Posture sequence labels are predicted by a dynamic naive Bayes classifier which works in association with an adaptive windowing mechanism. The conducted experiments report up to 96.2% accurate results with mean accuracy of 95.2% on dynamic gesture dataset. Depth matrix computation takes a maximum of 2ms time.

Recently, the studies on biomedical effects of nanomaterials have already achieved many progresses tightly relating to human health. But the large amounts of research achievements in nanomaterial journals bomb too massive data to clarify the research highlight prediction. Fortunately, automatic text mining methods can complete the extracting information from a large set of documents efficiently by machining learning methods. We used both Naive Bayes and K-means clustering algorithms on manually labeled research data sets. It is 88.62% by the Naive Bayes algorithm classification result of 5-folds cross validation on sampled libraries. By applying the optimized Naive Bayes classification model, we made the research highlight trend prediction based on research achievements of biomedical effects of nanomaterials in 22 cutting edge nanomaterial journals including 350,000 original literatures in period from 2000 to 2017. The data mining clarified the polymer nanomaterial is the most researched nanomaterial but with a decreasing trend. The research interests of metallic and carbon based nanomaterial follow the polymer one, and possess increasing trend. We could predict that the research highlight trend on biomedical effects of nanomaterials is focused on polymer, metallic and carbon based material systems in the near future.

Term weighting is known as a text presentation strategy to assign appropriate value to each term to improve the performance of text classification in the task of transforming the content of textual document into a vector in the term space. Supervised weighting methods using the information on the membership of training documents in predefined classes are naturally expected to provide better results than the unsupervised ones. In this paper, a new weighting scheme is proposed via a matching score function based on a probabilistic model. We introduce a latent variable to indicate whether a term contains text classification information or not, specify conjugate priors and exploit the conjugacy by integrating out the latent indicator and the parameters. Then the non-discriminating terms can be assigned weights close to 0. Experimental results using kNN and SVM classifiers illustrate the effectiveness of the proposed approach on both small and large text data sets.

In this paper, we introduce an alternative framework for selecting a most relevant subset of the original set of features for the purpose of text categorization. Given a feature set and a local feature evaluation function (such as chi-square measure, mutual information etc.,) the proposed framework ranks the features in groups instead of ranking individual features. A group of features with rth rank is more powerful than the group of features with (r+1)th rank. Each group is made up of a subset of features which are supposed to be capable of discriminating every class from every other class. The added advantage of the proposed framework is that it automatically eliminates the redundant features while selecting features without requirement of study of features in combination. Further the proposed framework also helps in handling overlapping classes effectively through selection of low ranked yet powerful features. An extensive experimentation has been conducted on three benchmarking datasets using four different local feature evaluation functions with Support Vector Machine and Naive Bayes classifiers to bring out the effectiveness of the proposed framework over the respective conventional counterparts.

Class imbalance can be defined as a span among data mining, machine learning and pattern recognition domains that provides to learn from a data-space having unequal class distribution. Common classifiers when trained by imbalanced data tend to bias towards the class possessing bulk instances causing misclassification of upcoming patterns/instances. The study reveals that presence of redundant borderline instances and outliers in the data-space severely catalyzes the effect of class imbalance. The Condensed Nearest Neighbor and Tomek-link undersampling techniques are used as the baseline systems for the present study, and an improved undersampling algorithm is proposed to be employed in the pre-processing stage by amalgamating aspects of outlier and redundancy detection to the baseline system. The proposed scheme imparts to detect outlier, redundant and noisy instances having least contribution in estimating accurate class labels. Thus, a data-level solution has been offered to the concerned problem with novelty in effective elimination of majority instances without losing valuable information. The proposed scheme is implemented and validated with Back Propagation Neural Network (BPNN), K-Nearest-Neighbor (K-NN), Support Vector Machine (SVM) and Naive Bayes classifiers for 10 real-life datasets. The experimental results obtained clearly manifest the superiority of the proposed scheme over the baseline schemes.

UPTAN, a Bayesian network, for uncertain data under PU learning scenario is given.Uncertain Conditional Mutual Information (UCMI) is proposed.The algorithm for learning the structure of the Bayesian network is given.The approach for estimating parameters of the Bayesian network is given.UPTAN outperforms UPNB, a state-of-art algorithm, in our experiment. The current state-of-art for tackling the problem of classification of static uncertain data under PU learning (Positive Unlabeled Learning) scenario, is UPNB. It is based on the Bayesian assumption, which does not hold for real-life applications, and hence it may depress the classification performance of UPNB. In this paper, we propose UPTAN (Uncertain Positive Tree Augmented Naive Bayes), a Bayesian network algorithm, so as to utilize the dependence information among uncertain attributes for classification. We propose uncertain conditional mutual information (UCMI) for measuring the mutual information between uncertain attributes, and then use it to learn the tree structure of Bayesian network. Furthermore, we give our approach for estimating the parameters of the Bayesian network for uncertain data without negative training examples. Our experiments on 20 UCI datasets show that UPTAN has excellent classification performance, with average F1 being 0.8257, which outperforms UPNB by 3.73%.

A novel multi-sensor framework is proposed for Sign Language Recognition (SLR).The framework recognize dynamic sign words performed by hearing impaired persons.A recognition combination framework using Coupled-HMM (CHMM) is proposed for SLR.Results shows higher accuracy on CHMM over uni-modal and other fusion approaches. Recent development of low cost depth sensors such as Leap motion controller and Microsoft kinect sensor has opened up new opportunities for Human-Computer-Interaction (HCI). In this paper, we propose a novel multi-sensor fusion framework for Sign Language Recognition (SLR) using Coupled Hidden Markov Model (CHMM). CHMM provides interaction in state-space instead of observation states as used in classical HMM that fails to model correlation between inter-modal dependencies. The framework has been used to recognize dynamic isolated sign gestures performed by hearing impaired persons. The dataset has been tested using existing data fusion approaches. The best recognition accuracy has been achieved as high as 90.80% with CHMM. Our CHMM-based approach shows improvement in recognition performance over popular existing data fusion techniques.

Unconstrained correlation filters based trackers achieve superior performance with high speed in visual tracking. However, such unconstrained correlation filters do not impose any hard constraint to their responses to have a certain value, which brings about classification ambiguity on intractable samples (i.e., two similar samples from different classes). To tackle this issue, in this paper, constrained correlation filter is introduced into visual tracking framework to alleviate classification ambiguity for more accurate target location. By imposing distinguishable hard constraints on the response map to different classes, a supervised coding method is proposed to encode various candidate samples by a discriminative filter bank. The learned high-level feature vectors are sent to a Naive Bayes classifier to separate the target from the background. Besides, parameters updating schemes in the constrained filter and classifier are introduced to adapt to appearance changes of the target with less possibility of drifting. Both qualitative and quantitative evaluations on Object Tracking Benchmark (OTB) show that the proposed tracking method achieves favorable performance compared with other state-of-the-art methods.

Defining notion of Bayes covariance.Proving existence and uniqueness of Bayes covariant classifier of 3 categories.Explicit construction of a Bayes covariant classifier for any number of categories.A proof that previously considered methods are not Bayes covariant.Comparison of various methods for combining pairwise classifiers via MDS, and speech frame classification. We consider multi-class classification models built from complete sets of pairwise binary classifiers. The BradleyTerry model is often used to estimate posterior distributions in this setting. We introduce the notion of Bayes covariance, which holds if the multi-class classifier respects multiplicative group action on class priors. As a consequence, a Bayes covariant method yields the same result whether new priors are considered before or after combination of the individual classifiers, which has several practical advantages for systems with feedback. In the paper, we construct a Bayes covariant combining method and compare it with previously published methods in both Monte Carlo simulations as well as on a practical speech frame recognition task.

Cost-sensitive learning is often desirable in many real-world applications.We review the related work on naive Bayes and test-cost sensitive learning.We propose a new test-cost sensitive naive Bayes algorithm.The proposed algorithm selects an optimal attribute subset with minimal test cost.Experimental results on a large number of datasets validate its effectiveness. Some existing test-cost sensitive learning algorithms are about balancing act of the misclassification cost and the total test cost, and the others focus on the balance between the classification accuracy and the total test cost. By far, however, few works reduce the total test cost, yet at the same time maintain the high classification accuracy. In order to achieve this goal, this paper modifies the backward greedy search strategy employed in selective Bayesian classifiers (SBC), which is a state-of-the-art improved naive Bayes algorithm pursuing the high classification accuracy but ignoring the total test cost. We call the resulting model test-cost sensitive naive Bayes (TCSNB). TCSNB conducts a modified backward greedy search strategy to select an optimal attribute subset with the minimal total test cost, yet at the same time maintains the high classification accuracy that characterizes SBC. Extensive empirical study validates its effectiveness and efficiency.

This paper presents an exploratory analysis of sex prediction from swipe gestures.Details of the software and data collection protocol are provided for reproducibility.The BestFirst feature selection technique has been analysed.Naive Bayes, logistic regression, SVM and decision tree classifiers have been tested.The results confirm the possibility of sex prediction from swipe gestures. Touch and multi-touch gestures are becoming the most common way to interact with technology such as smart phones, tablets and other mobile devices. The latest touch-screen input capacities have tremendously increased the quantity and quality of available gesture data, which has led to the exploration of its use in multiple disciplines from psychology to biometrics. Following research studies undertaken in similar modalities such as keystroke and mouse usage biometrics, the present work proposes the use of swipe gesture data for the prediction of soft-biometrics, specifically the user's sex. This paper details the software and protocol used for the data collection, the feature set extracted and subsequent machine learning analysis. Within this analysis, the BestFirst feature selection technique and classification algorithms (naive Bayes, logistic regression, support vector machine and decision tree) have been tested. The results of this exploratory analysis have confirmed the possibility of sex prediction from the swipe gesture data, obtaining an encouraging 78% accuracy rate using swipe gesture data from two different directions. These results will hopefully encourage further research in this area, where the prediction of soft-biometrics traits from swipe gesture data can play an important role in enhancing the authentication processes based on touch-screen devices.

We approach the problem of early recognizing gestures in video.A naive Bayes classifier is used to approach the problem.Experiments are conducted in a variety of data sets.Results obtained with the proposed method outperform state of the at. Early gesture/action recognition is the task of determining the identity of a gesture/action with as few information as possible. Although the topic is relatively new, there are some methods that address this problem. However, existing methods rely on complex modeling procedures, that do not necessarily paid off the computational effort. Thus, simple yet effective and efficient techniques are required for this task. This paper describes a new methodology for early gesture recognition based on the well known naive Bayes classifier. The method is extremely simple and very fast, yet it compares favorably with more elaborated state of the art methodologies. The naive baseline is based on three main observations: (1) the effectiveness of the naive Bayes classifier in text mining problems; (2) the link between natural language processing and computer vision via the bag-of-words representation; and (3) the cumulative-evidence nature of the inference process of naive Bayes. We evaluated the proposed method in several collections that included segmented and continuous video. Experimental results show that the proposed methodology compares favorably with state of the art methodologies that are more elaborated or were specifically designed for this purpose.

The global selection index can be determined from the local selection indexes.The local selection index can be calculated in its own dimension.The prediction function can be factorized.The NB models can be selectively pruned by thresholding the LSIs.Feature selection and weighting work hand-in-hand to improve classification. Feature subset selection is known to improve text classification performance of various classifiers. The model using the selected features is often regarded as if it had generated the data. By taking its uncertainty into account, the discrimination capabilities can be measured by a global selection index (GSI), which can be used in the prediction function. In this paper, we propose a latent selection augmented naive (LSAN) Bayes classifier. By introducing a latent feature selection indicator, the GSI can be factorized into each local selection index (LSI). Using conjugate priors, the LSI for feature evaluation can be explicitly calculated. Then the feature subset selection models can be pruned by thresholding the LSIs, and the LSAN classifier can be achieved by the product of a small percentage of single feature model averages. The numerical results on some real datasets show that the proposed method outperforms the contrast feature weighting methods, and is very competitive if compared with some other commonly used classifiers such as SVM.

This paper proposes a human implicit intent recognition system based on electroencephalography (EEG) signals, for developing an advanced interactive web service engine. We focus on identifying brain state transitions between intentions, and classifying a user's implicit intentions while viewing an image on a web page, based on an EEG experiment. We measure brain state changes between a navigational intention and an informational intention by using phase synchrony; i.e., the phase locking value (PLV) in an EEG. Comparing PLVs that correspond to the two intention states is useful for determining a human's implicit intention. In order to discriminate between a user's implicit intentions using a PLV, we must extract features based on an EEG analysis. For this purpose, we identify the most reactive band within the full band of brain signals. Theta (4-7 Hz), alpha (8-13 Hz), beta-1 (14-22 Hz), and beta-2 (23-30 Hz) bands are used to extract the EEG features from the most reactive EEG band. Subsequently, we select the most significant pairs (MSPs) that are highly reactive and correspond to the intention. According to the proposed method, these features are useful for: (i) showing the brain state transitions regarding intentions, and (ii) classifying a human's implicit intention using several classifiers such as a support vector machine (SVM), Gaussian Mixture Model (GMM), and Naive Bayes. We then compare the results of these classifiers. This study demonstrates the potential uses of these identified brain electrode pairs for cognitive detection and task classification in future brain-computer interface (BCI) applications. We propose a human intent recognition based on EEG signals.In order to classify human's implicit intentions, we extract some features using EEG analysis.The most reactive band and the most significant electrode pairs in the brain.We distinguish between human's implicit intentions based on phase locking values (PLV).

Value difference metric (VDM) is one of the widely used distance functions.We propose local value difference metric (LVDM).LVDM uses a modified decision tree algorithm to find the neighborhood of the test instance.The experimental results on 36 UCI datasets validate its effectiveness. Value difference metric (VDM) is one of the widely used distance functions designed to work with nominal attributes. Research has indicated that the definition of VDM follows naturally from a simple probabilistic model called a naive Bayes (NB). NB assumes that all the attributes are independent given the class. To further improve the performance of NB, several techniques have been proposed. Among these, an effective technique is local learning. Because VDM has a close relationship with NB, in this paper, we propose a local learning method for VDM. The improved distance function is called local value difference metric (LVDM). When LVDM computes the distance between a test instance and each training instance, the conditional probabilities in VDM are estimated by counting from the neighborhood of the test instance only instead of from all the training data. A modified decision tree algorithm is proposed to determine the neighborhood of the test instance. The experimental results on 43 datasets downloaded from the University of California at Irvine (UCI) show that the proposed LVDM significantly outperforms VDM in terms of the class-probability estimation performance of distance-based learning algorithms.

Sequential pattern recognition assumes the features to be measured successively, one at a time, and therefore the key problem is to choose the next feature optimally. However, the choice of the features may be strongly influenced by the previous feature measurements and therefore the on-line ordering of features is difficult. There are numerous methods to estimate class-conditional probability distributions but it is usually computationally intractable to derive the corresponding conditional marginals. In literature there is no exact method of on-line feature ordering except for the strongly simplifying naive Bayes models. We show that the problem of sequential recognition has an explicit analytical solution which is based on approximation of the class-conditional distributions by mixtures of product components. As the marginal distributions of product mixtures are directly available by omitting superfluous terms in the products, we have a unique non-trivial possibility to evaluate at any decision level the conditional informativity of unobserved features for a general problem of statistical recognition. In this way the most informative feature guarantees, for any given set of preceding measurements, the maximum decrease of decision uncertainty.

It is well known that the naive Bayesian classifier assumes the attribute independence given the class. According to our observation, some distance functions also assume the attribute independence, such as Value Difference Metric (VDM). Short and Fukunaga Metric (SFM) is another widely used distance function, which does not assume the attribute independence. In this paper, we investigate the attribute independence assumption in VDM, and propose a Modified Short and Fukunaga Metric (MSFM) based on the attribute independence assumption. We find that MSFM is surprisingly similar to VDM. In fact, based on some assumptions, our MSFM can be regarded as a logarithmic modification of VDM. That is to say, in some sense, a logarithmic modification of SFM is equivalent to a logarithmic modification of VDM. Our experimental results on a large number of UCI benchmark datasets show that MSFM significantly outperforms SFM and SF2LOG (another improved version of SFM), and almost ties VDM.

This paper addresses the dynamic recognition of basic facial expressions in videos using feature subset selection. Feature selection has been already used by some static classifiers where the facial expression is recognized from one single image. Past work on dynamic facial expression recognition has emphasized the issues of feature extraction and classification, however, less attention has been given to the critical issue of feature selection in the dynamic scenario. The main contributions of the paper are as follows. First, we show that dynamic facial expression recognition can be casted into a classical classification problem. Second, we combine a facial dynamics extractor algorithm with a feature selection scheme for generic classifiers. We show that the paradigm of feature subset selection with a wrapper technique can improve the dynamic recognition of facial expressions. We provide evaluations of performance on real video sequences using five standard machine learning approaches: Support Vector Machines, K Nearest Neighbor, Naive Bayes, Bayesian Networks, and Classification Trees.

Many approaches attempt to improve naive Bayes and have been broadly divided into five main categories: (1) structure extension; (2) attribute weighting; (3) attribute selection; (4) instance weighting; (5) instance selection, also called local learning. In this paper, we work on the approach of structure extension and single out a random Bayes model by augmenting the structure of naive Bayes. We called it random one-dependence estimators, simply RODE. In RODE, each attribute has at most one parent from other attributes and this parent is randomly selected from log"2m (where m is the number of attributes) attributes with the maximal conditional mutual information. Our work conducts the randomness into Bayesian network classifiers. The experimental results on a large number of UCI data sets validate its effectiveness in terms of classification, class probability estimation, and ranking.

This paper proposes a supervised multiscale Bayesian texture classifier. The classifier exploits the dual-tree\ud complex wavelet transform (DT-CWT) to obtain complex-valued multiscale representations of training texture samples for each texture class. The high-pass subbands of DT-CWT decomposition of a texture image are used to form a multiscale feature vector representing magnitude and phase features. For computational efficiency, the dimensionality of feature vectors is reduced using principal component analysis (PCA). The class conditional probability density function of low-dimensional feature vectors for each texture class is then estimated by using Parzen-window estimate with identical Gaussian kernels and is used to represent the texture class. A query texture image is classified as the corresponding texture class with the highest a posteriori probability according to a Bayesian inferencing. The superior performance and robustness of the proposed classifier is demonstrated for classifying texture images from image databases. The proposed multiscale texture feature vector extracted from both magnitude and phase of DT-CWT subbands of a query image is also shown to be effective for texture retrieval.

Motivation: With the wealth of sequence data and the huge amount of data generated from molecular technologies, the issue of gene classification/prediction has become a central challenge in the field of microarray data analysis. This has led to the application of many well-established supervised learning (SL) algorithms in an attempt to provide more accurate and automatic diagnosis class (cancer/non cancer) prediction. Virtually all research on SL addresses the task of learning to classify complete domain instances. However, in some research situations we often have to classify instances given incomplete vectors, which can affect the predictive accuracy of learned classifiers. The task of learning an accurate incomplete data classifier from instances raises a number of new issues some of which have not been properly addressed by bioinformatics research. Thus, an effective missing value estimation method is required for improving predictive accuracy. Results: The essence of the approach is the proposal that prediction using supervised learning can be improved in probabilistic terms given incomplete microarray data. This imputation approach is based on the a priori probability of each value determined from the instances at that node of a decision tree (PDT) that have specified values. The proposed approach exploits the total probability and Bayes' theorems and it has three versions. We evaluate our approach with other supervised learning techniques including C5.0, classification and regression trees (CART), k-nearest neighbour (k-NN), linear discrimination (LD) naive Bayes classifier (NBC), Repeated Incremental Pruning to Produce Error Reduction (RIPPER) and support vector machines (SVMs), from the point of view of their effect or tolerance of incomplete test data. Eight cancer related gene expression datasets are utilized for this task. Experimental results are provided to illustrate the efficiency and the robustness of the proposed algorithm.

In this paper, we describe a Bayesian classification method that informatively combines diverse sources of information and multiple feature spaces for multiclass problems. The proposed method is based on recent advances in kernel approaches where the integration of multiple object descriptors, or feature spaces, is achieved via kernel combination. Each kernel constructs a similarity metric between objects in a particular feature space and then having a common metric across modalities an overall combination can be constructed. We follow a hierarchical Bayesian approach, which introduces prior distributions over random variables and we construct a Gibbs sampling Markov chain Monte Carlo (MCMC) solution which is naturally derived from the employed multinomial probit likelihood. The methodology is the basis for possible deterministic approximations such as variational or maximum-a-posteriori estimators, and it is compared against the well-known classifier combination methods on the classification of handwritten numerals. The results of the proposed method show a significant improvement over the best individual classifier and match the performance of the best multiple classifier combination, whilst reducing the computational requirements of combining classifiers and offering additional information on the significance of the contributing sources.

Bayesian classifier is an effective and fundamental methodology for solving classification problems. However, it is computationally efficient when all features are considered simultaneously. But sometimes all the features do not contribute significantly to classification. Also the noisy attributes sometimes may decrease the accuracy of classifier. So before classification feature selection is used as a pre-processing step. When the features are added one by one in Bayesian classifier in batch mode in forward selection method huge computation is involved. In this paper, an incremental Bayesian classifier for multivariate normal distribution datasets are proposed. The proposed incremental Bayesian classifier is computationally efficient over batch Bayesian classifier in terms of time. The effectiveness of the proposed incremental Bayesian classifier has been demonstrated through experiments on different datasets. It is found on the basis of experiments that the incremental Bayesian classifier has an equivalent power compared to batch Bayesian classifier in terms classification accuracy. However, the proposed incremental Bayesian classifier has very high speed efficiency in comparison to batch Bayesian classifier.

In this work, we propose an approach to binary classification based on an extension of Bayes Point Machines. Particularly, we take into account the whole set of hypotheses that are consistent with the data (the so-called version space) and the intrinsic noise in class labeling. We follow a Bayesian approach and compute an approximate posterior distribution for the model parameters, which leads to a predictive distribution over unseen data. The most compelling feature of the proposed model is that it is able to learn the noise present in the data with no additional cost. All the computations are carried out by means of the approximate Bayesian inference algorithm Expectation Propagation. Experimental results indicate that the proposed approach outperforms Support Vector Machines over several of the classification problems studied and is competitive with other Bayesian classification algorithms based on Gaussian Processes.

A classifier based on dynamic time warping (DTW) has been developed to perform the classification of human chromosomes. DTW is used in speech recognition applications to compare two time-sequences. This paper describes a method to adapt the DTW technique in order to deal with the length and the density profile, which are common features used in classifying chromosomes. The DTW classifier is able to compare chromosomes with different elongations. Since chromosomes are non-rigid objects, the proposed classifier has the main advantage of requiring only a small training set in comparison with the conventional methods based on Bayesian classifiers or neural networks. For the same classification accuracy of 81.0%, we achieve a reduction of 88% of the size of the training set in comparison with a Bayesian classifier.

In this paper a new classifier has been designed based on the learning automata. This classifier can efficiently approximate the decision hyperplanes in the feature space without need to know the class distributions and the a priori probabilities. The performance of the proposed classifier has been tested on different kinds of benchmarks with nonlinear, overlapping class boundaries and different feature space dimensions. Extensive experimental results on these data sets are provided to show that the performance of the proposed classifier is comparable to, sometimes better than multi-layer perceptron, k-nearest neighbor classifier, genetic classifier, and particle swarm classifier. Also the comparative results are provided to show the effectiveness of the proposed method in comparison to similar researches. Furthermore the effect of the number of training points on the performance of the designed classifier is investigated. It is found that as the number of training data increases, the performance of the classifier tends to the performance of Bayes classifier which is an optimal one.

The positive unlabeled learning term refers to the binary classification problem in the absence of negative examples. When only positive and unlabeled instances are available, semi-supervised classification algorithms cannot be directly applied, and thus new algorithms are required. One of these positive unlabeled learning algorithms is the positive naive Bayes (PNB), which is an adaptation of the naïve Bayes induction algorithm that does not require negative instances. In this work we propose two ways of enhancing this algorithm. On one hand, we have taken the concept behind PNB one step further, proposing a procedure to build more complex Bayesian classifiers in the absence of negative instances. We present a new algorithm (named positive tree augmented naive Bayes, PTAN) to obtain tree augmented naive Bayes models in the positive unlabeled domain. On the other hand, we propose a new Bayesian approach to deal with the a priori probability of the positive class that models the uncertainty over this parameter by means of a Beta distribution. This approach is applied to both PNB and PTAN, resulting in two new algorithms. The four algorithms are empirically compared in positive unlabelled learning problems based on real and synthetic databases. The results obtained in these comparisons suggest that, when the predicting variables are not conditionally independent given the class, the extension of PNB to more complex networks increases the classification performance. They also show that our Bayesian approach to the a priori probability of the positive class can improve the results obtained by PNB and PTAN.

The probabilistic nearest neighbour (PNN) method for pattern recognition was introduced to overcome a number of perceived shortcomings of the nearest neighbour (NN) classifiers namely the lack of any probabilistic semantics when making predictions of class membership. In addition the NN method possesses no inherent principled framework for inferring the number of neighbours, K, nor indeed associated parameters related to the chosen metric. Whilst the Bayesian inferential methodology underlying the PNN classifier undoubtedly overcomes these shortcomings there has been to date no extensive systematic study of the performance of the PNN method nor any comparison with the standard non-probabilistic approach. We address this issue by undertaking an extensive empirical study which highlights the essential characteristics of PNN when compared to a cross-validated K-NN.

Traffic surveillance system capable of providing accurate real-time traffic measurements is a backbone of fully exploiting a variety of advanced traffic management systems. Vehicle classification information is one of the important measurements that we need to obtain in practice, which is invaluable for various aspects of transportation including engineering and planning. This study develops vehicle classification algorithms using inductive signatures obtained from a prototype innovative loop sensor, known as a 'blade'. A probabilistic neural network (PNN), a neural network implementation of multivariate Bayesian classification scheme, and a heuristic classification algorithm are employed to classify vehicle types. Vehicle feature vectors representing the vehicle shapes are extracted from blade signatures, and then utilized as inputs of the proposed algorithm. The classification performances are investigated with four different types of vehicles including passenger car, pick-up truck, sports utility vehicle, and van. N-fold cross validation is applied to evaluate the performances. Encouraging result of 70.8% overall correct classification rate obtained from the PNN-based classification algorithm demonstrates the technical feasibility of the proposed algorithm for obtaining vehicle classification information.

Accurate ranking, measured by AUC (the area under the ROC curve), is crucial in many real-world applications. Most traditional learning algorithms, however, aim only at high classification accuracy. It has been observed that traditional decision trees produce good classification accuracy but poor probability estimates. Since the ranking generated by a decision tree is based on the class probabilities, a probability estimation tree (PET) with accurate probability estimates is desired in order to yield high AUC. Some researchers ascribe the poor probability estimates of decision trees to the decision tree learning algorithms. To our observation, however, the representation also plays an important role. In this paper, we propose to extend decision trees to represent a joint distribution and conditional independence, called conditional independence trees (CITrees), which is a more suitable model for yielding high AUC. We propose a novel AUC-based algorithm for learning CITrees, and our experiments show that the CITree algorithm outperforms the state-of-the-art decision tree learning algorithm C4.4 (a variant of C4.5), naive Bayes, and NBTree in AUC. Our work provides an effective model and algorithm for applications in which an accurate ranking is required.

While Naive Bayes classifier (NB) is Bayes-optimal for independent features, we prove that it is also optimal for two equiprobable classes and two features with equal class-conditional covariances. Although strict optimality does not extend for three features, equal covariances are expected to be beneficial in higher-dimensional spaces.

In classifier combination, the relative values of a posteriori probabilities assigned to different hypotheses are more important than the accuracy of their estimates. Because of this, the independence requirement in naive Bayesian fusion should be examined from combined accuracy point of view. In this study, it is investigated whether there is a set of dependent classifiers which provides a better combined accuracy than independent classifiers when naive Bayesian fusion is used. For this purpose, two classes and three classifiers case is initially considered where the pattern classes are not equally probable. Taking into account the increased complexity in formulations, equal a priori probabilities are considered in the general case where N classes and K classifiers are used. The analysis carried out has shown that the combination of dependent classifiers using naive Bayesian fusion may provide much better combined accuracies compared to independent classifiers.

Many studies have been made to compare the many different methods of supervised classification which have been developed. While conducting a large meta-analysis of such studies, we spotted some anomalous results relating to the Naive Bayes method. This paper describes our detailed investigation into these anomalies. We conclude that a very large comparative study probably mislabelled another method as Naive Bayes, and that the Statlog project used the right method, but possibly incorrectly reported its provenance. Such mistakes, while not too harmful in themselves, can become seriously misleading if blindly propagated by citations which do not examine the source material in detail.

This paper deals with the recognition of human body postures from a cloud of 3D points acquired by a human body scanner. Motivated by finding a representation that embodies a high power of discrimination between posture classes, a new type of 3D shape descriptors is suggested, namely wavelet transform coefficients (WC). These features can be seen as an extension to 3D of the 2D wavelet shape descriptors developed by (Shen, D., Ip, H.H.S., 1999. Pattern Recognition, 32, 151-165). The WC is compared with other 3D shape descriptors, within a Bayesian classification framework. Experiments with real scan data show that the WC outperforms other standard 3D shape descriptors in terms of discrimination power and classification rate.

Existing gait recognition approaches do not give their theoretical or experimental performance predictions. Therefore, the discriminating power of gait as a feature for human recognition cannot be evaluated. In this paper, we first propose a kinematic-based approach to recognize human by gait. The proposed approach estimates 3D human walking parameters by performing a least squares fit of the 3D kinematic model to the 2D silhouette extracted from a monocular image sequence. Next, a Bayesian-based statistical analysis is performed to evaluate the discriminating power of extracted stationary gait features. Through probabilistic simulation, we not only predict the probability of correct recognition (PCR) with regard to different within-class feature variance, but also obtain the upper bound on PCR with regard to different human silhouette resolution. In addition, the maximum number of people in a database is obtained given the allowable error rate. This is extremely important for gait recognition in large databases.

We propose anti-spare filtering methods for agglutinative languages in general and for Turkish in particular. The methods are dynamic and are based on Artificial Neural Networks (ANN) and Bayesian Networks. The developed algorithms are user-specific and adapt themselves with the characteristics of the incoming e-mails. The algorithms have two main components. The first one deals with the morphology of the words and the second one classifies the e-mails by using the roots of the words extracted by the morphological analysis. Two ANN structures, single layer perceptron and multi-layer perceptron, are considered and the inputs to the networks are determined using binary model and probabilistic model. Similarly, for Bayesian classification, three different approaches are employed: binary model, probabilistic model, and advanced probabilistic model. In the experiments, a total of 750 e-mails (410 spare and 340 normal) were used and a success rate of about 90% was achieved.

The goal of this paper is to offer a framework for classification of images and video according to their "type", or "style"--a problem which is hard to define, but easy to illustrate; for example, identifying an artist by the Style of his/ her painting, or determining the activity in a video sequence. The paper offers a simple classification paradigm based on local properties of spatial or spatio-temporal blocks. The learning and classification are based on the naive Bayes classifier. A few experimental results are presented.

This paper presents a floating search approach for learning the network structure of Bayesian network classifiers. A Bayesian network classifier is used which in combination with the search algorithm allows simultaneous feature selection and determination of the structure of the classifier.The introduced search algorithm enables conditional exclusions of previously added attributes and/or arcs from the network classifier. Hence, this algorithm is able to correct the network structure by removing attributes and/or arcs between the nodes if they become superfluous at a later stage of the search. Classification results of selective unrestricted Bayesian network classifiers are compared to naive Bayes classifiers and tree augmented naive Bayes classifiers. Experiments on different data sets show that selective unrestricted Bayesian network classifiers achieve a better classification accuracy estimate in two domains compared to tree augmented naive Bayes classifiers, whereby in the remaining domains the performance is similar. However, the achieved network structure of selective unrestricted Bayesian network classifiers is simpler.

Rotation invariant texture features are derived from the even symmetric Gabor filtered images of texture. The feature used is modified average absolute deviation from mean. Sixty Brodatz textures rotated in 12 different directions are classified using these features. Equal number of samples are used for training and testing phase. The percentage correct classification is 81.02. Segmentation of texture images with rotated textures is demonstrated using the features.

We propose here an unsupervised Bayesian classifier based on a non-parametric expectation-maximization algorithm. The non-parametric aspect comes from the use of the orthogonal probability density function (pdf) estimation, which is reduced to the estimation of the first Fourier coefficients of the pdf with respect to a given orthogonal basis. So, the mixture identification step based on the maximization of the likelihood can be realized without hypothesis on the conditional pdf's distribution. This means that for the unsupervised image segmentation example we do not need any assumption for the gray level image pixels distribution. The generalization to the multivariate case can be obtained by considering the multidimensional orthogonal function basis. In this paper, we give some simulation results for the determination of the smoothing parameter and to compute the error of classification.

The problem studied is the behavior of a discrete classifier on a finite learning sample. With naive Bayes approach, the value of misclassification probability is represented as a random function, for which the first two moments are analytically derived. For arbitrary distributions, this allows evaluating learning sample size sufficient for the classification with given admissible misclassification probability and confidence level. The comparison with statistical learning theory shows that the suggested approach frequently recommends significantly smaller learning sample size.