Search

Highlights • Almost all existing attribute weighting approaches to naive Bayes are class-independent. • We propose a new class-specific attribute weighting paradigm for naive Bayes. • The resulting model is called class-specific attribute weighted naive Bayes (CAWNB). • To learn CAWNB, we propose two gradient-based learning algorithms. • The experimental results validate the effectiveness of the proposed algorithms. Abstract Due to its easiness to construct and interpret, along with its good performance, naive Bayes (NB) is widely used to address classification problems in real-world applications. In order to alleviate its conditional independence assumption, a mass of attribute weighting approaches have been proposed. However, almost all these approaches assign each attribute a same (global) weight for all classes. In this paper, we call them the general attribute weighting and argue that for NB attribute weighting should be class-specific (class-dependent). Based on this premise, we propose a new paradigm for attribute weighting called the class-specific attribute weighting, which discriminatively assigns each attribute a specific weight for each class. We call the resulting model class-specific attribute weighted naive Bayes (CAWNB). CAWNB selects class-specific attribute weights to maximize the conditional log likelihood (CLL) objective function or minimize the mean squared error (MSE) objective function, and thus two different versions are created, which we denote as CAWNBCLL and CAWNBMSE, respectively. Extensive empirical studies show that CAWNBCLL and CAWNBMSE all obtain more satisfactory experimental results compared with NB and other existing state-of-the-art general attribute weighting approaches. We believe that for NB class-specific attribute weighting could be a more fine-grained attribute weighting approach than general attribute weighting. [ABSTRACT FROM AUTHOR]

Highlights • A Bayesian denoising hashing method is proposed for robust image retrieval. • We impose extra constraints in the variational Bayes framework to learn robust hash code. • Our method can be combined with many traditional hashing algorithms to achieve state-of-the-art performance on real-world datasets. Abstract Learning to hash is one of the most popular techniques in image retrieval, but few work investigates its robustness to noise corrupted images in which the unknown pattern of noise would heavily deteriorate the performance. To deal with this issue, we present in this paper a Bayesian denoising hashing algorithm whose output can be regarded a denoised version of the input hash code. We show that our method essentially seeks to reconstruct a new but more robust hash code by preserving the original input information while imposing extra constraints so as to correct the corrupted bits. We optimized this model in variational Bayes framework which has a closed-form update in each iteration that is more efficient than numerical optimization. Furthermore, our method can be added at the top of any original hashing layer, serving as a post-processing denoising layer with no change to previous training procedure. Experiments on three popular datasets demonstrate that the proposed method yields robust and meaningful hash code, which significantly improves the performance of state-of-the-art hash learning methods on challenging tasks such as large-scale natural image retrieval and retrieval with corrupted images. [ABSTRACT FROM AUTHOR]

Topic modeling is a powerful approach for modeling data represented as high-dimensional histograms. While the high dimensionality of such input data is extremely beneficial in unsupervised applications including language modeling and text data exploration, it introduces difficulties in cases where class information is available to boost up prediction performance. Feeding such input directly to a classifier suffers from the curse of dimensionality. Performing dimensionality reduction and classification disjointly, on the other hand, cannot enjoy optimal performance due to information loss in the gap between these two steps unaware of each other. Existing supervised topic models introduced as a remedy to such scenarios have thus far incorporated only linear classifiers in order to keep inference tractable, causing a dramatical sacrifice from expressive power. In this paper, we propose the first Bayesian construction to perform topic modeling and non-linear classification jointly. We use the well-known Latent Dirichlet Allocation (LDA) for topic modeling and sparse Gaussian processes for non-linear classification. We combine these two components by a latent variable encoding the empirical topic distribution of each document in the corpus. We achieve a novel variational inference scheme by adapting ideas from the newly emerging deep Gaussian processes into the realm of topic modeling. We demonstrate that our model outperforms other existing approaches such as: (i) disjoint LDA and non-linear classification, (ii) joint LDA and linear classification, (iii) joint non-LDA linear subspace modeling and linear classification, and (iv) non-linear classification without topic modeling, in three benchmark data sets from two real-world applications: text categorization and image tagging. [ABSTRACT FROM AUTHOR]

SuperParent-One-Dependence Estimators (SPODEs) represent a family of semi-naive Bayesian classifiers which relax the attribute independence assumption of Naive Bayes (NB) to allow each attribute to depend on a common single attribute (superparent). SPODEs can effectively handle data with attribute dependency but still inherent NB׳s key advantages such as computational efficiency and robustness for high dimensional data. In reality, determining an optimal superparent for SPODEs is difficult. One common approach is to use weighted combinations of multiple SPODEs, each having a different superparent with a properly assigned weight value ( i.e ., a weight value is assigned to each attribute). In this paper, we propose a self-adaptive SPODEs, namely SODE, which uses immunity theory in artificial immune systems to automatically and self-adaptively select the weight for each single SPODE. SODE does not need to know the importance of individual SPODE nor the relevance among SPODEs, and can flexibly and efficiently search optimal weight values for each SPODE during the learning process. Extensive experiments and comparisons on 56 benchmark data sets, and validations on image and text classification, demonstrate that SODE outperforms state-of-the-art weighted SPODE algorithms and is suitable for a wide range of learning tasks. Results also confirm that SODE provides an appropriate balance between runtime efficiency and accuracy. [ABSTRACT FROM AUTHOR]

Abstract: The gold standard for a classifier is the condition of optimality attained by the Bayesian classifier. Within a Bayesian paradigm, if we are allowed to compare the testing sample with only a single point in the feature space from each class, the optimal Bayesian strategy would be to achieve this based on the (Mahalanobis) distance from the corresponding means. The reader should observe that, in this context, the mean, in one sense, is the most central point in the respective distribution. In this paper, we shall show that we can obtain optimal results by operating in a diametrically opposite way, i.e., a so-called “anti-Bayesian” manner. Indeed, we assert a completely counter-intuitive result that by working with a very few points distant from the mean, one can obtain remarkable classification accuracies. The number of points can sometimes be as small as two. Further, if these points are determined by the order statistics of the distributions, the accuracy of our method, referred to as Classification by Moments of Order Statistics (CMOS), attains the optimal Bayes'' bound. This claim, which is totally counter-intuitive, has been proven for many uni-dimensional, and some multi-dimensional distributions within the exponential family, and the theoretical results have been verified by rigorous experimental testing. Apart from the fact that these results are quite fascinating and pioneering in their own right, they also give a theoretical foundation for the families of Border Identification (BI) algorithms reported in the literature. [Copyright &y& Elsevier]

Abstract: In this paper, we present a novel way of analyzing and summarizing a collection of curves, based on piecewise constant density estimation. The curves are partitioned into clusters, and the dimensions of the curves points are discretized into intervals. The cross-product of these univariate partitions forms a data grid of cells, which represents a nonparametric estimator of the joint density of the curves and point dimensions. The best model is selected using a Bayesian model selection approach and retrieved using combinatorial optimization algorithms. The proposed method requires no parameter setting and makes no assumption regarding the curves; beyond functional data, it can be applied to distributional data. The practical interest of the approach for functional data and distributional data exploratory analysis is presented on two real world datasets. [Copyright &y& Elsevier]

Abstract: The common spatial pattern (CSP) algorithm is effective in decoding the spatial patterns of the corresponding neuronal activities from electroencephalogram (EEG) signal patterns in brain–computer interfaces (BCIs). However, its effectiveness depends on the subject-specific time segment relative to the visual cue and on the temporal frequency band that is often selected manually or heuristically. This paper presents a novel statistical method to automatically select the optimal subject-specific time segment and temporal frequency band based on the mutual information between the spatial–temporal patterns from the EEG signals and the corresponding neuronal activities. The proposed method comprises four progressive stages: multi-time segment and temporal frequency band-pass filtering, CSP spatial filtering, mutual information-based feature selection and naïve Bayesian classification. The proposed mutual information-based selection of optimal spatial–temporal patterns and its one-versus-rest multi-class extension were evaluated on single-trial EEG from the BCI Competition IV Datasets IIb and IIa respectively. The results showed that the proposed method yielded relatively better session-to-session classification results compared against the best submission. [Copyright &y& Elsevier]

Abstract: This paper introduces a novel approach, i.e. block oriented-restoration, based on a Family of Full Range Autoregressive (FRAR) model to restore the information lost, and this adopts the Bayesian approach to estimate the parameters of the model. The Bayesian approach, by combining the prior information and the observed data known as posterior distribution, makes inferences. The loss of information caused is due to errors in communication channels, through which the data are transmitted. In most applications, the data are transmitted block wise. Even if there is loss of a single bit in a block, it causes loss in the whole block and the impact may reflect on its consecutive blocks. In the proposed technique, such damaged blocks are identified, and to restore it, a priori information is searched and extracted from uncorrupted regions of the image; this information and the pixels in the neighboring region of the damaged block are utilized to estimate the parameters of the model. The estimated parameters are employed to restore the damaged block. The proposed algorithm takes advantage of linear dependency of the neighboring pixels of the damaged block and takes them as source to predict the pixels of the damaged block. The restoration is performed at two stages: first, the lone blocks are restored; second, the contiguous blocks are restored. It produces very good results and is comparable with other existing schemes. [Copyright &y& Elsevier]

Abstract: Feature selection is an important preprocessing step for building efficient, generalizable and interpretable classifiers on high dimensional data sets. Given the assumption on the sufficient labelled samples, the Markov Blanket provides a complete and sound solution to the selection of optimal features, by exploring the conditional independence relationships among the features. In real-world applications, unfortunately, it is usually easy to get unlabelled samples, but expensive to obtain the corresponding accurate labels on the samples. This leads to the potential waste of valuable classification information buried in unlabelled samples. In this paper, we propose a new BAyesian Semi-SUpervised Method, or BASSUM in short, to exploit the values of unlabelled samples on classification feature selection problem. Generally speaking, the inclusion of unlabelled samples helps the feature selection algorithm on (1) pinpointing more specific conditional independence tests involving fewer variable features and (2) improving the robustness of individual conditional independence tests with additional statistical information. Our experimental results show that BASSUM enhances the efficiency of traditional feature selection methods and overcomes the difficulties on redundant features in existing semi-supervised solutions. [Copyright &y& Elsevier]

Abstract: The Student''s-t hidden Markov model (SHMM) has been recently proposed as a robust to outliers form of conventional continuous density hidden Markov models, trained by means of the expectation–maximization algorithm. In this paper, we derive a tractable variational Bayesian inference algorithm for this model. Our innovative approach provides an efficient and more robust alternative to EM-based methods, tackling their singularity and overfitting proneness, while allowing for the automatic determination of the optimal model size without cross-validation. We highlight the superiority of the proposed model over the competition using synthetic and real data. We also demonstrate the merits of our methodology in applications from diverse research fields, such as human computer interaction, robotics and semantic audio analysis. [Copyright &y& Elsevier]

Abstract: This paper provides a comparative study on the different techniques of classifying human activities that are performed using body-worn miniature inertial and magnetic sensors. The classification techniques implemented and compared in this study are: Bayesian decision making (BDM), a rule-based algorithm (RBA) or decision tree, the least-squares method (LSM), the k-nearest neighbor algorithm (k-NN), dynamic time warping (DTW), support vector machines (SVM), and artificial neural networks (ANN). Human activities are classified using five sensor units worn on the chest, the arms, and the legs. Each sensor unit comprises a tri-axial gyroscope, a tri-axial accelerometer, and a tri-axial magnetometer. A feature set extracted from the raw sensor data using principal component analysis (PCA) is used in the classification process. A performance comparison of the classification techniques is provided in terms of their correct differentiation rates, confusion matrices, and computational cost, as well as their pre-processing, training, and storage requirements. Three different cross-validation techniques are employed to validate the classifiers. The results indicate that in general, BDM results in the highest correct classification rate with relatively small computational cost. [Copyright &y& Elsevier]

Abstract: The linear discriminant analysis (LDA) is a linear classifier which has proven to be powerful and competitive compared to the main state-of-the-art classifiers. However, the LDA algorithm assumes the sample vectors of each class are generated from underlying multivariate normal distributions of common covariance matrix with different means (i.e., homoscedastic data). This assumption has restricted the use of LDA considerably. Over the years, authors have defined several extensions to the basic formulation of LDA. One such method is the heteroscedastic LDA (HLDA) which is proposed to address the heteroscedasticity problem. Another method is the nonparametric DA (NDA) where the normality assumption is relaxed. In this paper, we propose a novel Bayesian logistic discriminant (BLD) model which can address both normality and heteroscedasticity problems. The normality assumption is relaxed by approximating the underlying distribution of each class with a mixture of Gaussians. Hence, the proposed BLD provides more flexibility and better classification performances than the LDA, HLDA and NDA. A subclass and multinomial versions of the BLD are proposed. The posterior distribution of the BLD model is elegantly approximated by a tractable Gaussian form using variational transformation and Jensen''s inequality, allowing a straightforward computation of the weights. An extensive comparison of the BLD to the LDA, support vector machine (SVM), HLDA, NDA and subclass discriminant analysis (SDA), performed on artificial and real data sets, has shown the advantages and superiority of our proposed method. In particular, the experiments on face recognition have clearly shown a significant improvement of the proposed BLD over the LDA. [Copyright &y& Elsevier]

In this paper, we propose a Bayesian network framework for explicitly modeling strokes and their relationships of characters. A character is modeled as a composition of stroke models, and a stroke as a composition of point models. A point is modeled with 2-D Gaussian distribution for its X–Y position. Relationships between points and strokes are modeled as their positional dependencies. All the models and relationships are represented probabilistically in Bayesian networks. The recognition experiment with on-line handwritten digits showed promising results; the recognition errors of the proposed system were greatly reduced by dependency modeling, and its recognition rates were higher than those of previous methods. [Copyright &y& Elsevier]

In this paper, we propose and integrate two Bayesian methods, one of them for junction detection, and the other one for junction grouping. Our junction detection method relies on a probabilistic edge model and a log-likelihood test. Our junction grouping method relies on finding connecting paths between pairs of junctions. Path searching is performed by applying a Bayesian A* algorithm. Such algorithm uses both an intensity and geometric model for defining the rewards of a partial path and prunes those paths with low rewards. We have extended such a pruning with an additional rule which favors the stability of longer paths against shorter ones. We have tested experimentally the efficiency and robustness of the methods in an indoor image sequence. [Copyright &y& Elsevier]

• A novel hybrid approach which integrates a Faster R-CNN for crack patch detection, a DCNN for crack orientation recognition, and a Bayesian algorithm for integration. It provides a novel framework to combine deep learning models and Bayesian analysis to address challenging vision problems where the deep learning approaches with simple end-to-end learning strategy might not be effective. • A distinctive approach to apply Faster R-CNN for the challenging task of crack detection by training it to detect crack patches of suitable SNR, and a semi-automatic method to annotate crack patches of suitable scales to train a Faster R-CNN. • A new Bayesian integration algorithm based on local spatial proximity, orientation consistency and alignment consistency to connect associated neighboring crack patches and suppress false detections, as well as an efficient algorithm to learn the optimal parameters. Vision-based crack detection is of crucial importance in various industries, and it is very challenging due to weak signals in noisy backgrounds. In this paper, we propose a novel hybrid approach for crack detection in raw images, which combines deep learning models and Bayesian probabilistic analysis for robust crack detection. First, we re-train a state-of-the-art object detector (e.g. a Faster R-CNN) to detect crack patches of suitable SNR (signal-noise-ratio). We design a semi-automatic method to generate ground truths of crack patches along crack lines for training. To further improve the accuracy of crack detections over the whole image, we propose a Bayesian integration algorithm to suppress false detections. Specifically, we use a deep CNN to recognize the orientation of the crack segment in each detected patch. Then, a Bayesian probability is computed on the accumulated evidence from detected adjacent patches within a neighborhood based on spatial proximity, orientation consistency and alignment consistency. The patch which lacks local supports is suppressed as false detection. An algorithm to learn the parameters of Bayesian integration is also derived. Extensive experiments and evaluations are performed on a new comprehensive dataset of crack images. The results show that our approach outperforms the state-of-the-art baseline approach on deep CNN classifier. Ablation experiments are also conducted to show the effectiveness of proposed techniques. [ABSTRACT FROM AUTHOR]