Search

Structure learning is the core of graph model Bayesian Network learning, and the current mainstream single search algorithm has problems such as poor learning effect, fuzzy initial network, and easy falling into local optimum. In this paper, we propose a heuristic learning algorithm HC-PSO combining the HC (Hill Climbing) algorithm and PSO (Particle Swarm Optimization) algorithm, which firstly uses HC algorithm to search for locally optimal network structures, takes these networks as the initial networks, then introduces mutation operator and crossover operator, and uses PSO algorithm for global search. Meanwhile, we use the DE (Differential Evolution) strategy to select the mutation operator and crossover operator. Finally, experiments are conducted in four different datasets to calculate BIC (Bayesian Information Criterion) and HD (Hamming Distance), and comparative analysis is made with other algorithms, the structure shows that the HC-PSO algorithm is superior in feasibility and accuracy. [ABSTRACT FROM AUTHOR]

Aiming at the deficiency that K2 algorithm depends on the maximum number of parent nodes and node of order, this paper proposed an improved Bayesian network structure learning algorithm (MWST-CS-K2). The algorithm firstly obtained the maximum number of parent nodes by constructing the maximum support tree (MWST). Then it used the correlation degree and updated coefficient between variables to set rules for adding, subtracting and reversing edges. The paper used improved cuckoo algorithm to optimize the nest location, and applied the breadth first search strategy to search and traverse to get the node order. Finally, the paper used maximum number of parent nodes and the order of nodes as the input of K2 algorithm to get the final network. Experiments show that in the standard ASIA,SACHS and CHILD network data tests, the average correct edge ratio of MWST-CS-K2 algorithm reaches 97.3%, 87.7 % and 95.6% respectively. The learning effect is better than other corresponding comparison algorithms, and the network structure obtains the most similar to the standard network structure. [ABSTRACT FROM AUTHOR]

Multi-label classification is an important research topic in machine learning, for which exploiting label dependencies is an effective modeling principle. Recently, probabilistic models have shown great potential in discovering dependencies among labels. In this paper, motivated by the recent success of multi-view learning to improve the generalization performance, we propose a novel multi-view probabilistic model named latent conditional Bernoulli mixture (LCBM) for multi-label classification. LCBM is a generative model taking features from different views as inputs, and conditional on the latent subspace shared by the views a Bernoulli mixture model is adopted to build label dependencies. Inside each component of the mixture, the labels have a weak correlation which facilitates computational convenience. The mean field variational inference framework is used to carry out approximate posterior inference in the probabilistic model, where we propose a Gaussian mixture variational autoencoder (GMVAE) for effective posterior approximation. We further develop a scalable stochastic training algorithm for efficiently optimizing the model parameters and variational parameters, and derive an efficient prediction procedure based on greedy search. Experimental results on multiple benchmark datasets show that our approach outperforms other state-of-the-art methods under various metrics. [ABSTRACT FROM AUTHOR]

Parameters tuning of the model and the controller is an essential problem for autonomous vehicles. Traditionally, parameter tuning work is accomplished by manual operation or grid search, which is a tedious and time-consuming work. Recently, thanks to the development of machine learning community, several automatic controller parameter tuning approaches emerged, which usually model the performance function as a Gaussian process, and complete the automatic tuning procedure via Bayesian optimization. However, the existing approaches rarely consider the time-varying feature of the system performance, which is practical in many scenarios, induced by the unmodeled interactions between the system and the environment. In this paper, we take both of the dynamic model uncertainty and the controller parameters uncertainty into account, and tune them to find a global optimal choice for minimizing the time-varying control costs, which is modeled by the time-varying Gaussian process. We provide a novel algorithm, named as time-varying controller optimization. We validate our approach on synthetic simulation and real experiment, respectively. Taking the cumulative regret as the performance metric, we find that our approach has a better performance compared with the stationary algorithm. [ABSTRACT FROM AUTHOR]

In reinforcement learning (RL), the exploration/exploitation (E/E) dilemma is a very crucial issue, which can be described as searching between the exploration of the environment to find more profitable actions, and the exploitation of the best empirical actions for the current state. We focus on the single trajectory RL problem where an agent is interacting with a partially unknown MDP over single trajectories, and try to deal with the E/E in this setting. Given the reward function, we try to find a good E/E strategy to address the MDPs under some MDP distribution. This is achieved by selecting the best strategy in mean over a potential MDP distribution from a large set of candidate strategies, which is done by exploiting single trajectories drawn from plenty of MDPs. In this paper, we mainly make the following contributions: (1) We discuss the strategy-selector algorithm based on formula set and polynomial function. (2) We provide the theoretical and experimental regret analysis of the learned strategy under an given MDP distribution. (3) We compare these methods with the 'state-of-the-art' Bayesian RL method experimentally. [ABSTRACT FROM AUTHOR]

This paper proposes a pattern-based prognostic methodology that combines logical analysis of data (LAD) as an event-driven diagnostic technique, and Kaplan-Meier (KM) estimator as a time-driven technique. LAD captures the effect of the instantaneous conditions on the health state of a monitored system, while KM estimates the baseline reliability curve that reflects the effect of aging, based on the observed historical failure times. LAD is used to generate a set of patterns from the observed values of covariates that represent the operating conditions and condition indicators. A pattern selection procedure is carried out to select the set of significant patterns from all the generated patterns. A survival curve is estimated, for each subset of observations covered by each selected pattern. A weight that reflects the coverage of each pattern is assigned to its survival curve. Given a recently collected observation, the survival curve of a monitored system is updated on the basis of the patterns covering that observation. The updated curve is then used to predict the remaining useful life of the monitored system. The proposed methodology is validated using a common dataset in prognostics: the turbofan degradation dataset that is available at NASA prognostic repository. Copyright © 2017 John Wiley & Sons, Ltd. [ABSTRACT FROM AUTHOR]

Bayesian networks have been at the core of the research pertaining to probabilistic reasoning. Several machine learning algorithms and techniques highly rely on Bayesian networks for their reasoning capabilities. Since several decades Bayesian networks proved to be the essential tool in the hands of researchers working in artificial intelligence domain. Yet Bayesian networks do have certain limitations [1], Bayesian networks need extensions to be more expressive and functional for probabilistic reasoning needs of various domains. Multi Entity Bayesian Networks (MEBN) is a theory combining expressivity of first-order logic principles and probabilistic reasoning of Bayesian networks. The paper thoroughly introduces the MEBN theory with an example modelling for a problem description and discusses various other works which included MEBN theory as a core of their research. The study in this paper concludes with the current highlights and challenges in MEBN and future developments. [ABSTRACT FROM AUTHOR]

Extreme learning machine (ELM) is an effective machine learning technique with simple theory and fast implementation, which has gained increasing interest from various research fields recently. A new method that combines ELM with probabilistic model method is proposed in this paper to classify the electroencephalography (EEG) signals in synchronous brain-computer interface (BCI) system. In the proposed method, the softmax function is used to convert the ELM output to classification probability. The Chernoff error bound, deduced from the Bayesian probabilistic model in the training process, is adopted as the weight to take the discriminant process. Since the proposed method makes use of the knowledge from all preceding training datasets, its discriminating performance improves accumulatively. In the test experiments based on the datasets from BCI competitions, the proposed method is compared with other classification methods, including the linear discriminant analysis, support vector machine, ELM and weighted probabilistic model methods. For comparison, the mutual information, classification accuracy and information transfer rate are considered as the evaluation indicators for these classifiers. The results demonstrate that our method shows competitive performance against other methods. [ABSTRACT FROM AUTHOR]

In the upcoming Internet-of-Things (IoT) era, the communication is often featured by massive connection, sporadic transmission, and small-sized data packets, which poses new requirements on the delay expectation and resource allocation efficiency of the Random Access (RA) mechanisms of the IoT communication stack. A grant-free non-orthogonal random access (NORA) system is considered in this paper, which could simultaneously reduce the access delay and support more Machine Type Communication (MTC) devices with limited resources. In order to address the joint user activity detection (UAD) and channel estimation (CE) problem in the grant-free NORA system, we propose a deep neural network-aided message passing-based block sparse Bayesian learning (DNN-MP-BSBL) algorithm. In the DNN-MP-BSBL algorithm, the iterative message passing process is transferred from a factor graph to a deep neural network (DNN). Weights are imposed on the messages in the DNN and trained to minimize the estimation error. It is shown that the trained weights could alleviate the convergence problem of the MP-BSBL algorithm, especially on crowded RA scenarios. Simulation results show that the proposed DNN-MP-BSBL algorithm could improve the UAD and CE accuracy with a smaller number of iterations, indicating its advantages for low-latency grant-free NORA systems. [ABSTRACT FROM AUTHOR]

As an important model of machine learning, Bayesian networks (BNs) have received a lot of attentions since they can be used for classification via probabilistic inference. However, since it is a complicated combination optimization problem, BN structure learning cannot be solved with classic convex optimization algorithms. Hence, evolutionary algorithms provide an alternative way to find a global solution to BN structure learning problem. In this paper, we improve the biased random-key genetic algorithm to solve the BN structure learning problem. Meanwhile, we apply a local optimization model as its decoder to improve the performance of the proposed algorithm. Finally, we conduct our experiments on nine benchmark networks and a real dataset of cross-site scripting (XSS) attack. Experimental results show that the proposed algorithm can obtain more accurate solutions than other state-of-the-art algorithms and achieve a good performance in XSS attack detection for web security. [ABSTRACT FROM AUTHOR]

Bayesian network (BN) structure learning is the basis of BN applications and plays a pivotal role in many machine learning tasks. Whereas remarkable progress in structure learning has been achieved in the past, making further improvements in the efficiency and accuracy of structure learning is a significant challenge. In this paper, we propose an efficient skeleton learning approach-based hybrid algorithm (ESLH), which consists of two phases. In the constraint-based phase, a dynamic threshold (DTH) strategy and a skeleton learning method based on triangle breaking (SLTB) are proposed to learn the skeleton of a BN structure efficiently. DTH designs a dynamic threshold to remove redundant edges in the initial skeleton with little time overhead. By the result of DTH, SLTB first finds, tests and breaks triangles in the initial skeleton to efficiently remove redundant edges and then removes the remaining redundant edges to discover the final skeleton. In the score-and-search phase, ESLH employs the hill-climbing algorithm to find the highest-scored structure. We propose a novel strategy to divide this phase into three steps, both utilizing the learned skeleton to constrain the search space and preventing the errors of the learned skeleton from reducing the quality of the final learned structure. Extensive experiments on benchmark BNs validate the effectiveness of DTH and SLTB and demonstrate that ESLH is more than five times faster than the state-of-the-art structure learning algorithms while maintaining the highest average accuracy. [ABSTRACT FROM AUTHOR]

Co-clustering is known to be a very powerful and efficient approach in unsupervised learning because of its ability to partition data based on both the observations and the variables of a given dataset. However, in high-dimensional context co-clustering methods may fail to provide a meaningful result due to the presence of noisy and/or irrelevant features. In this paper, we tackle this issue by proposing a novel co-clustering model which assumes the existence of a noise cluster, that contains all irrelevant features. A variational expectation-maximization-based algorithm is derived for this task, where the automatic variable selection as well as the joint clustering of objects and variables are achieved via a Bayesian framework. Experimental results on synthetic datasets show the efficiency of our model in the context of high-dimensional noisy data. Finally, we highlight the interest of the approach on two real datasets which goal is to study genetic diversity across the world. [ABSTRACT FROM AUTHOR]

In this paper, we present a formal model of human decision making in explore–exploit tasks using the context of multiarmed bandit problems, where the decision maker must choose among multiple options with uncertain rewards. We address the standard multiarmed bandit problem, the multiarmed bandit problem with transition costs, and the multiarmed bandit problem on graphs. We focus on the case of Gaussian rewards in a setting where the decision maker uses Bayesian inference to estimate the reward values. We model the decision maker's prior knowledge with the Bayesian prior on the mean reward. We develop the upper-credible-limit (UCL) algorithm for the standard multiarmed bandit problem and show that this deterministic algorithm achieves logarithmic cumulative expected regret, which is optimal performance for uninformative priors. We show how good priors and good assumptions on the correlation structure among arms can greatly enhance decision-making performance, even over short time horizons. We extend to the stochastic UCL algorithm and draw several connections to human decision-making behavior. We present empirical data from human experiments and show that human performance is efficiently captured by the stochastic UCL algorithm with appropriate parameters. For the multiarmed bandit problem with transition costs and the multiarmed bandit problem on graphs, we generalize the UCL algorithm to the block UCL algorithm and the graphical block UCL algorithm, respectively. We show that these algorithms also achieve logarithmic cumulative expected regret and require a sublogarithmic expected number of transitions among arms. We further illustrate the performance of these algorithms with numerical examples. [ABSTRACT FROM PUBLISHER]

The bias-variance dilemma is a well-known and important problem in Machine Learning. It basically relates the generalization capability (goodness of fit) of a learning method to its corresponding complexity. When we have enough data at hand, it is possible to use these data in such a way so as to minimize overfitting (the risk of selecting a complex model that generalizes poorly). Unfortunately, there are many situations where we simply do not have this required amount of data. Thus, we need to find methods capable of efficiently exploiting the available data while avoiding overfitting. Different metrics have been proposed to achieve this goal: the Minimum Description Length principle (MDL), Akaike’s Information Criterion (AIC) and Bayesian Information Criterion (BIC), among others. In this paper, we focus on crude MDL and empirically evaluate its performance in selecting models with a good balance between goodness of fit and complexity: the so-called bias-variance dilemma, decomposition or tradeoff. Although the graphical interaction between these dimensions (bias and variance) is ubiquitous in the Machine Learning literature, few works present experimental evidence to recover such interaction. In our experiments, we argue that the resulting graphs allow us to gain insights that are difficult to unveil otherwise: that crude MDL naturally selects balanced models in terms of bias-variance, which not necessarily need be the gold-standard ones. We carry out these experiments using a specific model: a Bayesian network. In spite of these motivating results, we also should not overlook three other components that may significantly affect the final model selection: the search procedure, the noise rate and the sample size. [ABSTRACT FROM AUTHOR]

Bayesian network classifiers (BNCs) have demonstrated competitive classification performance in a variety of real-world applications. A highly scalable BNC with high expressivity is extremely desirable. This paper proposes Redundant Dependence Elimination (RDE) for improving the classification performance and expressivity of k-dependence Bayesian classifier (KDB). To demonstrate the unique characteristics of each case, RDE identifies redundant conditional dependencies and then substitute/remove them. The learned personalized k-dependence Bayesian Classifier (PKDB) can achieve high-confidence conditional probabilities, and graphically interpret the dependency relationships between attributes. Two thyroid cancer datasets and four other cancer datasets from the UCI machine learning repository are selected for our experimental study. The experimental results prove the effectiveness of the proposed algorithm in terms of zero-one loss, bias, variance and AUC. [ABSTRACT FROM AUTHOR]

Factor analysis is broadly used as a powerful unsupervised machine learning tool for reconstruction of hidden features in recorded mixtures of signals. In the case of a linear approximation, the mixtures can be decomposed by a variety of model-free Blind Source Separation (BSS) algorithms. Most of the available BSS algorithms consider an instantaneous mixing of signals, while the case when the mixtures are linear combinations of signals with delays is less explored. Especially difficult is the case when the number of sources of the signals with delays is unknown and has to be determined from the data as well. To address this problem, in this paper, we present a new method based on Nonnegative Matrix Factorization (NMF) that is capable of identifying: (a) the unknown number of the sources, (b) the delays and speed of propagation of the signals, and (c) the locations of the sources. Our method can be used to decompose records of mixtures of signals with delays emitted by an unknown number of sources in a nondispersive medium, based only on recorded data. This is the case, for example, when electromagnetic signals from multiple antennas are received asynchronously; or mixtures of acoustic or seismic signals recorded by sensors located at different positions; or when a shift in frequency is induced by the Doppler effect. By applying our method to synthetic datasets, we demonstrate its ability to identify the unknown number of sources as well as the waveforms, the delays, and the strengths of the signals. Using Bayesian analysis, we also evaluate estimation uncertainties and identify the region of likelihood where the positions of the sources can be found. [ABSTRACT FROM AUTHOR]

We give an overview of the basic principles of approximate Bayesian computation (ABC), a class of stochastic methods that enable flexible and likelihood-free model comparison and parameter estimation. Our new open-source software called ABrox is used to illustrate ABC for model comparison on two prominent statistical tests, the two-sample t-test and the Levene-Test. We further highlight the flexibility of ABC compared to classical Bayesian hypothesis testing by computing an approximate Bayes factor for two multinomial processing tree models. Last but not least, throughout the paper, we introduce ABrox using the accompanied graphical user interface. [ABSTRACT FROM AUTHOR]

Conventional direction-of-arrival (DOA) estimation methods are sensitive to outlier measurements. Therefore, their performance may degrade substantially in the presence of impulsive noise. In this paper, we address the problem of DOA estimation in additive outliers from the perspective of sparse Bayesian learning (SBL). A Bayes-optimal algorithm is devised for robust DOA estimation, which can achieve excellent performance in terms of resolution and accuracy. To reduce the computational complexity of the SBL scheme, a fast alternating algorithm is also developed. New grid-refining procedures are further introduced into these two proposed algorithms to efficiently fix the off-grid gap. As our solutions do not require the prior knowledge of the number of sources and can resolve highly correlated or coherent sources, it is expected that they have higher applicability. Simulation results verify the outlier-robust performance of the SBL approach. [ABSTRACT FROM AUTHOR]

We conduct a large-scale comparative study on linearly combining superparent-one-dependence estimators (SPODEs), a popular family of seminaive Bayesian classifiers. Altogether, 16 model selection and weighing schemes, 58 benchmark data sets, and various statistical tests are employed. This paper's main contributions are threefold. First, it formally presents each scheme's definition, rationale, and time complexity and hence can serve as a comprehensive reference for researchers interested in ensemble learning. Second, it offers bias-variance analysis for each scheme's classification error performance. Third, it identifies effective schemes that meet various needs in practice. This leads to accurate and fast classification algorithms which have an immediate and significant impact on real-world applications. Another important feature of our study is using a variety of statistical tests to evaluate multiple learning methods across multiple data sets. [ABSTRACT FROM AUTHOR]

In the area of cost-sensitive learning, inductive learning algorithms have been extended to handle different types of costs to better represent misclassification errors. Most of the previous works have only focused on how to deal with misclassification costs. In this paper, we address the equally important issue of how to handle the test costs associated with querying the missing values in a test case. When an attribute contains a missing value in a test case, it may or may not be worthwhile to take the extra effort in order to obtain a value for that attribute, or attributes, depending on how much benefit the new value will bring about in increasing the accuracy. In this paper, we consider how to integrate test-cost-sensitive learning with the handling of missing values in a unified framework that includes model building and a testing strategy. The testing strategies determine which attributes to perform the test on in order to minimize the sum of the classification costs and test costs. We show how to instantiate this framework in two popular machine learning algorithms: decision trees and naive Bayesian method. We empirically evaluate the test-cost-sensitive methods for handling missing values on several data sets. [ABSTRACT FROM AUTHOR]

Naïve Bayes is one of the most efficient and effective inductive learning algorithms for machine learning and data mining. Its competitive performance in classification is surprising, because the conditional independence assumption on which it is based is rarely true in real-world applications. An open question is: what is the true reason for the surprisingly good performance of Naïve Bayes in classification? In this paper, we propose a novel explanation for the good classification performance of Naïve Bayes. We show that, essentially, dependence distribution plays a crucial role. Here dependence distribution means how the local dependence of an attribute distributes in each class, evenly or unevenly, and how the local dependences of all attributes work together, consistently (supporting a certain classification) or inconsistently (canceling each other out). Specifically, we show that no matter how strong the dependences among attributes are, Naïve Bayes can still be optimal if the dependences distribute evenly in classes, or if the dependences cancel each other out. We propose and prove a sufficient and necessary condition for the optimality of Naïve Bayes. Further, we investigate the optimality of Naïve Bayes under the Gaussian distribution. We present and prove a sufficient condition for the optimality of Naïve Bayes, in which the dependences among attributes exist. This provides evidence that dependences may cancel each other out. Our theoretic analysis can be used in designing learning algorithms. In fact, a major class of learning algorithms for Bayesian networks are conditional independence-based (or CI-based), which are essentially based on dependence. We design a dependence distribution-based algorithm by extending the ChowLiu algorithm, a widely used CI based algorithm. Our experiments show that the new algorithm outperforms the ChowLiu algorithm, which also provides empirical evidence to support our new explanation. [ABSTRACT FROM AUTHOR]

This paper describes models for detecting individual and team ball possession in soccer based on position data. The types of ball possession are classified as Individual Ball Possession (IBC), Individual Ball Action (IBA), Individual Ball Control (IBC), Team Ball Possession (TBP), Team Ball Control (TBC) und Team Playmaking (TPM) according to different starting points and endpoints and the type of ball control involved. The machine learning approach used is able to determine how long the ball spends in the sphere of influence of a player based on the distance between the players and the ball together with their direction of motion, speed and the acceleration of the ball. The degree of ball control exhibited during this phase is classified based on the spatio-temporal configuration of the player controlling the ball, the ball itself and opposing players using a Bayesian network. The evaluation and application of this approach uses data from 60 matches in the German Bundesliga season of 2013/14, including 69,667 IBA intervals. The identification rate was F = .88 for IBA and F = .83 for IBP, and the classification rate for IBC was κ = .67. Match analysis showed the following mean values per match: TBP 56:04 ± 5:12 min, TPM 50:01 ± 7:05 min and TBC 17:49 ± 8:13 min. There were 836 ± 424 IBC intervals per match and their number was significantly reduced by -5.1% from the 1st to 2nd half. The analysis of ball possession at the player level indicates shortest accumulated IBC times for the central forwards (0:49 ± 0:43 min) and the longest for goalkeepers (1:38 ± 0:58 min), central defenders (1:38 ± 1:09 min) and central midfielders (1:27 ± 1:08 min). The results could improve performance analysis in soccer, help to detect match events automatically, and allow discernment of higher value tactical structures, which is based on individual ball possession. [ABSTRACT FROM AUTHOR]

Abstract: An unsupervised image segmentation method for natural images is proposed in this paper. We assume that texture features in natural images are distributed as a mixture of Gaussians. In order to cluster the extracted feature vectors, we modify a clustering algorithm based on Bayesian Ying–Yang (BYY) harmony learning theory with Dirichlet-Normal-Wishart prior. This algorithm can determine the number of components automatically during the clustering procedure, as long as we give a large enough initial component number. Our works in this paper have presented a complete pipeline of clustering-based image segmentation including feature extraction, robust feature clustering and methodological effective post processing. The experiments reported in this paper demonstrate that the proposed method is efficient (in terms of visual evaluation and quantitative performance measures) and performs competitively compared to the existing state-of-the-art segmentation methods. [Copyright &y& Elsevier]

Due to the infeasibility of randomized controlled experiments, the existence of unobserved variables and the fact that equivalent direct acyclic graphs obtained generally can not be distinguished, it is difficult to learn the true causal relations of original graph. This paper presents an algorithm called BSPC based on adjacent nodes to learn the structure of Causal Bayesian Networks with unobserved variables by using observational data. It does not have to adjust the structure as the existing algorithms FCI and MBCS*, while it can guarantee to obtain the true adjacent nodes. More important is that algorithm BSPC reduces computational complexity and improves reliability of conditional independence tests. Theoretical results show that the new algorithm is correct. In addition, the advantages of BSPC in terms of the number of conditional independence tests and the number of orientation errors are illustrated with simulation experiments from which we can see that it is more suitable in order to learn the structure of Causal Bayesian Networks with latent variables. Moreover a better latent structure representation is returned. [ABSTRACT FROM AUTHOR]

Interval data are widely used in real applications to represent the values of quantities in uncertain situations. However, the implied probabilistic causal relationships among interval-valued variables with interval data cannot be represented and inferred by general Bayesian networks with point-based probability parameters. Thus, it is desired to extend the general Bayesian network with effective mechanisms of representation, learning and inference of probabilistic causal relationships implied in interval data. In this paper, we define the interval probabilities, the bound-limited weak conditional interval probabilities and the probabilistic description, as well as the multiplication rules. Furthermore, we propose the method for learning the Bayesian network structure from interval data and the algorithm for corresponding approximate inferences. Experimental results show that our methods are feasible, and we conclude that the Bayesian network with interval probability parameters is the expansion of the general Bayesian network. [ABSTRACT FROM AUTHOR]

Within the framework of Bayesian networks (BNs), most classifiers assume that the variables involved are of a discrete nature, but this assumption rarely holds in real problems. Despite the loss of information discretization entails, it is a direct easy-to-use mechanism that can offer some benefits: sometimes discretization improves the run time for certain algorithms; it provides a reduction in the value set and then a reduction in the noise which might be present in the data; in other cases, there are some Bayesian methods that can only deal with discrete variables. Hence, even though there are many ways to deal with continuous variables other than discretization, it is still commonly used. This paper presents a study of the impact of using different discretization strategies on a set of representative BN classifiers, with a significant sample consisting of 26 datasets. For this comparison, we have chosen Naive Bayes (NB) together with several other semi-Naive Bayes classifiers: Tree-Augmented Naive Bayes (TAN), k-Dependence Bayesian (KDB), Aggregating One-Dependence Estimators (AODE) and Hybrid AODE (HAODE). Also, we have included an augmented Bayesian network created by using a hill climbing algorithm (BNHC). With this comparison we analyse to what extent the type of discretization method affects classifier performance in terms of accuracy and bias-variance discretization. Our main conclusion is that even if a discretization method produces different results for a particular dataset, it does not really have an effect when classifiers are being compared. That is, given a set of datasets, accuracy values might vary but the classifier ranking is generally maintained. This is a very useful outcome, assuming that the type of discretization applied is not decisive future experiments can be d times faster, d being the number of discretization methods considered. [ABSTRACT FROM AUTHOR]

The finite mixture model is widely used in various statistical learning problems. However, the model obtained may contain a large number of components, making it inefficient in practical applications. In this paper, we propose to simplify the mixture model by minimizing an upper bound of the approximation error between the original and the simplified model, under the use of the L2 distance measure. This is achieved by first grouping similar components together and then performing local fitting through function approximation. The simplified model obtained can then be used as a replacement of the original model to speed up various algorithms involving mixture models during training (e.g., Bayesian filtering, belief propagation) and testing [e.g., kernel density estimation, support vector machine (SVM) testing]. Encouraging results are observed in the experiments on density estimation, clustering-based image segmentation, and simplification of SVM decision functions. [ABSTRACT FROM AUTHOR]

Cardiovascular diseases constitute one of the main causes of mortality in the world, and machine learning has become a powerful tool for analysing medical data in the last few years. In this paper we present an interdisciplinary work based on an ambulatory blood pressure study and the development of a new classification algorithm named REMED. We focused on the discovery of new patterns for abnormal blood pressure variability as a possible cardiovascular risk factor. We compared our results with other classification algorithms based on Bayesian methods, decision trees, and rule induction techniques. In the comparison, REMED showed similar accuracy to these algorithms but it has the advantage of being superior in its capacity to classify sick people correctly. Therefore, our method could represent an innovative approach that might be useful in medical decision support for cardiovascular disease prognosis. [ABSTRACT FROM AUTHOR]

This article proposes a methodology for the analysis of the causes and types of workplace accidents (in this paper we focus specifically on floor-level falls). The approach is based on machine learning techniques: Bayesian networks trained using different algorithms (with and without a priori information), classification trees, support vector machines and extreme learning machines. The results obtained using the different techniques are compared in terms of explanatory capacity and predictive potential, both factors facilitating the development of risk prevention measures. Bayesian networks are revealed to be the best all-round technique for this type of study, as they combine a powerful interpretative capacity with a predictive capacity that is comparable to that of the best available techniques. Moreover, the Bayesian networks force experts to apply a scientific approach to the construction and progressive enrichment of their models and also enable the basis to be laid for an accident prevention policy that is solidly grounded. Furthermore, the procedure enables better variable definition, better structuring of the data capture, coding, and quality control processes. [ABSTRACT FROM AUTHOR]

In this paper, we present an incremental method for model selection and learning of Gaussian mixtures based on the recently proposed variational Bayes approach. The method adds components to the mixture using a Bayesian splitting test procedure: a component is split into two components and then variational update equations are applied only to the parameters of the two components. As a result, either both components are retained in the model or one of them is found to be redundant and is eliminated from the model. In our approach, the model selection problem is treated locally, in a region of the data space, so we can set more informative priors based on the local data distribution. A modified Bayesian mixture model is presented to implement this approach, along with a learning algorithm that iteratively applies a splitting test on each mixture component. Experimental results and comparisons with two other techniques testify for the adequacy of the proposed approach. [ABSTRACT FROM AUTHOR]

Supervised classification is one of the tasks most frequently carried out by so-called Intelligent Systems. Thus, a large number of techniques have been developed based on Artificial Intelligence (Logic-based techniques, Perceptron-based techniques) and Statistics (Bayesian Networks, Instance-based techniques). The goal of supervised learning is to build a concise model of the distribution of class labels in terms of predictor features. The resulting classifier is then used to assign class labels to the testing instances where the values of the predictor features are known, but the value of the class label is unknown. This paper describes various classification algorithms and the recent attempt for improving classification accuracy--ensembles of classifiers. [ABSTRACT FROM AUTHOR]

During the last years, development of open learning environments that support effectively their users has been a challenge for the research community of educational technologies. The open interactive nature of these environments results in users experiencing difficulties in coping with the plethora of available functions, especially during their initial efforts to use the system. In addition, -from the tutors' perspective- the problem solving strategies of the students are often particularly difficult to identify. In this paper, we argue that such problems could be tackled using machine learning techniques such as Bayesian Networks. We show how we can take advantage of log files obtained during field studies to build an adaptive help system providing the most useful support to the student, according to the state of interaction. On the other hand, we attempt to support the tutor, by automating the process of diagnosing students' problem solving strategies using Bayesian Networks. The presented approaches are discussed through examples of two prototypes that have been developed and corresponding evaluation studies. These studies have shown that the proposed approach can effectively support the tasks of students and tutors in such open learning environments. [ABSTRACT FROM AUTHOR]

With the continuous development of new technologies, the scale of training data is also expanding. Machine learning algorithms are gradually beginning to be studied and applied in places where the scale of data is relatively large. Because the current structure of learning algorithms only focus on the identification of dependencies and ignores the direction of dependencies, it causes multiple labeled samples not to identify categories. Multiple labels need to be classified using techniques such as machine learning and then applied to solve the problem. In the environment of more training data, it is very meaningful to explore the structure extension to identify the dependencies between attributes and take into account the direction of dependencies. In this article, Bayesian network structure learning, analysis of the shortcomings of traditional algorithms, and binary evolutionary algorithm are applied to the randomized algorithm to generate the initial population. In the optimization process of the algorithm, it uses a Bayesian network to do a local search and uses a depth-first algorithm to break the loop. Finally, it finds a higher score for the network structure. In the simulation experiment, the classic data sets, ALARM and INSURANCE, are introduced to verify the effectiveness of the algorithm. Compared with NOTEARS and the Expectation-Maximization (EM) algorithm, the weight evaluation index of this article was 4.5% and 7.3% better than other schemes. The clustering effect was improved by 13.5% and 15.2%. The smallest error and the highest accuracy are also better than other schemes. The discussion of Bayesian reasoning in this article has very important theoretical and practical significance. This article further improves the Bayesian network structure and optimizes the performance of the classifier, which plays a very important role in promoting the expansion of the network structure and provides innovative thinking. [ABSTRACT FROM AUTHOR]

A discrete network design problem (DNDP) is conventionally formulated as an analytical bi-level programming problem to acquire an optimal network design strategy for an existing traffic network. In recent years, multimodal network design problems have benefited from simulation-based models. The nonconvexity and implicity of bi-level DNDPs make it challenging to obtain an optimal solution, especially for simulation-related models. Bayesian optimization (BO) has been proven to be an effective method for optimizing the costly black-box functions of simulation-based continuous network design problems. However, there are only discrete inputs in DNDPs, which cannot be processed using standard BO algorithms. To address this issue, we develop a hybrid method (BO-B&B) that combines Bayesian optimization and a branch-and-bound algorithm to deal with discrete variables. The proposed algorithm exploits the advantages of the cutting-edge machine-learning parameter-tuning technique and the exact mathematical optimization method, thereby balancing efficiency and accuracy. Our experimental results show that the proposed method outperforms benchmarking discrete optimization heuristics for simulation-based DNDPs in terms of total computational time. Thus, BO-B&B can potentially aid decision makers in mapping practical network design schemes for large-scale networks. [ABSTRACT FROM AUTHOR]

Decomposition hybrid algorithms with the recursive framework which recursively decompose the structural task into structural subtasks to reduce computational complexity are employed to learn Bayesian network (BN) structure. Merging rules are commonly adopted as the combination method in the combination step. The direction determination rule of merging rules has problems in using the idea of keeping v-structures unchanged before and after combination to determine directions of edges in the whole structure. It breaks down in one case due to appearances of wrong v-structures, and is hard to operate in practice. Therefore, we adopt a novel approach for direction determination and propose a two-stage combination method. In the first-stage combination method, we determine nodes, links of edges by merging rules and adopt the idea of permutation and combination to determine directions of contradictory edges. In the second-stage combination method, we restrict edges between nodes that do not satisfy the decomposition property and their parent nodes by determining the target domain according to the decomposition property. Simulation experiments on four networks show that the proposed algorithm can obtain BN structure with higher accuracy compared with other algorithms. Finally, the proposed algorithm is applied to the thickening process of gold hydrometallurgy to solve the practical problem. [ABSTRACT FROM AUTHOR]

This paper deals with the wideband direction-of-arrival (DOA) estimation by exploiting the multiple measurement vectors (MMV) based sparse Bayesian learning (SBL) framework. First, the array covariance matrices at different frequency bins are focused to the reference frequency by the conventional focusing technique and then transformed into the vector form. Then a matrix called the Khatri-Rao dictionary is constructed by using the Khatri-Rao product and the multiple focused array covariance vectors are set as the new observations. DOA estimation is to find the sparsest representations of the new observations over the Khatri-Rao dictionary via SBL. The performance of the proposed method is compared with other well-known focusing based wideband algorithms and the Cramer-Rao lower bound (CRLB). The results show that it achieves higher resolution and accuracy and can reach the CRLB under relative demanding conditions. Moreover, the method imposes no restriction on the pattern of signal power spectral density and due to the increased number of rows of the dictionary, it can resolve more sources than sensors. [ABSTRACT FROM AUTHOR]

Bayesian Ying–Yang (BYY) harmony learning system is a powerful tool for statistical learning. Via the BYY harmony leaning of finite mixtures, model selection, i.e., the selection of an appropriate number of components for the mixture, can be made automatically during parameter learning on a given dataset. In this paper, an adaptive gradient BYY harmony learning algorithm is proposed for log-normal mixtures to implement parameter learning with automated model selection. It is demonstrated by the experiments on both synthetic and real-world datasets that the proposed BYY harmony learning algorithm not only has the ability of automated model selection, but also leads to a rather good estimation of the parameters in the original log-normal mixture. [ABSTRACT FROM AUTHOR]

Abstract: The next big thing in computing is the Internet of Things (IoT), referring to dynamic and ever-evolving environments, generating high-volume streams of heterogeneous yet correlated contextual information of varying quality and complexity. Moreover, the increase in user mobility and unreliable sensor availability in IoT, necessitates the context-aware applications to dynamically adapt their behavior at run time. In this paper, we elicit the need for different self-learning techniques to tackle the openness of the IoT environments and propose enabling algorithms to achieve them. First, we present possible application scenarios which can benefit from both supervised and unsupervised self-learning. Later, we propose correlation mining algorithms based on Kullback-Leibler (KL) divergence and frequent set mining that exploits correlated contexts to enable unsupervised self-learning. These algorithms help to identify alternate sources for a context and semantically describe the previously unseen contexts in terms of already known contexts. We have realized the proposed algorithms on top of a Bayesian framework (HARD-BN) which supports autonomous learning. Our experiments demonstrate the applicability of the proposed correlation mining algorithms and their feasibility to enable self-learning in open and ever-evolving IoT environments. [Copyright &y& Elsevier]

This paper proposes the Bayesian Extreme Learning Machine Kohonen Network (BELMKN) framework to solve the clustering problem. The BELMKN framework uses three levels in processing nonlinearly separable datasets to obtain efficient clustering in terms of accuracy. In the first level, the Extreme Learning Machine (ELM)-based feature learning approach captures the nonlinearity in the data distribution by mapping it onto a d-dimensional space. In the second level, ELM-based feature extracted data is used as an input for Bayesian Information Criterion (BIC) to predict the number of clusters termed as a cluster prediction. In the final level, feature-extracted data along with the cluster prediction is passed to the Kohonen Network to obtain improved clustering accuracy. The main advantage of the proposed method is to overcome the problem of having a priori identifiers or class labels for the data; it is difficult to obtain labels in most of the cases for the real world datasets. The BELMKN framework is applied to 3 synthetic datasets and 10 benchmark datasets from the UCI machine learning repository and compared with the state-of-the-art clustering methods. The experimental results show that the proposed BELMKN-based clustering outperforms other clustering algorithms for the majority of the datasets. Hence, the BELMKN framework can be used to improve the clustering accuracy of the nonlinearly separable datasets. [ABSTRACT FROM AUTHOR]

Abstract: Resting state functional connectivity MRI (rs-fcMRI) is a popular technique used to gauge the functional relatedness between regions in the brain for typical and special populations. Most of the work to date determines this relationship by using Pearson's correlation on BOLD fMRI timeseries. However, it has been recognized that there are at least two key limitations to this method. First, it is not possible to resolve the direct and indirect connections/influences. Second, the direction of information flow between the regions cannot be differentiated. In the current paper, we follow-up on recent work by Smith et al. (2011), and apply PC algorithm to both simulated data and empirical data to determine whether these two factors can be discerned with group average, as opposed to single subject, functional connectivity data. When applied on simulated individual subjects, the algorithm performs well determining indirect and direct connection but fails in determining directionality. However, when applied at group level, PC algorithm gives strong results for both indirect and direct connections and the direction of information flow. Applying the algorithm on empirical data, using a diffusion-weighted imaging (DWI) structural connectivity matrix as the baseline, the PC algorithm outperformed the direct correlations. We conclude that, under certain conditions, the PC algorithm leads to an improved estimate of brain network structure compared to the traditional connectivity analysis based on correlations. [Copyright &y& Elsevier]

Abstract: In this paper, we introduce a subspace-augmented MUSIC technique for recovering the joint sparse support of a signal ensemble corrupted by additive impulsive noise. Our approach uses multiple vectors of random compressed measurements and employs fractional lower-order moments stemming from modeling the underlying signal statistics with symmetric alpha-stable distributions. We show through simulations that the recovery performance of the proposed method is particularly robust for a wide range of highly impulsive environments. Our subspace-augmented MUSIC achieves higher recovery rates than a recently introduced sparse Bayesian learning algorithm, which was shown to outperform many state-of-the-art techniques for joint sparse support recovery. [Copyright &y& Elsevier]

Abstract: Causal structure learning algorithms construct Bayesian networks from observational data. Using non-interventional data, existing constraint-based algorithms may return I-equivalent partially directed acyclic graphs. However, these algorithms do not fully exploit the graphical properties of Bayesian networks, and require many redundant tests that reduce both speed and accuracy. In this paper, we introduce ideas to exploit such properties to increase the speed and accuracy of causal structure learning for multivariate normal data. In numerical experiments on five benchmarking networks our proposed algorithm was faster and more accurate than recently-developed algorithms. [Copyright &y& Elsevier]

Co-training (or more generally, co-regularization) has been a popular algorithm for semi-supervised learning in data with two feature representations (or views), but the fundamental assumptions underlying this type of models are still unclear. In this paper we propose a Bayesian undirected graphical model for co-training, or more generally for semi-supervised multi-view learning. This makes explicit the previously unstated assumptions of a large class of co-training type algorithms, and also clarifies the circumstances under which these assumptions fail. Building upon new insights from this model, we propose an improved method for co-training, which is a novel co-training kernel for Gaussian process classifiers. The resulting approach is convex and avoids local-maxima problems, and it can also automatically estimate how much each view should be trusted to accommodate noisy or unreliable views. The Bayesian co-training approach can also elegantly handle data samples with missing views, that is, some of the views are not available for some data points at learning time. This is further extended to an active sensing framework, in which the missing (sample, view) pairs are actively acquired to improve learning performance. The strength of active sensing model is that one actively sensed (sample, view) pair would improve the joint multi-view classification on all the samples. Experiments on toy data and several real world data sets illustrate the benefits of this approach. [ABSTRACT FROM AUTHOR]

Editor's note: [ABSTRACT FROM PUBLISHER]

Abstract: In this paper, we consider the problem of generating concise but accurate naive Bayes classifiers using taxonomy of propositionalized attributes. For the problem, we introduce propositionalized attribute taxonomy guided naive Bayes Learner (PAT-NBL), a machine learning algorithm that effectively utilizes taxonomy to generate compact classifiers. We extend classical naive Bayes learner to the PAT-NBL algorithm that traverses over a propositionalized taxonomy to search for a locally optimal cut. PAT-NBL uses bottom-up search to find the locally optimal cut on a given taxonomy. For the evaluation of candidate cuts, we apply conditional log-likelihood, conditional minimum description length, and conditional Akaike information criterion. The detected cut enables PAT-NBL to construct an instance space which corresponds to the taxonomy and the data. That is, after PAT-NBL determines a cut according to its information-theoretic criteria, the algorithm generates a concise naive Bayes classifier based on the cut. Our experimental results on UCI Machine Learning benchmark data sets indicate that the proposed algorithm can generate naive Bayes classifiers that are more compact and often comparably accurate to those produced by standard naive Bayes learners. [Copyright &y& Elsevier]

Co-training (or more generally, co-regularization) has been a popular algorithm for semi-supervised learning in data with two feature representations (or views), but the fundamental assumptions underlying this type of models are still unclear. In this paper we propose a Bayesian undirected graphical model for co-training, or more generally for semi-supervised multi-view learning. This makes explicit the previously unstated assumptions of a large class of co-training type algorithms, and also clarifies the circumstances under which these assumptions fail. Building upon new insights from this model, we propose an improved method for co-training, which is a novel co-training kernel for Gaussian process classifiers. The resulting approach is convex and avoids local-maxima problems, and it can also automatically estimate how much each view should be trusted to accommodate noisy or unreliable views. The Bayesian co-training approach can also elegantly handle data samples with missing views, that is, some of the views are not available for some data points at learning time. This is further extended to an active sensing framework, in which the missing (sample, view) pairs are actively acquired to improve learning performance. The strength of active sensing model is that one actively sensed (sample, view) pair would improve the joint multi-view classification on all the samples. Experiments on toy data and several real world data sets illustrate the benefits of this approach. [ABSTRACT FROM AUTHOR]

Abstract: During the early stages of language acquisition, young infants face the task of learning a basic vocabulary without the aid of prior linguistic knowledge. Attempts have been made to model this complex behaviour computationally, using a variety of machine learning algorithms, a.o. non-negative matrix factorization (NMF). In this paper, we replace NMF in a vocabulary learning setting with a conceptually similar algorithm, probabilistic latent semantic analysis (PLSA), which can learn word representations incrementally by Bayesian updating. We further show that this learning framework is capable of modelling certain cognitive behaviours, e.g. forgetting, in a simple way. [Copyright &y& Elsevier]

Abstract: Feature selection is an important preprocessing step for building efficient, generalizable and interpretable classifiers on high dimensional data sets. Given the assumption on the sufficient labelled samples, the Markov Blanket provides a complete and sound solution to the selection of optimal features, by exploring the conditional independence relationships among the features. In real-world applications, unfortunately, it is usually easy to get unlabelled samples, but expensive to obtain the corresponding accurate labels on the samples. This leads to the potential waste of valuable classification information buried in unlabelled samples. In this paper, we propose a new BAyesian Semi-SUpervised Method, or BASSUM in short, to exploit the values of unlabelled samples on classification feature selection problem. Generally speaking, the inclusion of unlabelled samples helps the feature selection algorithm on (1) pinpointing more specific conditional independence tests involving fewer variable features and (2) improving the robustness of individual conditional independence tests with additional statistical information. Our experimental results show that BASSUM enhances the efficiency of traditional feature selection methods and overcomes the difficulties on redundant features in existing semi-supervised solutions. [Copyright &y& Elsevier]

This paper addresses the problem of learning Bayesian network structures from data based on score functions that are decomposable. It describes properties that strongly reduce the time and memory costs of many known methods without losing global optimality guarantees. These properties are derived for different score criteria such as Minimum Description Length (or Bayesian Information Criterion), Akaike Information Criterion and Bayesian Dirichlet Criterion. Then a branch-and-bound algorithm is presented that integrates structural constraints with data in a way to guarantee global optimality. As an example, structural constraints are used to map the problem of structure learning in Dynamic Bayesian networks into a corresponding augmented Bayesian network. Finally, we show empirically the benefits of using the properties with state-of-the-art methods and with the new algorithm, which is able to handle larger data sets than before. [ABSTRACT FROM AUTHOR]

Abstract: The inclusion of irrelevant, redundant, and inconsistent features in the data-mining model results in poor predictions and high computational overhead. This paper proposes a novel information theoretic-based interact (IT-IN) algorithm, which concerns the relevance, redundancy, and consistency of the features. The proposed IT-IN algorithm is compared with existing Interact, FCBF, Relief and CFS feature selection algorithms. To evaluate the classification accuracy of IT-IN and remaining four feature selection algorithms, Naïve Bayes, SVM, and ELM classifier are used for ten UCI repository datasets. The proposed IT-IN performs better than existing above algorithms in terms of number of features. The specially designed hash function is used to speed up the IT-IN algorithms and provides minimum computation time than the Interact algorithms. The result clearly reveals that the proposed feature selection algorithm improves the classification accuracy for ELM, Naïve Bayes, and SVM classifiers. The performance of proposed IT-IN with ELM classifier is superior to other classifiers. [ABSTRACT FROM AUTHOR]