Search

In decision support systems the motivation and justification of the system's diagnosis or classification is crucial for the acceptance of the system by the human user. In Bayesian networks a diagnosis or classification is typically formalized as the computation of the most probable joint value assignment to the hypothesis variables, given the observed values of the evidence variables (generally known as the MAP problem). While solving the MAP problem gives the most probable explanation of the evidence, the computation is a black box as far as the human user is concerned and it does not give additional insights that allow the user to appreciate and accept the decision. For example, a user might want to know to whether an unobserved variable could potentially (upon observation) impact the explanation, or whether it is irrelevant in this aspect. In this paper we introduce a new concept, MAP-independence, which tries to capture this notion of relevance, and explore its role towards a potential justification of an inference to the best explanation. We formalize several computational problems based on this concept and assess their computational complexity. [ABSTRACT FROM AUTHOR]

Learning the structure of a Bayesian Network (BN) with score-based solutions involves exploring the search space of possible graphs and moving towards the graph that maximises a given objective function. Some algorithms offer exact solutions that guarantee to return the graph with the highest objective score, while others offer approximate solutions in exchange for reduced computational complexity. This paper describes an approximate BN structure learning algorithm, which we call Model Averaging Hill-Climbing (MAHC), that combines two novel strategies with hill-climbing search. The algorithm starts by pruning the search space of graphs, where the pruning strategy can be viewed as an aggressive version of the pruning strategies that are typically applied to combinatorial optimisation structure learning problems. It then performs model averaging in the hill-climbing search process and moves to the neighbouring graph that maximises the objective function, on average, for that neighbouring graph and over all its valid neighbouring graphs. Comparisons with other algorithms spanning different classes of learning suggest that the combination of aggressive pruning with model averaging is both effective and efficient, particularly in the presence of data noise. [ABSTRACT FROM AUTHOR]

In this paper we propose a Bayesian approach for inferring Bayesian network (BN) structures from ordinal data. Our approach can be seen as the Bayesian counterpart of a recently proposed frequentist approach, referred to as the 'ordinal structure expectation maximization' (OSEM) method. Like for the OSEM method, the key idea is to assume that each ordinal variable originates from a Gaussian variable that can only be observed in discretized form, and that the dependencies in the latent Gaussian space can be modeled by BNs; i.e. by directed acyclic graphs (DAGs). Our Bayesian method combines the 'structure MCMC sampler' for DAG posterior sampling, a slightly modified version of the 'Bayesian metric for Gaussian networks having score equivalence' (BGe score), the concept of the 'extended rank likelihood', and a recently proposed algorithm for posterior sampling the parameters of Gaussian BNs. In simulation studies we compare the new Bayesian approach and the OSEM method in terms of the network reconstruction accuracy. The empirical results show that the new Bayesian approach leads to significantly improved network reconstruction accuracies. [ABSTRACT FROM AUTHOR]

Subject-specific causal discovery methods could help to make better decisions in domains where there are variations on causal relations across subjects, such as neuroscience and genetics. However, discovering subject-specific causal models from limited data sets could be challenging. Although there are several causal discovery methods, most of them are focused on finding the common causal relations of a population from data sets with enough samples. The issue of discovering subject-specific causal relations from limited data sets has not been sufficiently explored. In this paper, we propose a knowledge transfer method for discovering subject-specific causal models. Our method discovers causal probabilistic graphical models, up to Markov equivalence classes, from discrete and continuous data sets. We hypothesized that transferring weighted instances of additional sources according to their discrepancy with target instances helps to compensate for the lack of target data and improves the performance of a scored-based causal discovery method. Experimental results on synthetic data sets and benchmark causal Bayesian networks show that our proposal, under few target samples, outperforms significantly in adjacency and arrowhead recovery the baseline and alternative methods. [ABSTRACT FROM AUTHOR]

Score functions for learning the structure of Bayesian networks in the literature assume that data are a homogeneous set of observations; whereas it is often the case that they comprise different related, but not homogeneous, data sets collected in different ways. In this paper we propose a new Bayesian Dirichlet score, which we call Bayesian Hierarchical Dirichlet (BHD). The proposed score is based on a hierarchical model that pools information across data sets to learn a single encompassing network structure, while taking into account the differences in their probabilistic structures. We derive a closed-form expression for BHD using a variational approximation of the marginal likelihood, we study the associated computational cost and we evaluate its performance using simulated data. We find that, when data comprise multiple related data sets, BHD outperforms the Bayesian Dirichlet equivalent uniform (BDeu) score in terms of reconstruction accuracy as measured by the Structural Hamming distance, and that it is as accurate as BDeu when data are homogeneous. This improvement is particularly clear when either the number of variables in the network or the number of observations is large. Moreover, the estimated networks are sparser and therefore more interpretable than those obtained with BDeu thanks to a lower number of false positive arcs. [ABSTRACT FROM AUTHOR]

The label ranking problem consists in learning preference models from training datasets labeled with (possibly incomplete) rankings of the class labels. The goal is then to predict a ranking for a given unlabeled instance. This work focuses on a more general interpretation where both the training dataset and the prediction given as output allow tied class labels, i.e., there is no particular preference between them. This problem is known as the partial label ranking problem. This paper tackles the partial label ranking problem by transforming the ranking with ties into a set of discrete variables representing the preference relations (ranked ahead , tied with , and ranked behind) between each pair of class labels. The posterior probabilities for each pair are then used to fill the values of a preference matrix. This preference matrix is the basis for solving the rank aggregation problem required to obtain the output ranking with ties. This paper aims to exploit the resemblance of this problem with multi-label and multi-dimensional classification by studying the use of Bayesian network classifiers to compute the posterior probabilities for the new class structure, i.e., pairs of class labels. In particular, binary relevance with naive Bayes and averaged one-dependence estimators between the new class structure are used to solve the partial label ranking problem. Furthermore, bivariate relationships between all the pairs of class labels are considered. However, the complexity of the model grows significantly, which makes it necessary to reduce the number of allowed bivariate relationships between pairs. Thus, a feature selection method is included to select the more relevant subset of bivariate relationships. The experimental evaluation shows that our proposals are competitive in accuracy with the current instance-based and decision tree induction algorithms. Moreover, they outperform the existing mixture-based probabilistic graphical models, while the algorithms proposed are much faster. [ABSTRACT FROM AUTHOR]

Variational autoencoders and Helmholtz machines use a recognition network (encoder) to approximate the posterior distribution of a generative model (decoder). In this paper we establish some necessary and some sufficient properties of a recognition network so that it can model the true posterior distribution exactly. These results are derived in the general context of probabilistic graphical modelling / Bayesian networks, for which the network represents a set of conditional independence statements. We derive both global conditions, in terms of d-separation, and local conditions for the recognition network to have the desired qualities. It turns out that for the local conditions the perfectness property (for every node, all parents are joined) plays an important role. [ABSTRACT FROM AUTHOR]

In this paper we discuss learning Bayesian networks whose conditional probability tables are either Noisy-OR models or general conditional probability tables. We refer to these models as Mixed Noisy-OR Bayesian Networks. To learn their structure, we modify the Bayesian Information Criterion used for standard Bayesian networks to reflect the number of parameters of a Noisy-OR model. We prove that the log-likelihood function of a Noisy-OR model has a unique maximum and adapt the EM-learning method for the leaky Noisy-OR model. We propose a structure learning algorithm that learns optimal Mixed Noisy-OR Bayesian Networks. We evaluate the proposed approach on synthetic data, where it performs substantially better than standard Bayesian networks. We perform experiments with Bipartite Noisy-OR Bayesian networks of different complexity to find out when the results of Mixed Noisy-OR Bayesian Networks are significantly better than the results of standard Bayesian networks and when they perform similarly. We also study how different penalties based on the number of model parameters affect the quality of the results. Finally, we apply the suggested approach to a problem from the domain of linguistics. Specifically, we use Mixed Noisy-OR Bayesian Networks to model the spread of loanwords in the South-East Asian Archipelago. We perform numerical experiments in which we compare the prediction ability of standard Bayesian networks with Mixed Noisy-OR Bayesian networks and test different pruning methods to reduce the number of parent sets considered. [ABSTRACT FROM AUTHOR]

In 2018, Jiroušek and Shenoy proposed a definition of entropy for Dempster-Shafer (D-S) belief functions called decomposable entropy (d-entropy). This paper provides an algorithm for computing the d-entropy of directed graphical D-S belief function models. We illustrate the algorithm using Almond's Captain's Problem example. For belief function undirected graphical models, assuming that the set of belief functions in the model is non-informative, the belief functions are distinct. We illustrate this using Haenni-Lehmann's Communication Network problem. As the joint belief function for this model is quasi-consonant, it follows from a property of d-entropy that the d-entropy of this model is zero, and no algorithm is required. For a class of undirected graphical models, we provide an algorithm for computing the d-entropy of such models. Finally, the d-entropy coincides with Shannon's entropy for the probability mass function of a single random variable and for a large multi-dimensional probability distribution expressed as a directed acyclic graph model called a Bayesian network. We illustrate this using Lauritzen-Spiegelhalter's Chest Clinic example represented as a belief-function directed graphical model. [ABSTRACT FROM AUTHOR]

This paper exploits knowledge made available by an external source in the form of a predictive distribution in order to elicit a parameter prior. It uses the terminology of Bayesian transfer learning, one of many domains dealing with reasoning as coherent knowledge processing. An empirical solution of the addressed problem was provided in [19] , based on an interpretation of the external predictor as an empirical distribution constructed from fictitious data. In this paper, two main contributions are provided. First, the problem is solved using formal hierarchical Bayesian modeling [25] , and the knowledge transfer is achieved optimally, i.e. in the minimum-KLD sense. Second, this hierarchical setting yields a distribution on the set of possible priors, with the choice [19] acting as the base distribution. This allows randomized choices of the prior to be generated, avoiding costly and/or intractable estimation of this prior. It also provides measures of uncertainty in the prior choice, allowing subsequent learning tasks to be assessed for robustness to this prior choice. The instantiation of the method in already published applications in knowledge elicitation, recursive learning and flat cooperation of adaptive controllers is recalled, and prospective application domains are also mentioned. [ABSTRACT FROM AUTHOR]

Possibilistic networks offer a qualitative approach for modeling epistemic uncertainty. Their practical implementation requires the specification of conditional possibility tables, as in the case of Bayesian networks for probabilities. The elicitation of probability tables by experts is made much easier by means of noisy logical gates that enable multidimensional tables to be constructed from the knowledge of a few parameters. This paper presents the possibilistic counterparts of usual noisy connectives (and, or, max, min, …). Their interest and limitations are illustrated on an example taken from a human geography modeling problem. The difference of behavior between probabilistic and possibilistic connectives is discussed in detail. Results in this paper may be useful to bring possibilistic networks closer to applications. [ABSTRACT FROM AUTHOR]

Abstract Based on the notion of information and knowledge granularity, sequential three-way decisions can be interpreted and implemented as a multilevel trisecting-and-acting framework which leads to a faster decision-making with the less cost of decision process. At each level, objects are divided into the triplet, consisting of three pair-wise disjoint regions. By considering different combinations of regions, the uncertain objects should be further investigated at the next level due to insufficient information. In fact, sequential strategy is an effective idea to deal with multi-class problem in granular computing. On the basis of this recognition, we focus on a sequential three-way approach to multi-class decision in this paper. To solve the issues of unequal cost and rule conflict simultaneously, we present a novel matrix-based multi-class decision method based on Bayesian decision procedure. Subsequently, a sequence of the proposed multi-class models is conducted under a multilevel granular structure, and objects are assigned into one certain decision class gradually from the lower levels to the higher levels via a top-down manner. Finally, the results of empirical study involving six multi-class data sets on UCI are reported to validate the feasibility and effectiveness of our proposal. [ABSTRACT FROM AUTHOR]

Abstract This paper presents results of computer simulations for a number of different probabilistic versions of inference to the best explanation (IBE), which are distinguished by the probabilistic measures used to identify the best explanation. Simulation results are presented which include cases involving ignorance of a catch-all hypothesis, uncertainty regarding the prior probability distribution over the remaining hypotheses, initial elimination of implausible hypotheses, and variations in the number of pieces of evidence taken into consideration. The results show that at least some versions of IBE perform very well in a wide range of cases. In particular, the results for all approaches remain very similar (or improve in some cases) when just the two hypotheses with the highest prior probabilities are retained and the rest are eliminated from consideration. Highlights • Probabilistic models of Inference to the Best Explanation (IBE) are proposed. • Realistic aspects include elimination of implausible hypotheses and uncertainty. • Computer simulations are used to implement and evaluate these models. • Results demonstrate the effectiveness of IBE as a mode of inference. [ABSTRACT FROM AUTHOR]

Abstract Cox's proportional hazards (CPH) model is quite likely the most popular modeling technique in survival analysis. While the CPH model is able to represent a relationship between a collection of risks and their common effect, Bayesian networks have become an attractive alternative with an increased modeling power and far broader applications. Our paper focuses on a Bayesian network interpretation of the CPH model (BN-Cox). We provide a method of encoding knowledge from existing CPH models in the process of knowledge engineering for Bayesian networks. This is important because in practice we often have CPH models available in the literature and no access to the original data from which they have been derived. We compare the accuracy of the resulting BN-Cox model to the original CPH model, Kaplan–Meier estimate, and Bayesian networks learned from data, including Naive Bayes, Tree Augmented Naive Bayes, Noisy-Max, and parameter learning by means of the EM algorithm. BN-Cox model came out as the most accurate of all BN approaches and very close to the original CPH model. We study two approaches for simplifying the BN-Cox model for the sake of representational and computational efficiency: (1) parent divorcing and (2) removing less important risk factors. We show that removing less important risk factors leads to smaller loss of accuracy. Highlights • Bayesian networks can be created from existing models, i.e. the CPH models, without access to data. • Resulting BN-Cox models produce results very close to the original CPH models. • Two approaches were studied to make BN-Cox models more computationally efficient. • BN-Cox models cannot be decomposed by parent divorcing. • Marginalizing least influential risk factors is recommended for model simplification. [ABSTRACT FROM AUTHOR]

In this paper we propose a scalable importance sampling algorithm for computing Gaussian mixture posteriors in conditional linear Gaussian Bayesian networks. Our contribution is based on using a stochastic gradient ascent procedure taking as input a stream of importance sampling weights, so that a mixture of Gaussians is dynamically updated with no need to store the full sample. The algorithm has been designed following a Map/Reduce approach and is therefore scalable with respect to computing resources. The implementation of the proposed algorithm is available as part of the AMIDST open-source toolbox for scalable probabilistic machine learning ( http://www.amidsttoolbox.com ). [ABSTRACT FROM AUTHOR]

Safety and reliability are rigorously assessed during the design of dependable systems. Probabilistic risk assessment (PRA) processes are comprehensive, structured and logical methods widely used for this purpose. PRA approaches include, but not limited to Fault Tree Analysis (FTA), Failure Mode and Effects Analysis (FMEA), and Event Tree Analysis (ETA). In conventional PRA, failure data about components is required for the purposes of quantitative analysis. In practice, it is not always possible to fully obtain this data due to unavailability of primary observations and consequent scarcity of statistical data about the failure of components. To handle such situations, fuzzy set theory has been successfully used in novel PRA approaches for safety and reliability evaluation under conditions of uncertainty. This paper presents a review of fuzzy set theory based methodologies applied to safety and reliability engineering, which include fuzzy FTA, fuzzy FMEA, fuzzy ETA, fuzzy Bayesian networks, fuzzy Markov chains, and fuzzy Petri nets. Firstly, we describe relevant fundamentals of fuzzy set theory and then we review applications of fuzzy set theory to system safety and reliability analysis. The review shows the context in which each technique may be more appropriate and highlights the overall potential usefulness of fuzzy set theory in addressing uncertainty in safety and reliability engineering. [ABSTRACT FROM AUTHOR]

This paper introduces a new probabilistic graphical model called gated Bayesian network (GBN). This model evolved from the need to represent processes that include several distinct phases. In essence, a GBN is a model that combines several Bayesian networks (BNs) in such a manner that they may be active or inactive during queries to the model. We use objects called gates to combine BNs, and to activate and deactivate them when predefined logical statements are satisfied. In this paper we also present an algorithm for semi-automatic learning of GBNs. We use the algorithm to learn GBNs that output buy and sell decisions for use in algorithmic trading systems. We show how the learnt GBNs can substantially lower risk towards invested capital, while they at the same time generate similar or better rewards, compared to the benchmark investment strategy buy-and-hold . We also explore some differences and similarities between GBNs and other related formalisms. [ABSTRACT FROM AUTHOR]

PRISM is a probabilistic logic programming formalism which allows defining a probability distribution over possible worlds. This paper investigates learning a class of generative PRISM programs known as failure-free. The aim is to learn recursive PRISM programs which can be used to model stochastic processes. These programs generalise dynamic Bayesian networks by defining a halting distribution over the generative process. Dynamic Bayesian networks model infinite stochastic processes. Sampling from infinite process can only be done by specifying the length of sequences that the process generates. In this case, only observations of a fixed length of sequences can be obtained. On the other hand, the recursive PRISM programs considered in this paper are self-terminating upon some halting conditions. Thus, they generate observations of different lengths of sequences. The direction taken by this paper is to combine ideas from inductive logic programming and learning Bayesian networks to learn PRISM programs. It builds upon the inductive logic programming approach of learning from entailment. [ABSTRACT FROM AUTHOR]

While decision trees are a popular formal and quantitative method for determining an optimal decision from a finite set of choices, for all but very simple problems they are computationally intractable. For this reason, Influence Diagrams (IDs) have been used as a more compact and efficient alternative. However, most algorithmic solutions assume that all chance variables are discrete, whereas in practice many are continuous. For such ‘Hybrid’ IDs (HIDs) the current-state-of-the-art algorithms suffer from various limitations on the kinds of inference that can be performed. This paper presents a novel method that overcomes a number of these limitations. The method solves a HID by transforming it to a Hybrid Bayesian Network (HBN) and carrying out inference on this HBN using Dynamic Discretization (DD). It generates a simplified decision tree from the propagated HBN to compute and present the optimal decisions under different decision scenarios. To provide satisfactory performance the method uses ‘inconsistent evidence’ to model functional and structural asymmetry. By using the entire marginal probability distribution of the continuous utility and chance nodes, rather than expected values alone, our method also enhances decision analysis by offering the possibility to consider additional statistics other than expected utility, such as measures of risk. We illustrate our method by using the oil wildcatter example and its variations with continuous nodes. We also use a financial score to combine risk and return measures, for illustration. [ABSTRACT FROM AUTHOR]

Second-order Bayesian networks extend Bayesian networks by incorporating uncertainty in the conditional probabilities. This paper develops a method for inference in a binary second-order Bayesian network with a singly-connected graph that builds upon the message-passing algorithm for regular belief propagation by leveraging recent developments in subjective logic. The method applies the moment-matching approach to the Beta representation of the uncertain probabilities. We provide experimental analysis which shows that the introduced method effectively captures the bounds for the actual error in a consistent manner and, at the same time, does not decrease the efficiency of the performance compared to the other similar approaches. [ABSTRACT FROM AUTHOR]

Multidimensional Bayesian network classifiers have gained popularity over the last few years due to their expressive power and their intuitive graphical representation. A drawback of this approach is that their use to perform multidimensional classification, a generalization of multi-label classification, can be very computationally demanding when there are a large number of class variables. Thus, a key challenge in this field is to ensure the tractability of these models during the learning process. In this paper, we show how information about the most common queries of multidimensional Bayesian network classifiers affects the complexity of these models. We provide upper bounds for the complexity of the most probable explanations and marginals of class variables conditioned to an instantiation of all feature variables. We use these bounds to propose efficient strategies for bounding the complexity of multidimensional Bayesian network classifiers during the learning process, and provide a simple learning method with an order-based search that guarantees the tractability of the returned models. Experimental results show that our approach is competitive with other methods in the state of the art and also ensures the tractability of the learned models. [ABSTRACT FROM AUTHOR]

Starting with Dempster's seminal work, several approaches to statistical inference based on belief functions have been proposed. Some of these approaches can be seen as implementing some form of prior-free Bayesian inference, while some others put the emphasis on long-run frequency properties and are more related to classical frequentist methods. This paper focusses on the latter class of techniques, which have been developed independently and had not been put in perspective until now. Existing definitions for frequency-calibrated belief functions as well as corresponding construction methods are reviewed, and some new notions and techniques are introduced. The connections with other frequentist notions such as confidence distributions and confidence curves are also explored. The different construction techniques are illustrated on simple inference problems, with a focus on interpretation and implementation issues. [ABSTRACT FROM AUTHOR]

The space of Bayesian network structures is prohibitively large and hence numerous techniques have been developed to prune this search space, but without eliminating the optimal structure. Such techniques are critical for structure learning to scale to larger datasets with more variables. Prior works exploited properties of the MDL score to prune away large regions of the search space that can be safely ignored by optimal structure learning algorithms. In this paper, we propose new techniques for pruning regions of the search space that can be safely ignored by algorithms that enumerate the k -best Bayesian network structures. Empirically, these techniques allow a state-of-the-art structure enumeration algorithm to scale to datasets with significantly more variables. [ABSTRACT FROM AUTHOR]

The majority of Bayesian networks learning and inference algorithms rely on the assumption that all random variables are discrete, which is not necessarily the case in real-world problems. In situations where some variables are continuous, a trade-off between the expressive power of the model and the computational complexity of inference has to be done: on one hand, conditional Gaussian models are computationally efficient but they lack expressive power; on the other hand, mixtures of exponentials (MTE), basis functions (MTBF) or polynomials (MOP) are expressive but this comes at the expense of tractability. In this paper, we introduce an alternative model called a ctdBN that lies in between. It is composed of a “discrete” Bayesian network (BN) combined with a set of univariate conditional truncated densities modeling the uncertainty over the continuous random variables given their discrete counterpart resulting from a discretization process. We prove that ctdBNs can approximate (arbitrarily well) any Lipschitz mixed probability distribution. They can therefore be exploited in many practical situations. An efficient inference algorithm is also provided and its computational complexity justifies theoretically why inference computation times in ctdBNs are very close to those in discrete BNs. Experiments confirm the tractability of the model and highlight its expressive power, notably by comparing it with BNs on classification problems and with MTEs and MOPs on marginal distributions estimations. [ABSTRACT FROM AUTHOR]

To improve the learning accuracy of the parameters in a Bayesian network from a small data set, domain knowledge is normally incorporated into the learning process as parameter constraints. MAP-based (Maximum a Posteriori) methods that utilize both sample data and domain knowledge have been well studied in the literature. Among all the MAP-based methods, the QMAP (Qualitatively Maximum a Posteriori) method exhibits the best learning performance. However, when the data is insufficient, the estimation given by the QMAP often fails to satisfy all the convex parameter constraints, and this has made the overall QMAP estimation unreliable. To ensure that QMAP estimation does not violate any given parameter constraints and to further improve the learning accuracy, a FC-QMAP (Further Constrained Qualitatively Maximum a Posteriori) algorithm is proposed in this paper. The algorithm regulates QMAP estimation by replacing data estimation with a further constrained estimation via convex optimization. Experiments and theoretical analysis show that the proposed algorithm outperforms most of the existing parameter learning methods (namely, Maximum Likelihood, Constrained Maximum Likelihood, Maximum Entropy, Constrained Maximum Entropy, Maximum a Posteriori, and Qualitatively Maximum a Posteriori). [ABSTRACT FROM AUTHOR]

Probabilistic rough set approximations are proposed based on a conditional probability to describe the desired levels of precision between the equivalence classes and an approximated set. This definition shows the detailed information on individuals satisfying some conditions but ignores the structural information. In this paper, applying the structured granules in a coarsened-grained universe, we introduce structured probabilistic rough set approximations between the power sets of the original universe and the coarsened-grained universe. By using the zooming-in and structured probabilistic rough approximation operators, two pairs of probabilistic rough lower and upper approximations on the same universe are investigated. Related properties and relationships of them are investigated. Furthermore, applying the Bayesian decision procedure, conditional probability and loss functions, three-way classifications in structured probabilistic rough set approximations are then proposed to classify the structured granules of the coarsened-grained universe into three disjoint structured probabilistic regions. This method gives the values of the pair of thresholds. Meanwhile, by using the minimum-risk decision rules, we also can construct the structured probabilistic rough lower and upper approximations. Finally, we discuss the monotonicity of structured probabilistic positive and negative regions. [ABSTRACT FROM AUTHOR]

Probabilistic Argumentation (PA) is a recent line of research in AI aiming to combine the strengths of argumentation and probabilistic reasoning. Though several models of PA have been proposed, the development of practical applications is still hindered by the lack of inference procedures and reasoning engines. In this paper, we present a reduction method to compute a recently proposed model of PA called PABA. Using the method we design inference procedures to compute the credulous semantics, the ideal semantics and the grounded semantics for a general class of PABA frameworks, that we refer to as Bayesian PABA frameworks. We also show that, though restricting to Bayesian PABA frameworks, the inference procedures can be used to compute other PA models thanks to simple translations. Finally, we implement the inference procedures to obtain a multi-semantics engine for probabilistic argumentation and demonstrate its usage. [ABSTRACT FROM AUTHOR]

In this paper we present an approach for scaling up Bayesian learning using variational methods by exploiting distributed computing clusters managed by modern big data processing tools like Apache Spark or Apache Flink, which efficiently support iterative map-reduce operations. Our approach is defined as a distributed projected natural gradient ascent algorithm, has excellent convergence properties, and covers a wide range of conjugate exponential family models. We evaluate the proposed algorithm on three real-world datasets from different domains (the Pubmed abstracts dataset, a GPS trajectory dataset, and a financial dataset) and using several models (LDA, factor analysis, mixture of Gaussians and linear regression models). Our approach compares favorably to stochastic variational inference and streaming variational Bayes, two of the main current proposals for scaling up variational methods. For the scalability analysis, we evaluate our approach over a network with more than one billion nodes and approx. 75 % latent variables using a computer cluster with 128 processing units (AWS). The proposed methods are released as part of an open-source toolbox for scalable probabilistic machine learning ( http://www.amidsttoolbox.com ) Masegosa et al. (2017) [29] . [ABSTRACT FROM AUTHOR]

Advancements in autonomous robotic systems have been impeded by the lack of a specialized computational hardware that makes real-time decisions based on sensory inputs. We have developed a novel circuit structure that efficiently approximates naïve Bayesian inference with simple Muller C-elements. Using a stochastic computing paradigm, this system enables real-time approximate decision-making with an area-energy-delay product nearly one billion times smaller than a conventional general-purpose computer. In this paper, we propose several techniques to improve the approximation of Bayesian inference by reducing stochastic bitstream autocorrelation. We also evaluate the effectiveness of these techniques for various naïve inference tasks and discuss hardware considerations, concluding that these circuits enable approximate Bayesian inferences while retaining orders-of-magnitude hardware advantages compared to conventional general-purpose computers. [ABSTRACT FROM AUTHOR]

A credal network under epistemic irrelevance is a generalised type of Bayesian network that relaxes its two main building blocks. On the one hand, the local probabilities are allowed to be partially specified. On the other hand, the assessments of independence do not have to hold exactly. Conceptually, these two features turn credal networks under epistemic irrelevance into a powerful alternative to Bayesian networks, offering a more flexible approach to graph-based multivariate uncertainty modelling. However, in practice, they have long been perceived as very hard to work with, both theoretically and computationally. The aim of this paper is to demonstrate that this perception is no longer justified. We provide a general introduction to credal networks under epistemic irrelevance, give an overview of the state of the art, and present several new theoretical results. Most importantly, we explain how these results can be combined to allow for the design of recursive inference methods. We provide numerous concrete examples of how this can be achieved, and use these to demonstrate that computing with credal networks under epistemic irrelevance is most definitely feasible, and in some cases even highly efficient. We also discuss several philosophical aspects, including the lack of symmetry, how to deal with probability zero, the interpretation of lower expectations, the axiomatic status of graphoid properties, and the difference between updating and conditioning. [ABSTRACT FROM AUTHOR]

Particle Markov Chain Monte Carlo (pMCMC) is a stochastic algorithm designed to generate samples from a probability distribution, when the density of the distribution does not admit a closed form expression. pMCMC is most commonly used to sample from the Bayesian posterior distribution in State-Space Models (SSMs), a class of probabilistic models used in numerous scientific applications. Nevertheless, this task is prohibitive when dealing with complex SSMs with massive data, due to the high computational cost of pMCMC and its poor performance when the posterior exhibits multi-modality. This paper aims to address both issues by: 1) Proposing a novel pMCMC algorithm (denoted ppMCMC), which uses multiple Markov chains (instead of the one used by pMCMC) to improve sampling efficiency for multi-modal posteriors, 2) Introducing custom, parallel hardware architectures, which are tailored for pMCMC and ppMCMC. The architectures are implemented on Field Programmable Gate Arrays (FPGAs), a type of hardware accelerator with massive parallelization capabilities. The new algorithm and the two FPGA architectures are evaluated using a large-scale case study from genetics. Results indicate that ppMCMC achieves 1.96x higher sampling efficiency than pMCMC when using sequential CPU implementations. The FPGA architecture of pMCMC is 12.1x and 10.1x faster than state-of-the-art, parallel CPU and GPU implementations of pMCMC and up to 53x more energy efficient; the FPGA architecture of ppMCMC increases these speedups to 34.9x and 41.8x respectively and is 173x more power efficient, bringing previously intractable SSM-based data analyses within reach. [ABSTRACT FROM AUTHOR]

Reconstruction of the tridimensional geometry of a visual scene using the binocular disparity information is an important issue in computer vision and mobile robotics, which can be formulated as a Bayesian inference problem. However, computation of the full disparity distribution with an advanced Bayesian model is usually an intractable problem, and proves computationally challenging even with a simple model. In this paper, we show how probabilistic hardware using distributed memory and alternate representation of data as stochastic bitstreams can solve that problem with high performance and energy efficiency. We put forward a way to express discrete probability distributions using stochastic data representations and perform Bayesian fusion using those representations, and show how that approach can be applied to diparity computation. We evaluate the system using a simulated stochastic implementation and discuss possible hardware implementations of such architectures and their potential for sensorimotor processing and robotics. [ABSTRACT FROM AUTHOR]

Living organisms survive and multiply even though they have uncertain and incomplete information about their environment and imperfect models to predict the consequences of their actions. Bayesian models have been proposed to face this challenge. Indeed, Bayesian inference is a way to do optimal reasoning when only uncertain and incomplete information is available. Various perceptive, sensory-motor, and cognitive functions have been successfully modeled this way. However, the biological mechanisms allowing animals and humans to represent and to compute probability distributions are not known. It has been proposed that neurons and assemblies of neurons could be the appropriate scale to search for clues to probabilistic reasoning. In contrast, in this paper, we propose that interacting populations of macromolecules and diffusible messengers can perform probabilistic computation. This suggests that probabilistic reasoning, based on cellular signaling pathways, is a fundamental skill of living organisms available to the simplest unicellular organisms as well as the most complex brains. [ABSTRACT FROM AUTHOR]

Capturing heterogeneous dynamic systems in a probabilistic model is a challenging problem. A single time granularity, such as employed by dynamic Bayesian networks, provides insufficient flexibility to capture the dynamics of many real-world processes. The alternative is to assume that time is continuous, giving rise to continuous time Bayesian networks. Here the problem is that the level of temporal detail is too precise to match available probabilistic knowledge. In this paper, we present a novel class of models, called hybrid time Bayesian networks, which combine discrete-time and continuous-time Bayesian networks. The new formalism allows us to more naturally model dynamic systems with regular and irregularly changing variables. We also present a mechanism to construct discrete-time versions of hybrid models and an EM-based algorithm to learn the parameters of the resulting BNs. Its usefulness is illustrated by means of a real-world medical problem. [ABSTRACT FROM AUTHOR]

Shape constrained regression models are useful for analyzing data with specific shape responses, such as (monotone) dose-response curves, the (concave) utility functions of a risk averse decision maker, the (increasing) growth curves of children's height through time, that are particularly common in medicine, economic and epidemiological studies. This paper proposes a new adaptive Bayesian approach towards constructing prior distributions, with known normalizing constants, which enables us to take into account combinations of shape constraints and to localize each shape constraint on a given interval. Our strategy enables us to compute the simulation from the posterior distribution using a reversible jump Metropolis–Hastings scheme. The major advantages of the proposal are its flexibility achieved by adjusting local shape restrictions to detect better the high and low variability regions of the data and to facilitate the control of the regression function shape when there is no data at all in some regions. We give asymptotic results that show that our Bayesian method provides consistent function estimator from the adaptive prior. The performance of our method is investigated through a simulation study with small samples. An analysis of two real data sets is presented to illustrate the new approach. [ABSTRACT FROM AUTHOR]

Errors in reasoning about probabilistic evidence can have severe consequences. In the legal domain a number of recent miscarriages of justice emphasises how severe these consequences can be. These cases, in which forensic evidence was misinterpreted, have ignited a scientific debate on how and when probabilistic reasoning can be incorporated in (legal) argumentation. One promising approach is to use Bayesian networks (BNs), which are well-known scientific models for probabilistic reasoning. For non-statistical experts, however, Bayesian networks may be hard to interpret. Especially since the inner workings of Bayesian networks are complicated, they may appear as black box models. Argumentation models, on the contrary, can be used to show how certain results are derived in a way that naturally corresponds to everyday reasoning. In this paper we propose to explain the inner workings of a BN in terms of arguments. We formalise a two-phase method for extracting probabilistically supported arguments from a Bayesian network. First, from a Bayesian network we construct a support graph , and, second, given a set of observations we build arguments from that support graph. Such arguments can facilitate the correct interpretation and explanation of the relation between hypotheses and evidence that is modelled in the Bayesian network. [ABSTRACT FROM AUTHOR]

Learning Bayesian networks with bounded tree-width has attracted much attention recently, because low tree-width allows exact inference to be performed efficiently. Some existing methods [24,29] tackle the problem by using k -trees to learn the optimal Bayesian network with tree-width up to k . Finding the best k -tree, however, is computationally intractable. In this paper, we propose a sampling method to efficiently find representative k -trees by introducing an informative score function to characterize the quality of a k -tree. To further improve the quality of the k -trees, we propose a probabilistic hill climbing approach that locally refines the sampled k -trees. The proposed algorithm can efficiently learn a quality Bayesian network with tree-width at most k . Experimental results demonstrate that our approach is more computationally efficient than the exact methods with comparable accuracy, and outperforms most existing approximate methods. [ABSTRACT FROM AUTHOR]

The current paper addresses two problems observed in structure learning applications to computational biology. The first one is dealing with mixed data. Most optimization criteria for learning algorithms are applicable to either discrete or continuous data. Mixed datasets are usually handled by discretization of continuous data, which often leads to the loss of information. In order to address this problem, we adapted discrete scoring functions to continuous data. Consequently, the same score is used to both types of variables, and the network structure may be learned from mixed data directly. The second problem is the control of the type I error level. Usually, learning algorithms output a network that is the best according to some optimization criteria, but the reliability of particular relationships represented by this network is unknown. We address this problem by allowing the user to specify the expected error level and adjusting the parameters of the scoring criteria to this level. [ABSTRACT FROM AUTHOR]

Posterior and predictive distributions for m future trials, given the first n elements of an infinite exchangeable sequence ξ ˜ 1 , ξ ˜ 2 , … , are considered in a nonparametric Bayesian setting. The former distribution is compared to the unit mass at the empirical distribution e ˜ n : = 1 n ∑ i = 1 n δ ξ ˜ i of the n past observations, while the latter is compared to the m -fold product e ˜ n m . Comparisons are made by means of distinguished probability distances inducing topologies that are equivalent to (or finer than) the topology of weak convergence of probability measures. After stating almost sure convergence to zero of these distances as n goes to infinity, the paper focuses on the analysis of the rate of approach to zero, so providing a quantitative evaluation of the approximation of posterior and predictive distributions through their frequentistic counterparts δ e ˜ n and e ˜ n m , respectively. Characteristic features of the present work, with respect to more common literature on Bayesian consistency, are: first, comparisons are made between entities which depend on the n past observation only; second, the approximations are studied under the actual (exchangeable) law of the ξ ˜ n 's, and not under hypothetical product laws p 0 ∞ , as p 0 varies among the admissible determinations of a random probability measure. [ABSTRACT FROM AUTHOR]

The paper by Denœux justifies the use of a consonant belief function to represent the information provided by a likelihood function and proposes some extensions to low-quality data. In my comments I consider the point of view of imprecise probabilities for the representation of likelihood information and the relationships with the proposal in the paper. I also argue for some alternatives to the use of consonant belief functions. Finally, I add some clarifications about the comparison with the Bayesian approach. [ABSTRACT FROM AUTHOR]

For risk assessment to be a relevant tool in the study of any type of system or activity, it needs to be based on a framework that allows for jointly analyzing both unique and repetitive events. Separately, unique events may be handled by predictive probability assignments on the events, and repetitive events with unknown/uncertain frequencies are typically handled by the probability of frequency (or Bayesian) approach. Regardless of the nature of the events involved, there may be a problem with imprecision in the probability assignments. Several uncertainty representations with the interpretation of lower and upper probability have been developed for reflecting such imprecision. In particular, several methods exist for jointly propagating precise and imprecise probabilistic input in the probability of frequency setting. In the present position paper we outline a framework for the combined analysis of unique and repetitive events in quantitative risk assessment using both precise and imprecise probability. In particular, we extend an existing method for jointly propagating probabilistic and possibilistic input by relaxing the assumption that all events involved have frequentist probabilities; instead we assume that frequentist probabilities may be introduced for some but not all events involved, i.e. some events are assumed to be unique and require predictive – possibly imprecise – probabilistic assignments, i.e. subjective probability assignments on the unique events without introducing underlying frequentist probabilities for these. A numerical example related to environmental risk assessment of the drilling of an oil well is included to illustrate the application of the resulting method. [ABSTRACT FROM AUTHOR]

A major difficulty in building Bayesian network (BN) models is the size of conditional probability tables, which grow exponentially in the number of parents. One way of dealing with this problem is through parametric conditional probability distributions that usually require only a number of parameters that is linear in the number of parents. In this paper, we introduce a new class of parametric models, the Probabilistic Independence of Causal Influences (PICI) models, that aim at lowering the number of parameters required to specify local probability distributions, but are still capable of efficiently modeling a variety of interactions. A subset of PICI models is decomposable and this leads to significantly faster inference as compared to models that cannot be decomposed. We present an application of the proposed method to learning dynamic BNs for modeling a woman's menstrual cycle. We show that PICI models are especially useful for parameter learning from small data sets and lead to higher parameter accuracy than when learning CPTs. [ABSTRACT FROM AUTHOR]

Variable elimination (VE) and join tree propagation (JTP) are two alternatives to inference in Bayesian networks (BNs). VE, which can be viewed as one-way propagation in a join tree, answers each query against the BN meaning that computation can be repeated. On the other hand, answering a single query with JTP involves two-way propagation, of which some computation may remain unused. In this paper, we propose marginal tree inference (MTI) as a new approach to exact inference in discrete BNs. MTI seeks to avoid recomputation, while at the same time ensuring that no constructed probability information remains unused. Thereby, MTI stakes out middle ground between VE and JTP. The usefulness of MTI is demonstrated in multiple probabilistic reasoning sessions. [ABSTRACT FROM AUTHOR]

Multi-label classification problems require each instance to be assigned a subset of a defined set of labels. This problem is equivalent to finding a multi-valued decision function that predicts a vector of binary classes. In this paper we study the decision boundaries of two widely used approaches for building multi-label classifiers, when Bayesian network-augmented naive Bayes classifiers are used as base models: Binary relevance method and chain classifiers . In particular extending previous single-label results to multi-label chain classifiers, we find polynomial expressions for the multi-valued decision functions associated with these methods. We prove upper boundings on the expressive power of both methods and we prove that chain classifiers provide a more expressive model than the binary relevance method. [ABSTRACT FROM AUTHOR]

Probabilistic causal interaction models have become quite popular among Bayesian-network engineers as elicitation of all probabilities required often proves the main bottleneck in building a real-world network with domain experts. The best-known interaction models are the noisy-OR model and its generalisations. These models in essence are parameterised conditional probability tables for which just a limited number of parameter probabilities are required. The models assume specific properties of intercausal interaction and cannot be applied uncritically. Given their clear engineering advantages however, they are subject to ill-considered use. This paper demonstrates that such ill-considered use can result in poorly calibrated output probabilities from a Bayesian network. By studying, in an analytical way, the propagation effects of noisy-OR calculated probability values, we identify conditions under which use of the model can be harmful for a network's performance. These conditions demonstrate that use of the noisy-OR model for mere pragmatic reasons is sometimes warranted, even when the model's underlying assumptions are not met in reality. [ABSTRACT FROM AUTHOR]

In this paper we study how different theoretical concepts of Bayesian networks have been extended to chain graphs. Today there exist mainly three different interpretations of chain graphs in the literature. These are the Lauritzen–Wermuth–Frydenberg, the Andersson–Madigan–Perlman and the multivariate regression interpretations. The different chain graph interpretations have been studied independently and over time different theoretical concepts have been extended from Bayesian networks to also work for the different chain graph interpretations. This has however led to confusion regarding what concepts exist for what interpretation. In this article we do therefore study some of these concepts and how they have been extended to chain graphs as well as what results have been achieved so far. More importantly we do also identify when the concepts have not been extended and contribute within these areas. Specifically we study the following theoretical concepts: Unique representations of independence models, the split and merging operators, the conditions for when an independence model representable by one chain graph interpretation can be represented by another chain graph interpretation and finally the extension of Meek's conjecture to chain graphs. With our new results we give a coherent overview of how each of these concepts is extended for each of the different chain graph interpretations. [ABSTRACT FROM AUTHOR]

This note is a rejoinder to comments by Dubois and Moral about my paper "Likelihood-based belief function: justification and some extensions to low-quality data" published in this issue. The main comments concern (1) the axiomatic justification for defining a consonant belief function in the parameter space from the likelihood function and (2) the Bayesian treatment of statistical inference from uncertain observations, when uncertainty is quantified by belief functions. Both issues are discussed in this note, in response to the discussants' comments. [ABSTRACT FROM AUTHOR]

Abstract: System failures, for example in electrical power systems, can have catastrophic impact on human life and high-cost missions. Due to an electrical fire in Swissair flight 111 on September 2, 1998, all 229 passengers and crew on board sadly lost their lives. A battery failure most likely took place on the Mars Global Surveyor, which unfortunately last communicated with Earth and thus ended its mission on November 2, 2006. Fault diagnosis techniques that seek to hinder similar accidents in the future are being developed in this article. We present comprehensive fault diagnosis methods for dynamic and hybrid domains with uncertainty, and validate them using electrical power system data. Our approach relies on the use of Bayesian networks, which model the electrical power system, compiled to arithmetic circuits. We handle in an integrated way varying fault dynamics (both persistent and intermittent faults), fault progression (both abrupt and drift faults), and fault behavior cardinality (both discrete and continuous behaviors). Our work has resulted in a software system for fault diagnosis, ProDiagnose, that has been the top performer in three of the four international diagnostics competitions in which it participated. In this paper we comprehensively present our methods as well as novel and extensive experimental results on data from a NASA electrical power system. [Copyright &y& Elsevier]

Abstract: We develop a global sensitivity analysis to measure the robustness of the Bayesian estimators with respect to a class of prior distributions. This class arises when we consider multiplicative contamination of a base prior distribution. A similar structure was presented by van der Linde [12]. Some particular specifications for this multiplicative contamination class coincide with well known families of skewed distributions. In this paper, we explore the skew-normal multiplicative contamination class for the prior distribution of the location parameter of a normal model. Results of a Bayesian conjugation and expressions for some measures of distance between posterior means and posterior variance are obtained. We also elaborate on the behavior of the posterior means and of the posterior variances through a simulation study. [Copyright &y& Elsevier]