Search

Objectives: To discover the most relevant features for predicting mortality of patients with Covid-19 and their change over the waves of the pandemic and patient stay at the hospital. Methods: A retrospective study of 83,665 patients hospitalized in different centres belonging to the Fundacion Jimenez Diaz from February 2020 to March 2022. Recorded features for each patient were evaluated for relevance in a classification task using random forests, generalized boosting models and single-variable binary classificators, repeating the process for data of each wave of the pandemic and from different moments in time during hospital stay. The resulting rankings were combined using Coppeland’s method. The results are available through a dashboard at https://covid-dashboard-cig.herokuapp.com/ Results: We find that for the common ranking, top variables coincide with the literature being: age, Oxygen saturation and common inflammation markers. Over the stay of the patient, signs of kidney failure become more relevant predictors while other inflammation markers lose relevance. Meanwhile, we find that more features stay relevant over the later waves of the pandemic, especially those relating to hepatic failure, possibly indicating patients with more comorbidities or poor responses to medication.

Traditional estimation of distribution algorithms (EDAs) often use Gaussian densities to optimize continuous functions, such as the estimation of Gaussian network algorithms (EGNAs) which use Gaussian Bayesian networks (GBNs). However, this assumes a parametric density function, and, in GBNs, linear dependencies between variables. Furthermore, the EGNA baseline learns a GBN at each iteration based on the best individuals in the last iteration, which may lead to local optimum convergence or large variance between solutions across multiple independent runs of the algorithm. In this work we propose a semiparametric EDA in which the restriction of assuming Gaussianity in the variables is relaxed using semiparametric Bayesian networks, in which nodes estimated by kernels coexist with nodes that assume Gaussianity, and the algorithm itself is able to determine where to use each type of node. Additionally, our approach takes into account information from several past iterations to learn the semiparametric Bayesian network from which the new solutions are sampled in each iteration. The empirical results show that semiparametric EDAs are a useful tool for continuous scenarios compared to different kinds of EDAs and other optimization techniques in continuous environments.

Real-world problems are commonly characterized by a high feature dimensionality, which hinders the modelling and descriptive analysis of the data. However, some of these data may be irrelevant or redundant for the learning process. Different approaches can be used to reduce this information, improving not only the speed of building models but also their performance and interpretability. In this review, we focus on feature subset selection (FSS) techniques, which select a subset of the original feature set without making any transformation on the attributes. Traditional batch FSS algorithms may not be adequate to efficiently handle large volumes of data, either because memory problems arise or data are received in a sequential manner. Thus, this article aims to survey the state of the art of incremental FSS algorithms, which can perform more efficiently under these circumstances. Different strategies are described, such as incrementally updating feature weights, applying information theory or using rough set-based FSS, as well as multiple supervised and unsupervised learning tasks where the application of FSS is interesting.

The increasing amount of real-time data collected from sensors in industrial environments has accelerated the application of machine learning in decision-making. Reinforcement learning (RL) is a powerful tool to find optimal policies for achieving a given goal. However, RL’s typical application is risky and insufficient in environments where actions can have irreversible consequences and require interpretability and fairness. While new trends in RL may provide guidance based on expert knowledge, they do not often consider uncertainty or include prior knowledge in the learning process. We propose a causal reinforcement learning alternative based on Bayesian networks (RLBNs) to address this challenge. The RLBN simultaneously models a policy and takes advantage of the joint distribution of the state and action space, reducing uncertainty in unknown situations. We propose a training algorithm for the network’s parameters and structure based on the reward function and likelihood of the effects and measurements taken. Our experiment with the CartPole benchmark and industrial fouling using ordinary differential equations (ODEs) demonstrates that RLBNs are interpretable, secure, flexible, and more robust than their competitors. Our contributions include a novel method that incorporates expert knowledge into the decision-making engine. It uses Bayesian networks with a predefined structure as a causal graph and a hybrid learning strategy that considers both likelihood and reward. This would avoid losing the virtues of the Bayesian network.

Learning the structure of continuous-time Bayesian networks directly from data has traditionally been performed using score-based structure learning algorithms. Only recently has a constraint-based method been proposed, proving to be more suitable under specific settings, as in modelling systems with variables having more than two states. As a result, studying diverse structure learning algorithms is essential to learn the most appropriate models according to data characteristics and task-related priorities, such as learning speed or accuracy. This article proposes alternative algorithms for learning multidimensional continuous-time Bayesian network classifiers, introducing, for the first time, constraint-based and hybrid algorithms for these models. Nevertheless, these contributions also apply to the simpler one-dimensional classification problem for which only score-based solutions exist in the literature. More specifically, the aforementioned constraint-based structure learning algorithm is first adapted to the supervised classification setting. Then, a novel algorithm of this kind, specifically tailored for the multidimensional classification problem, is presented to improve the learning times for the induction of multidimensional classifiers. Finally, a hybrid algorithm is introduced, attempting to combine the strengths of the score- and constraint-based approaches. Experiments with synthetic and real-world data are performed not only to validate the capabilities of the proposed algorithms but also to conduct a comparative study of the available competitors.

Traditional hidden Markov models have been a useful tool to understand and model stochastic dynamic data; in the case of nonGaussian data, models such as mixture of Gaussian hidden Markov models can be used. However, these suffer from the computation of precision matrices and have a lot of unnecessary parameters. As a consequence, such models often perform better when it is assumed that all variables are independent, a hypothesis that may be unrealistic. Hidden Markov models based on kernel density estimation are also capable of modeling non-Gaussian data, but they assume independence between variables. In this article, we introduce a new hidden Markov model based on kernel density estimation, which is capable of capturing kernel dependencies using context-specific Bayesian networks. The proposed model is described, together with a learning algorithm based on the expectation-maximization algorithm. Additionally, the model is compared to related HMMs on synthetic and real data. From the results, the benefits in likelihood and classification accuracy from the proposed model are quantified and analyzed.

With the increase of computational power and memory capacity, it is possible to record and analyse lots of features of different nature in real time or in a data stream manner. Nonetheless, in many applications, not all variables may be relevant to explain the nature of the data. Feature subset selection strategies are therefore required to extract the relevant features to understand the data generating process. Much work has been performed in the area of supervised classification problems regarding feature selection, whereas for unsupervised environments, scarce studies have been conducted. In current feature subset selection methodologies for unsupervised data streams, the set of relevant variables are reevaluated or forgotten whenever a new instance or chunk of data arrives: this approach can be computationally expensive or unnecessary. Therefore, in this article, we provide a method to perform feature subset selection in unsupervised data streams. An embedded feature subset selection methodology based on asymmetric hidden Markov models and novel concept discovery strategies is used. This methodology can be used to detect novel concepts in data, and update the set of relevant features when a drift in the data stream is detected. Thus, the relevant variables are updated only when needed. To make this possible, we provide a model for batch and online problems, that is capable of dynamically determining the relevant variables to address feature selection in data streams. Additionally, the model provides domain insights using context-specific Bayesian networks which can be helpful to understand the dynamic process. To validate the proposed methodology, synthetic and real data from ball-bearings are used. Additionally, the proposed methodology is compared with other state of the art methodologies. The proposed method can be used to update the relevant features when needed and is more stable than its competitors.

When we face patients arriving to a hospital suffering from the effects of some illness, one of the main problems we can encounter is evaluating whether or not said patients are going to require intensive care in the near future. This intensive care requires allotting valuable and scarce resources, and knowing beforehand the severity of a patients illness can improve both its treatment and the organization of resources. We illustrate this issue in a dataset consistent of Spanish COVID-19 patients from the sixth epidemic wave where we label patients as critical when they either had to enter the intensive care unitor passed away. We then combine the use of dynamic Bayesian networks, to forecast the vital signs and the blood analysis results of patients over the next 40 hours, and neural networks, to evaluate the severity of a patients disease in that interval of time. Our empirical results show that the transposition of the current state of a patient to future values with the DBN for its subsequent use in classification obtains better the accuracy and g-means core than a direct application with a classifier.

We present the Fast Greedy Equivalence Search (FGES)-Merge, a new method for learning the structure of gene regulatory networks via merging locally learned Bayesian networks, based on the fast greedy equivalent search algorithm. The method is competitive with the state of the art in terms of the Matthews correlation coefficient, which takes into account both precision and recall, while also improving upon it in terms of speed, scaling up to tens of thousands of variables and being able to use empirical knowledge about the topological structure of gene regulatory networks. To showcase the ability of our method to scale to massive networks, we apply it to learning the gene regulatory network for the full human genome using data from samples of different brain structures (from the Allen Human Brain Atlas). Furthermore, this Bayesian network model should predict interactions between genes in a way that is clear to experts, following the current trends in explainable artificial intelligence. To achieve this, we also present a new open-access visualization tool that facilitates the exploration of massive networks and can aid in finding nodes of interest for experimental tests.

The automatic induction of machine learning models capable of addressing supervised learning, feature selection, clustering and reinforcement learning problems requires sophisticated intelligent search procedures. These searches are usually performed in the possible model structure spaces, leading to combinatorial optimization problems, and in the parameter spaces, where it is necessary to solve continuous optimization problems. This paper reviews how the estimation of distribution algorithms, a kind of evolutionary algorithm, can be used to address these problems. Topics include preprocessing, mining association rules, selecting variables, searching for the optimal supervised learning model (both probabilistic and nonprobabilistic models), finding the best hierarchical, partitional or probabilistic clustering, obtaining the optimal policy in reinforcement learning and performing inference and structural learning in Bayesian networks for association discovery. Interesting guidelines for future work in this area are also provided.

MAP-independence is a novel concept concerned with explaining the (MAP)relevance of intermediate nodes for maximum a posteriori (MAP) computations in Bayesian networks. Building upon properties of MAP-independence, we introduce and experiment with methods for finding sets of relevant nodes using both an exhaustive and a heuristic approach. Our experiments show that these properties significantly speed up run time for both approaches. In addition, we link MAP-independence to defeasible reasoning, a type of reasoning that analyses how new evidence may invalidate an already established conclusion. Ways to present users with an explanation using MAP-independence are also suggested.

This paper presents a new class of Bayesian networks called hybrid semiparametric Bayesian networks, which can model hybrid data (discrete and continuous data) by mixing parametric and nonparametric estimation models. The parametric estimation models can represent a conditional linear Gaussian relationship between variables, while the nonparametric estimation model can represent other types of relationships, such as non-Gaussian and nonlinear relationships. This new class of Bayesian networks generalizes the conditional linear Gaussian Bayesian networks, including them as a special case. In addition, we describe a learning procedure for the structure and the parameters of our proposed type of Bayesian network. This learning procedure finds the best combination of parametric and nonparametric models automatically from data. This requires the definition of a cross-validated score. We also detail how new data can be sampled from a hybrid semiparametric Bayesian network, which in turn can be useful to solve other related tasks, such as inference. Furthermore, we intuitively relate our proposal with adaptive kernel density estimation models. The experimental results show that hybrid semiparametric Bayesian networks are a valuable contribution when dealing with data that do not meet the parametric assumptions that are expected for other models, such as conditional linear Gaussian Bayesian networks. We include experiments with synthetic data and real-world data from the UCI repository which demonstrate the good performance and the ability to extract useful information about the relationship between the variables in the model.

Bayesian network structure learning is an NP-hard problem that has been faced by a number of traditional approaches in recent decades. Currently, quantum technologies offer a wide range of advantages that can be exploited to solve optimization tasks that cannot be addressed in an efficient way when utilizing classic computing approaches. In this work, a specific type of variational quantum algorithm, the quantum approximate optimization algorithm, was used to solve the Bayesian network structure learning problem, by employing3n(n−1)/2 qubits, where n is the number of nodes in the Bayesian network to be learned. Our results showed that the quantum approximate optimization algorithm approach offers competitive results with state-of-the-art methods and quantitative resilience to quantum noise. The approach was applied to a cancer benchmark problem, and the results justified the use of variational quantum algorithms for solving the Bayesian network structure learning problem.

Real‐world applications often involve multifaceted data with several reasonable interpretations. To cluster this data, we need methods that are able to produce multiple clustering solutions. To this purpose, it is interesting to learn a finite mixture model with multiple latent variables, where each latent variable represents a unique way to partition the data. However, although there is an extensive literature on multipartition clustering methods for categorical data and for continuous data, there is a lack of work for mixed data. In this paper, we propose a multipartition clustering method that is able to efficiently deal with mixed data by exploiting the Bayesian network factorization and the variational Bayes framework. We show the flexibility and applicability of the proposed method by solving clustering, density estimation, and missing data imputation tasks in real‐world data sets. For reproducibility, all code, data, and results can be found in the following public repository: https://github.com/ferjorosa/mpc‐mixed.

Many real-life problems are stated as nonlabeled high-dimensional data. Current strategies to select features are mainly focused on labeled data, which reduces the options to select relevant features for unsupervised problems, such as clustering. Recently, feature saliency models have been introduced and developed as clustering models to select and detect relevant variables/features as the model is learned. Usually, these models assume that all variables are independent, which narrows their applicability. This article introduces asymmetric hidden Markov models with feature saliencies, i.e., models capable of simultaneously determining during their learning phase relevant variables/features and probabilistic relationships between variables. The proposed models are compared with other state-of-the-art approaches using synthetic data and real data related to grammatical face videos and wear in ball bearings. We show that the proposed models have better or equal fitness than other state-of-the-art models and provide further data insights.

The degradation of critical components inside large industrial assets, such as ball-bearings, has a negative impact on production facilities, reducing the availability of assets due to an unexpectedly high failure rate. Machine learning-based monitoring systems can estimate the remaining useful life (RUL) of ball bearings, reducing the downtime by early failure detection.However, traditional approaches for predictive systems require run-to-failure (RTF) data as training data, which in real scenarios can be scarce and expensive to obtain as the expected useful life could be measured in years. Therefore, to overcome the need of RTF, we propose a new methodology based on online novelty detection and asymmetrical hidden Markov models (As-HMMs) to work out the health assessment. This new methodology does not require previous RTF data and can adapt to natural degradation of mechanical components over time in data-stream and online environments. As the system is designed to work online within the electrical cabinet of machines, it has to be deployed using embedded electronics. Therefore, a performance analysis of As-HMM is presented to detect the strengths and critical points of the algorithm. To validate our approach, we use real life ball-bearing data sets and compare our methodology with other methodologies where no RTF data are needed and check the advantages in RUL prediction and health monitoring. As a result, we showcase a complete end-to-end solution from the sensor to actionable insights regarding RUL estimation toward maintenance application in real industrial environments.

Many real-world optimization problems involve two different subsets of variables: decision variables, and those variables which are not present in the cost function but constrain the solutions, and thus, must be considered during optimization. Thus, dependencies between and within both subsets of variables must be considered. In this paper, an estimation of distribution algorithm (EDA) is implemented to solve this type of complex optimization problems. A Gaussian Bayesian network is used to build an abstraction model of the search space in each iteration to identify patterns among the variables. As the algorithm is initialized from data, we introduce a new hyper parameter to control the influence of the initial data in the decisions made during the EDA execution. The results show that our algorithm improves the cost function more than the expert knowledge does.

In a real life process evolving over time, the relationship between its relevant variables may change. Therefore, it is advantageous to have different inference models for each state of the process. Asymmetric hidden Markov models fulfil this dynamical requirement and provide a framework where the trend of the process can be expressed as a latent variable. In this paper, we modify these recent asymmetric hidden Markov models to have an asymmetric autoregressive component in the case of continuous variables, allowing the model to choose the order of autoregression that maximizes its penalized likelihood for a given training set. Additionally, we show how inference, hidden states decoding and parameter learning must be adapted to fit the proposed model. Finally, we run experiments with synthetic and real data to show the capabilities of this new model.

When modelling multivariate continuous time series, a common issue is to find that the original processes that generated the data are nonlinear or that they drift away from the original distribution as the system evolves over time. In these scenarios, using a linear model such as a Gaussian dynamic Bayesian network (DBN) can result in severe forecasting inaccuracies due to the structure and parameters of the model remaining constant without considering either the elapsed time or the state of the system. To approach this problem, we propose a hybrid model that combines a model tree with DBNs. The model first divides the original data set into different scenarios based on the splits made by the tree over the system variables and then performs apiece wise regression in each branch of the tree to obtain nonlinear forecasts. The experimental results on three different datasets show that our model out performs standard DBN models when dealing with nonlinear processes and is competitive with state‐of the‐art time series forecasting methods.

En este documento se procederá a investigar sobre la minería de datos ya que es un tema de actualidad que cada vez está cobrando más importancia, desde su aplicación en empresas para la toma de decisiones corporativas hasta en el ámbito social y cultural, en este caso nos centraremos en el estudio de conjuntos de datos que nos ayuden a comprender la brecha entre hombres y mujeres en el ámbito laboral del mundo STEM usando la minería de datos. La minería de datos es el proceso de análisis de datos que involucra ciencias como la estadística y varios ámbitos de la informática como la inteligencia artificial, el aprendizaje automático y la gestión de las bases de datos. Aplicada a conjuntos grandes de datos permite descubrir nuevo conocimiento, como patrones, irregularidades o correlaciones que permitan predecir resultados en el futuro. Para la realización de este trabajo de fin de grado primero se explicará en profundidad qué es la minería de datos, así como su importancia en el cuadro actual de la sociedad y cuáles son sus aplicaciones. Se definirán las dos técnicas de la minería de datos, que son la clasificación supervisada y la clasificación no supervisada, nombrando los algoritmos principales de cada una de ellas y su funcionamiento. Por último, con el conjunto de datos público seleccionado se procederá a aplicar el conocimiento discutido en los apartados anteriores en un caso real, del cual se extraerá información nueva, el objetivo es recopilar nueva información útil a través del análisis de los resultados obtenidos con los algoritmos de minería de datos aplicados.---ABSTRACT---In this document we will investigate data mining since is a topic of conversation that is becoming increasingly important, from its application in companies for corporate decision making to the social and cultural spheres. In this document, we will focus on the study of datasets that help us understand the gap between men and women in the STEM workplace using data mining. Data mining is t

When we face patients arriving to a hospital suffering from the effects of some illness, one of the main problems we can encounter is evaluating whether or not said patient is going to require intensive care in the near future. This intensive care requires allotting valuable and scarce resources, and knowing beforehand the severity of a patients illness can improve both its treatment and the organization of resources. We illustrate this issue in a dataset consistent of Spanish COVID-19 patients from the sixth epidemic wave and label patients as critical when they either had to enter the intensive care unit or passed away. We then combine the use of dynamic Bayesian networks, to forecast the vital signs and the blood analysis results of patients over the next 80 hours, and different classifier models, to evaluate the severity of a patients illness. Our empirical results show that the transposition of the current state of a patient for its subsequent use in classification obtains promising results for the tested models.

Learning Bayesian networks in the presence of non-linear relationships among random variables can be challenging. Nonparametric tests of conditional independence can exploit the kernel trick to e_ciently compute the appropriate statistics and have been used for constraint-based structure learning. An example of the use of the kernel trick for score-based learning rests on the assumption that the variables are Gaussian in the feature space. We have adapt this nonparametric approach into a semiparametric method. This may reduce complexity as some variables may well be modeled as Gaussians in the input space instead of being projected to a higher-dimensional feature space. We do this by combining the linear kernel, corresponding to Gaussians in the input space, and the Gaussian one, used exclusively in the original proposal. We propose a search algorithm to adaptively determine, during structure learning, which kernel to associate with each variable while maintaining score e_ciency which is due to score decomposability when the kernels are held _xed. Preliminary empirical evaluation shows that the proposal is promising.

Over the years, research studies have shown there is a key connection between the microbial community in the gut, genes, and immune system. Understanding this association may help discover the cause of complex chronic idiopathic disorders such as inflammatory bowel disease. Even though important efforts have been put into the field, the functions, dynamics, and causation of dysbiosis state performed by the microbial community remains unclear. Machine learning models can help elucidate important connections and relationships between microbes in the human host. Our study aims to extend the current knowledge of associations between the human microbiome and health and disease through the application of dynamic Bayesian networks to describe the temporal variation of the gut microbiota and dynamic relationships between taxonomic entities and clinical variables. We develop a set of preprocessing steps to clean, filter, select, integrate, and model informative metagenomics, metatranscriptomics, and metabolomics longitudinal data from the Human Microbiome Project. This study accomplishes novel network models with satisfactory predictive performance (accuracy = 0.648) for each inflammatory bowel disease state, validating Bayesian networks as a framework for developing interpretable models to help understand the basic ways the different biological entities (taxa, genes, metabolites) interact with each other in a given environment (human gut) over time. These findings can serve as a starting point to advance the discovery of novel therapeutic approaches and new biomarkers for precision medicine.

We introduce semiparametric Bayesian networks that combine parametric and nonparametric conditional probability distributions. Their aim is to incorporate the advantages of both components: the bounded complexity of parametric models and the flexibility of nonparametric ones. We demonstrate that semiparametric Bayesian networks generalize two well-known types of Bayesian networks: Gaussian Bayesian networks and kernel density estimation Bayesian networks. For this purpose, we consider two different conditional probability distributions required in a semiparametric Bayesian network. In addition, we present modifications of two well-known algorithms (greedy hill-climbing and PC) to learn the structure of a semiparametric Bayesian network from data. To realize this, we employ as core function based on cross-validation. In addition, using a validation dataset, we apply an early-stopping criterion to avoid overfitting. To evaluate the applicability of the proposed algorithm, we conduct an exhaustive experiment on synthetic data sampled by mixing linear and nonlinear functions, multivariate normal data sampled from Gaussian Bayesian networks, real data from the UCI repository, and bearings degradation data. Asa result of this experiment, we conclude that the proposed algorithm accurately learns the combination of parametric and nonparametric components, while achieving a performance comparable with those provided by state-of-the-art methods._ 2021 The Author(s). Published by Elsevier Inc. This is an open access article under the CCBY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/).

BayeSuites is the first web framework for learning, visualizing, and interpreting Bayesian networks (BNs)that can scale to tens of thousands of nodes while providing fast and friendly user experience. All the necessary features that enable this are reviewed in this paper; these features include scalability, extensibility, interoperability, ease of use, and interpretability. Scalability is the key factor in learning and processing massive networks within reasonable time; for a maintainable software open to new functionalities, extensibility and interoperability are necessary. Ease of use and interpretability are fundamental aspects of model interpretation, fairly similar to the case of the recent explainable artificial intelligence trend. We present the capabilities of our proposed framework by highlighting a real example of a BN learned from genomic data obtained from Allen Institute for Brain Science. The extensibility properties of the software are also demonstrated with the help of our BN-based probabilistic clustering implementation, together with another genomic-data example.

Multi-dimensional classification is a cutting-edge problem, in which the values of multiple class variables have to be simultaneously assigned to a given example. It is an extension of the well known multi-label subproblem, in which the class variables are all binary. In this article, we review and expand the set of performance evaluation measures suitable for assessing multi-dimensional classifiers. We focus on multi-dimensional Bayesian network classifiers, which directly cope with multi-dimensional classification and consider dependencies among class variables. A comprehensive survey of this state-of-the-art classification model is offered by covering aspects related to their learning and inference process complexities. We also describe algorithms for structural learning, provide real-world applications where they have been used, and compile a collection of related software.

As artificial intelligence is being increasingly used for high-stakes applications, it is becoming more and more important that the models used be interpretable. Bayesian networks offer a paradigm for interpretable artificial intelligence that is based on probability theory. They provide a semantics that enables a compact, declarative representation of a joint probability distribution over the variables of a domain by leveraging the conditional independencies among them. The representation consists of a directed acyclic graph that encodes the conditional independencies among the variables and a set of parameters that encodes conditional distributions. This representation has provided a basis for the development of algorithms for probabilistic reasoning (inference) and for learning probability distributions from data. Bayesian networks are used for a wide range of tasks in machine learning, including clustering, supervised classification, multi-dimensional supervised classification, anomaly detection, and temporal modeling. They also provide a basis for estimation of distribution algorithms, a class of evolutionary algorithms for heuristic optimization. We illustrate the use of Bayesian networks for interpretable machine learning and optimization by presenting applications in neuroscience, the industry, and bioinformatics, covering a wide range of machine learning and optimization tasks.

The multidimensional classification of multivariate timeseries deals with the assignment of multiple classes to time‐ordered data described by a set of feature variables. Although this challenging task has received almost no attention in the literature, it is present in a wide variety of domains, such as medicine, finance or industry. The complexity of this problem lies in two nontrivial tasks, the learning with multivariate time series in continuous time and the simultaneous classification of multiple class variables that may show dependencies between them. These can be addressed with different strategies, but most of them may involve a difficult preprocessing of the data, high space and classification complexity or ignoring useful interclass dependencies. Additionally, no attention has been given to the development of new multidimensional classifiers of time series based on probabilistic graphical models, even though transparent models can facilitate further understanding of the domain. In this paper, a novel probabilistic graphical model is proposed, which is able to classify a discrete multivariate temporal sequence into multiple class variables while modeling their dependencies. This model extends continuous time Bayesian networks to the multidimensional classification problem, which are able to explicitly represent the behavior of time series that evolve over continuous time. Different methods for the learning of the parameters and structure of the model are presented, and numerical experiments on synthetic and real‐world data show encouraging results in terms of performance and learning time with respect to independent classifiers, the current alternative approach under the continuous time Bayesian network paradigm.

Identifcation of Parkinson’s disease subtypes may help understand underlying disease mechanisms and provide personalized management. Although clustering methods have been previously used for subtyping, they have reported generic subtypes of limited relevance in real life practice because patients do not always ft into a single category. The aim of this study was to identify new subtypes assuming that patients could be grouped diferently according to certain sets of related symptoms. To this purpose, a novel model-based multi-partition clustering method was applied on data from an international, multi-center, cross-sectional study of 402 Parkinson’s disease patients. Both motor and non-motor symptoms were considered. As a result, eight sets of related symptoms were identifed. Each of them provided a diferent way to group patients: impulse control issues, overall non-motor symptoms, presence of dyskinesias and pyschosis, fatigue, axial symptoms and motor fuctuations, autonomic dysfunction, depression, and excessive sweating. Each of these groups could be seen as a subtype of the disease. Signifcant diferences between subtypes (P< 0.01) were found in sex, age, age of onset, disease duration, Hoehn & Yahr stage, and treatment. Independent confrmation of these results could have implications for the clinical management of Parkinson’s disease patients.

Pyramidal neurons are the most common neurons in the cerebral cortex. Understanding how they differ between species is a key challenge in neuroscience. We compared human temporal cortex and mouse visual cortex pyramidal neurons from the Allen Cell Types Database in terms of their electrophysiology and dendritic morphology. We found that, among other differences, human pyramidal neurons had a higher action potential threshold voltage, a lower input resistance, and larger dendritic arbors. We learned Gaussian Bayesian networks from the data in order to identify correlations and conditional independencies between the variables and compare them between the species. We found strong correlations between electrophysiological and morphological variables in both species. In human cells, electrophysiological variables were correlated even with morphological variables that are not directly related to dendritic arbor size or diameter, such as mean bifurcation angle and mean branch tortuosity. Cortical depth was correlated with both electrophysiological and morphological variables in both species, and its effect on electrophysiology could not be explained in terms of the morphological variables. For some variables, the effect of cortical depth was opposite in the two species. Overall, the correlations among the variables differed strikingly between human and mouse neurons. Besides identifying correlations and conditional independencies, the learned Bayesian networks might be useful for probabilistic reasoning regarding the morphology and electrophysiology of pyramidal neurons.

This article proposes a novel algorithm to detect anomalies during a laser-surface heat-treatment process recorded using a high-speed thermal camera. Our approach detects anomalies by tracking the movement of the laser spot along the surface of a production artifact. No previous knowledge of the anomalies is assumed. The model attempts to learn the behavior of a normal process so that anomalous video frames can be detected when the test data differ significantly from the learned model. The model is trained using real process data provided by a company operating in the automotive sector. First, laser-spot movements are obtained by computing a sequence of their positions. Second, the expected movement of the laser spot is accurately determined from the non anomalous data by using a model based on training multiple kernel density estimation models. Finally, an anomaly score is introduced to classify a workpiece as normal or anomalous using the trained model. Furthermore, our methodology is computationally efficient when compared to other techniques. Additionally, our objective is to perform in-process classification, that is, to perform the classification within a short period after the laser-surface heat-treatment process ends.

Objective: Drug-resistant temporal lobe epilepsy (TLE) is the most common type of epilepsy for which patients undergo surgery. Despite the best clinical judgment and currently available prediction algorithms, surgical outcomes remain variable. We aimed to build and to evaluate the performance of multidimensional Bayesian network classifiers (MBCs), a type of probabilistic graphical model, at predicting probability of seizure freedom after TLE surgery. Methods: Clinical, neurophysiological, and imaging variables were collected from 231 TLE patients who underwent surgery at the University of California, San Francisco (UCSF) or the Montreal Neurological Institute (MNI) over a 15-yearperiod. Postsurgical Engel outcomes at year 1 (Y1), Y2, and Y5 were analyzed as primary end points. We trained an MBC model on combined data sets from both institutions. Bootstrap bias corrected cross-validation (BBC-CV) was used to evaluate the performance of the models. Results: The MBC was compared with logistic regression and Cox proportional hazards according to the area under the receiver-operating characteristic curve (AUC).The MBC achieved an AUC of 0.67 at Y1, 0.72 at Y2, and 0.67 at Y5, which indicates modest performance yet superior to what has been reported in the state-of-the-art studies to date .Significance: The MBC can more precisely encode probabilistic relationships between predictors and class variables (Engel outcomes), achieving promising experimental results compared to other well-known statistical methods. Multisite application of the MBC could further optimize its classification accuracy with prospective datasets. Online access to the MBC is provided, paving the way for its use as an adjunct clinical tool in aiding pre-operative TLE surgical counseling.

Many of the data sets extracted from real-world industrial environments are time series that describe dynamic processes with characteristics that change over time. In this paper, we focus on the fouling process in an industrial furnace, which corresponds to a non-stationary multivariate time series with a season al component, non-homogeneous cycles and sporadic human interventions. We aim to forecast the evolution of the temperature inside the furnace over a long span of time of two and a half months. To accomplish this, we model the time series with dynamic Gaussian Bayesian networks (DGBNs) and compare their performance with convolutional recurrent neural networks. Our results show that DGBNs are capable of properly treating seasonal data and can capture the tendency of a time series without being distorted by the effect of interventions or by the varying length of the cycles.

This book describes statistical and machine learning methods to build computational models learned from real-world neuroscience data. These methods cover supervised and unsupervised classification with probabilistic and non-probabilistic models, association discovery with probabilistic graphical models, and spatial statistics with point processes.

En este trabajo se realiza un estudio sobre el machine learning y sus algoritmos más divulgados. En el segundo apartado, se expone una descripción del machine learning. Asimismo, se muestran sus principales hitos históricos y algunas de sus aplicaciones más interesantes. Posteriormente, se explica detalladamente el funcionamiento de los principales algoritmos de aprendizaje automático. En el tercer apartado, se analiza la evolución de las publicaciones y citaciones de estos algoritmos en las revistas científicas con factor de impacto. Se observa que los algoritmos estudiados han tenido un gran crecimiento, especialmente en los últimos 20 años. Cabe destacar el auge de los algoritmos Artificial Neural Network, Support Vector Machine, k-means y Logistic Regression, y de los metaclasificadores, como Random Forest y Boosting. Finalmente, se utiliza el modelo estadístico ARIMA para predecir la evolución de las publicaciones y citaciones de los algoritmos en los próximos 10 años. Los resultados manifiestan un crecimiento esperado significativo de los algoritmos en el mundo académico en el futuro cercano.---ABSTRACT---A study of machine learning and its most popular algorithms is carried out in this work. In the second section, a description of machine learning is presented. Its main historical milestones and some of its most interesting applications are also shown. In addition, some important applications of machine learning algorithms are explained. The third section analyzes the evolution of the publications and citations of these algorithms in the main scientific journals. It is observed that the algorithms studied have had a considerable growth, especially in the last 20 years. The rise of the Artificial Neural Network, Support Vector Machine, kmeans and Logistic Regression algorithms, and meta-classifiers such as Random Forest and Boosting, should be highlighted. Finally, the ARIMA statistical model is used to predict the evolution of publications and citations of th

The sparse Cholesky parametrization of the inverse covariance matrix is directly related to Gaussian Bayesian networks. Its counterpart, the covariance Cholesky factorization model, has a natural interpretation as a hidden variable model for ordered signal data. Despite this, it has received little attention so far, with few notable exceptions. To fill this gap, in this paper we focus on arbitrary zero patterns in the Cholesky factor of a covariance matrix. We discuss how these models can also be extended, in analogy with Gaussian Bayesian networks, to data where no apparent order is available. For the ordered scenario, we propose a novel estimation method that is based on matrix loss penalization, as opposed to the existing regression-based approaches. The performance of this sparse model for the Cholesky factor, together with our novel estimator, is assessed in a simulation setting, as well as over spatial and temporal real data where a natural ordering arises among the variables. We give guidelines, based on the empirical results, about which of the methods analysed is more appropriate for each setting.

Markov models lie at the interface between statistical independence in a probability distribution and graph separation properties. We review model selection and estimation in directed and undirected Markov models with Gaussian parametrization, emphasizing the main similarities and differences. These two model classes are similar but not equivalent, although they share a common intersection. We present the existing results from a historical perspective, taking into account the amount of literature existing from both the artificial intelligence and statistics research communities, where these models were originated. We cover classical topics such as maximum likelihood estimation and model selection via hypothesis testing, but also more modern approaches like regularization and Bayesian methods. We also discuss how the Markov models reviewed fit in the rich hierarchy of other, higher level Markov model classes. Finally, we close the paper overviewing relaxations of the Gaussian assumption and pointing out the main areas of application where these Markov models are nowadays used.

Structure learning methods for covariance and concentration graphs are often validated on synthetic models, usually obtained by randomly generating: (i) an undirected graph, and (ii) a compatible symmetric positive definite (SPD) matrix. In order to ensure positive definiteness in (ii), a dominant diagonal is usually imposed. In this work we investigate different methods to generate random symmetric positive definite matrices with undirected graphical constraints. We show that if the graph is chordal it is possible to sample uniformly from the set of correlation matrices compatible with the graph, while for general undirected graphs we rely on a partial orthogonalization method.

The combination of artificial intelligence with data, computing power, and new algorithms can provide important tools for solving engineering problems, such as machine-tool condition monitoring. However, many of these problems require algorithms that can perform in highly dynamic scenarios where the data stream shave extremely high sampling rates from different types of variables. The unsupervised learning algorithm based on Gaussian mixture models called Gaussian-based dynamic probabilistic clustering (GDPC) is one of these tools. However, this algorithm may have major limitations if a large amount of concept drifts associated with transients occurs within the data stream. GDPC becomes unstable under these conditions, so we propose anew algorithm called GDPC+ to increase its robustness. GDPC+ represents an important improvement because we introduce: (a) automatic selection of the number of mixture components based on the Bayesian information criterion (BIC), and (b) concept drift transition stabilization based on Cauchy–Schwarz divergence integrated with the Dickey–Fuller test. Thus, GDPC+ can perform better in highly dynamic scenarios than GDPC in terms of the number of false positives. The behavior of GDPC+ was investigated using random synthetic data streams and in a real data stream-based condition monitoring obtained from a machine-tool that produces engine crankshafts at high speed. We found that the initial temporal window size can be used to adapt the algorithm to different analytical requirements. The clustering results were also investigated by induction of the rules generated by the repeated incremental pruning to produce error reduction (RIPPER) algorithm in order to provide insights from the underlying monitored process and its associated concept drifts.

Pyramidal neurons are the most common cell type in the cerebral cortex. Understanding how they differ between species is a key challenge in neuroscience. A recent study provided a unique set of human and mouse pyramidal neurons of the CA1 region of the hippocampus, and used it to compare the morphology of apical and basal dendritic branches of the two species. The study found inter-species differences in the magnitude of the morphometrics and similarities regarding their variation with respect to morphological determinants such as branch type and branch order. We use the same data set to perform additional comparisons of basal dendrites. In order to isolate the heterogeneity due to intrinsic differences between species from the heterogeneity due to differences in morphological determinants, we ft multivariate models over the morphometrics and the determinants. In particular, we use conditional linear Gaussian Bayesian networks, which provide a concise graphical representation of the independencies and correlations among the variables. We also extend the previous study by considering additional morphometrics and by formally testing whether a morphometric increases or decreases with the distance from the soma. This study introduces a multivariate methodology for inter-species comparison of morphology.

In the analysis of real-world data, it is useful to learn a latent variable model that represents the data generation process. In this setting, latent tree models are useful because they are able to capture complex relationships while being easily interpretable. In this paper, we propose two incremental algorithms for learning forests of latent trees. Unlike current methods, the proposed algorithms are based on the variational Bayesian framework, which allows them to introduce uncertainty into the learning process and work with mixed data. The first algorithm, incremental learner, determines the forest structure and the cardinality of its latent variables in an iterative search process. The second algorithm, constrained incremental learner, modifies the previous method by considering only a subset of the most prominent structures in each step of the search. Although restricting each iteration to a fixed number of candidate models limits the search space, we demonstrate that the second algorithm returns almost identical results for a small fraction of the computational cost. We compare our algorithms with existing methods by conducting a comparative study using both discrete and continuous real-world data. In addition, we demonstrate the effectiveness of the proposed algorithms by applying them to data from the 2018 Spanish Living Conditions Survey. All code, data, and results are available at https://github.com/ferjorosa/incremental-latent-forests.

To understand the function of cortical circuits, it is necessary to catalog their cellular diversity. Past attempts to do so using anatomical, physiological or molecular features of cortical cells have not resulted in a unified taxonomy of neuronal or glial cell types, partly due to limited data. Single-cell transcriptomics is enabling, for the first time, systematic high-throughput measurements of cortical cells and generation of datasets that hold the promise of being complete, accurate and permanent. Statistical analyses of these data reveal clusters that often correspond to cell types previously defined by morphological or physiological criteria and that appear conserved a cross cortical areas and species. To capitalize on these new methods, we propose the adoption of a transcriptome-based taxonomy of cell types for mammalian neocortex. This classification should be hierarchical and use a standardized nomenclature. It should be based on a probabilistic definition of a cell type and incorporate data from different approaches, developmental stages and species. A community-based classification and data aggregation model, such as a knowledge graph, could provide a common foundation for the study of cortical circuits. This community-based classification, nomenclature and data aggregation could serve as an example for cell type atlases in other parts of the body.

Markov models lie at the interface between statistical independence in a probability distribution and graph separation properties. We review model selection and estimation in directed and undirected Markov models with Gaussian parametrization, emphasizing the main similarities and differences. These two model classes are similar but not equivalent, although they share a common intersection. We present the existing results from a historical perspective, taking into account the amount of literature existing from both the artificial intelligence and statistics research communities, where these models were originated. We cover classical topics such as maximum likelihood estimation and model selection via hypothesis testing, but also more modern approaches like regularization and Bayesian methods. We also discuss how the Markov models reviewed fit in the rich hierarchy of other, higher level Markov model classes. Finally, we close the paper overviewing relaxations of the Gaussian assumption and pointing out the main areas of application where these Markov models are nowadays used.

Structure learning methods for covariance and concentration graphs are often validated on synthetic models, usually obtained by randomly generating: (i) an undirected graph, and (ii) a compatible symmetric positive definite (SPD) matrix. In order to ensure positive definiteness in (ii), a dominant diagonal is usually imposed. In this work we investigate different methods to generate random symmetric positive definite matrices with undirected graphical constraints. We show that if the graph is chordal it is possible to sample uniformly from the set of correlation matrices compatible with the graph, while for general undirected graphs we rely on a partial orthogonalization method.

The sparse Cholesky parametrization of the inverse covariance matrix is directly related to Gaussian Bayesian networks. Its counterpart, the covariance Cholesky factorization model, has a natural interpretation as a hidden variable model for ordered signal data. Despite this, it has received little attention so far, with few notable exceptions. To fill this gap, in this paper we focus on arbitrary zero patterns in the Cholesky factor of a covariance matrix. We discuss how these models can also be extended, in analogy with Gaussian Bayesian networks, to data where no apparent order is available. For the ordered scenario, we propose a novel estimation method that is based on matrix loss penalization, as opposed to the existing regression-based approaches. The performance of this sparse model for the Cholesky factor, together with our novel estimator, is assessed in a simulation setting, as well as over spatial and temporal real data where a natural ordering arises among the variables. We give guidelines, based on the empirical results, about which of the methods analysed is more appropriate for each setting.