Search

Existing deterministic variational inference approaches for diffusion processes use simple proposals and target the marginal density of the posterior. We construct the variational process as a controlled version of the prior process and approximate the posterior by a set of moment functions. In combination with moment closure, the smoothing problem is reduced to a deterministic optimal control problem. Exploiting the path-wise Fisher information, we propose an optimization procedure that corresponds to a natural gradient descent in the variational parameters. Our approach allows for richer variational approximations that extend to state-dependent diffusion terms. The classical Gaussian process approximation is recovered as a special case., Comment: Appearing in Proceedings of the 24th International Conference on Artificial Intelligence and Statistics (AISTATS) 2021, San Diego, California, USA. PMLR: Volume 130

The recent mean field game (MFG) formalism facilitates otherwise intractable computation of approximate Nash equilibria in many-agent settings. In this paper, we consider discrete-time finite MFGs subject to finite-horizon objectives. We show that all discrete-time finite MFGs with non-constant fixed point operators fail to be contractive as typically assumed in existing MFG literature, barring convergence via fixed point iteration. Instead, we incorporate entropy-regularization and Boltzmann policies into the fixed point iteration. As a result, we obtain provable convergence to approximate fixed points where existing methods fail, and reach the original goal of approximate Nash equilibria. All proposed methods are evaluated with respect to their exploitability, on both instructive examples with tractable exact solutions and high-dimensional problems where exact methods become intractable. In high-dimensional scenarios, we apply established deep reinforcement learning methods and empirically combine fictitious play with our approximations., Comment: Accepted to the 24th International Conference on Artificial Intelligence and Statistics (AISTATS 2021); v2: Fixed B.6 and some typos

Quantitatively predictive models of biomolecular circuits are important tools for the design of synthetic biology and molecular communication circuits. The information content of typical time-lapse single-cell data for the inference of kinetic parameters is not only limited by measurement uncertainty and intrinsic stochasticity, but also by the employed perturbations. Novel microfluidic devices enable the synthesis of temporal chemical concentration profiles. The informativeness of a perturbation can be quantified based on mutual information. We propose an approximate method to perform optimal experimental design of such perturbation profiles. To estimate the mutual information we perform a multivariate log-normal approximation of the joint distribution over parameters and observations and scan the design space using Metropolis-Hastings sampling. The method is demonstrated by finding optimal perturbation sequences for synthetic case studies on a gene expression model with varying reporter characteristics., Comment: NanoCom 2018, accepted

A minimal model for gene expression, consisting of a switchable promoter together with the resulting messenger RNA, is equivalent to a Poisson channel with a binary Markovian input process. Determining its capacity is an optimization problem with respect to two parameters: the average sojourn times of the promoter's active (ON) and inactive (OFF) state. An expression for the mutual information is found by solving the associated filtering problem analytically on the level of distributions. For fixed peak power, three bandwidth-like constraints are imposed by lower-bounding (i) the average sojourn times (ii) the autocorrelation time and (iii) the average time until a transition. OFF-favoring optima are found for all three constraints, as commonly encountered for the Poisson channel. In addition, constraint (i) exhibits a region that favors the ON state, and (iii) shows ON-favoring local optima., Comment: This article was accepted for publication by IEEE, ISIT 2020

Detecting and segmenting object instances is a common task in biomedical applications. Examples range from detecting lesions on functional magnetic resonance images, to the detection of tumours in histopathological images and extracting quantitative single-cell information from microscopy imagery, where cell segmentation is a major bottleneck. Attention-based transformers are state-of-the-art in a range of deep learning fields. They have recently been proposed for segmentation tasks where they are beginning to outperforming other methods. We present a novel attention-based cell detection transformer (Cell-DETR) for direct end-to-end instance segmentation. While the segmentation performance is on par with a state-of-the-art instance segmentation method, Cell-DETR is simpler and faster. We showcase the method's contribution in a the typical use case of segmenting yeast in microstructured environments, commonly employed in systems or synthetic biology. For the specific use case, the proposed method surpasses the state-of-the-art tools for semantic segmentation and additionally predicts the individual object instances. The fast and accurate instance segmentation performance increases the experimental information yield for a posteriori data processing and makes online monitoring of experiments and closed-loop optimal experimental design feasible., Comment: IEEE BIBM 2020 (accepted)

Cell segmentation is a major bottleneck in extracting quantitative single-cell information from microscopy data. The challenge is exasperated in the setting of microstructured environments. While deep learning approaches have proven useful for general cell segmentation tasks, existing segmentation tools for the yeast-microstructure setting rely on traditional machine learning approaches. Here we present convolutional neural networks trained for multiclass segmenting of individual yeast cells and discerning these from cell-similar microstructures. We give an overview of the datasets recorded for training, validating and testing the networks, as well as a typical use-case. We showcase the method's contribution to segmenting yeast in microstructured environments with a typical synthetic biology application in mind. The models achieve robust segmentation results, outperforming the previous state-of-the-art in both accuracy and speed. The combination of fast and accurate segmentation is not only beneficial for a posteriori data processing, it also makes online monitoring of thousands of trapped cells or closed-loop optimal experimental design feasible from an image processing perspective., Comment: IEEE CIBCB 2020 (accepted)

Many processes, such as discrete event systems in engineering or population dynamics in biology, evolve in discrete space and continuous time. We consider the problem of optimal decision making in such discrete state and action space systems under partial observability. This places our work at the intersection of optimal filtering and optimal control. At the current state of research, a mathematical description for simultaneous decision making and filtering in continuous time with finite state and action spaces is still missing. In this paper, we give a mathematical description of a continuous-time partial observable Markov decision process (POMDP). By leveraging optimal filtering theory we derive a Hamilton-Jacobi-Bellman (HJB) type equation that characterizes the optimal solution. Using techniques from deep learning we approximately solve the resulting partial integro-differential equation. We present (i) an approach solving the decision problem offline by learning an approximation of the value function and (ii) an online algorithm which provides a solution in belief space using deep reinforcement learning. We show the applicability on a set of toy examples which pave the way for future methods providing solutions for high dimensional problems., Comment: published at Conference on Neural Information Processing Systems (NeurIPS) 2020

We analyze a data-processing system with $n$ clients producing jobs which are processed in \textit{batches} by $m$ parallel servers; the system throughput critically depends on the batch size and a corresponding sub-additive speedup function. In practice, throughput optimization relies on numerical searches for the optimal batch size, a process that can take up to multiple days in existing commercial systems. In this paper, we model the system in terms of a closed queueing network; a standard Markovian analysis yields the optimal throughput in $\omega\left(n^4\right)$ time. Our main contribution is a mean-field model of the system for the regime where the system size is large. We show that the mean-field model has a unique, globally attractive stationary point which can be found in closed form and which characterizes the asymptotic throughput of the system as a function of the batch size. Using this expression we find the \textit{asymptotically} optimal throughput in $O(1)$ time. Numerical settings from a large commercial system reveal that this asymptotic optimum is accurate in practical finite regimes., Comment: 15 pages

We study active matter systems where the orientational dynamics of underlying self-propelled particles obey second order equations. By primarily concentrating on a spatially homogeneous setup for particle distribution, our analysis combines theories of active matter and oscillatory networks. For such systems, we analyze the appearance of solitary states via a homoclinic bifurcation as a mechanism of the frequency clustering. By introducing noise, we establish a stochastic version of solitary states and derive the mean-field limit described by a partial differential equation for a one-particle probability density function, which one might call the continuum Kuramoto model with inertia and noise. By studying this limit, we establish second order phase transitions between polar order and disorder. The combination of both analytical and numerical approaches in our study demonstrates an excellent qualitative agreement between mean-field and finite size models.

The continuum description of active particle systems is an efficient instrument to analyze a finite size particle dynamics in the limit of a large number of particles. However, it is often the case that such equations appear as nonlinear integro-differential equations and purely analytical treatment becomes quite limited. We propose a general framework of finite volume methods (FVMs) to numerically solve partial differential equations (PDEs) of the continuum limit of nonlocally interacting chiral active particle systems confined to two dimensions. We demonstrate the performance of the method on spatially homogeneous problems, where the comparison to analytical results is available, and on general spatially inhomogeneous equations, where pattern formation is predicted by kinetic theory. We numerically investigate phase transitions of particular problems in both spatially homogeneous and inhomogeneous regimes and report the existence of different first and second order transitions.

Structured stochastic processes evolving in continuous time present a widely adopted framework to model phenomena occurring in nature and engineering. However, such models are often chosen to satisfy the Markov property to maintain tractability. One of the more popular of such memoryless models are Continuous Time Bayesian Networks (CTBNs). In this work, we lift its restriction to exponential survival times to arbitrary distributions. Current extensions achieve this via auxiliary states, which hinder tractability. To avoid that, we introduce a set of node-wise clocks to construct a collection of graph-coupled semi-Markov chains. We provide algorithms for parameter and structure inference, which make use of local dependencies and conduct experiments on synthetic data and a data-set generated through a benchmark tool for gene regulatory networks. In doing so, we point out advantages compared to current CTBN extensions., Comment: 10 main pages, 7 pages appendix, 3 figures, proceedings of ICML2020

We propose a novel probabilistic framework to model continuous-time interaction events data. Our goal is to infer the \emph{implicit} community structure underlying the temporal interactions among entities, and also to exploit how the community structure influences the interaction dynamics among these nodes. To this end, we model the reciprocating interactions between individuals using mutually-exciting Hawkes processes. The base rate of the Hawkes process for each pair of individuals is built upon the latent representations inferred using the hierarchical gamma process edge partition model (HGaP-EPM). In particular, our model allows the interaction dynamics between each pair of individuals to be modulated by their respective affiliated communities. Moreover, our model can flexibly incorporate the auxiliary individuals' attributes, or covariates associated with interaction events. Efficient Gibbs sampling and Expectation-Maximization algorithms are developed to perform inference via P\'olya-Gamma data augmentation strategy. Experimental results on real-world datasets demonstrate that our model not only achieves competitive performance for temporal link prediction compared with state-of-the-art methods, but also discovers interpretable latent structure behind the observed temporal interactions.

We consider stochastic dynamics of self-propelled particles with nonlocal normalized alignment interactions subject to phase lag. The role of the lag is to indirectly generate chirality into particle motion. To understand large scale behavior, we derive a continuum description of an active Brownian particle (ABP) flow with macroscopic scaling in the form of a partial differential equation (PDE) for a one-particle probability density function (DF). Due to indirect chirality, we find a new spatially homogeneous nonstationary analytic solution for this class of equations. Our development of kinetic and hydrodynamic theories towards such a solution reveals the existence of a wide variety of spatially nonhomogeneous patterns reminiscent of the traveling bands, clouds, and vortical structures of linear active matter. Our model may thereby serve as the basis for understanding the nature of chiral active media and designing multiagent swarms with designated behavior.

Coordinating multiple interacting agents to achieve a common goal is a difficult task with huge applicability. This problem remains hard to solve, even when limiting interactions to be mediated via a static interaction-graph. We present a novel approximate solution method for multi-agent Markov decision problems on graphs, based on variational perturbation theory. We adopt the strategy of planning via inference, which has been explored in various prior works. We employ a non-trivial extension of a novel high-order variational method that allows for approximate inference in large networks and has been shown to surpass the accuracy of existing variational methods. To compare our method to two state-of-the-art methods for multi-agent planning on graphs, we apply the method different standard GMDP problems. We show that in cases, where the goal is encoded as a non-local cost function, our method performs well, while state-of-the-art methods approach the performance of random guess. In a final experiment, we demonstrate that our method brings significant improvement for synchronization tasks.

Many decision-making problems naturally exhibit pronounced structures inherited from the characteristics of the underlying environment. In a Markov decision process model, for example, two distinct states can have inherently related semantics or encode resembling physical state configurations. This often implies locally correlated transition dynamics among the states. In order to complete a certain task in such environments, the operating agent usually needs to execute a series of temporally and spatially correlated actions. Though there exists a variety of approaches to capture these correlations in continuous state-action domains, a principled solution for discrete environments is missing. In this work, we present a Bayesian learning framework based on P\'olya-Gamma augmentation that enables an analogous reasoning in such cases. We demonstrate the framework on a number of common decision-making related problems, such as imitation learning, subgoal extraction, system identification and Bayesian reinforcement learning. By explicitly modeling the underlying correlation structures of these problems, the proposed approach yields superior predictive performance compared to correlation-agnostic models, even when trained on data sets that are an order of magnitude smaller in size., Comment: 15 pages, 4 figures

Continuous-time Bayesian Networks (CTBNs) represent a compact yet powerful framework for understanding multivariate time-series data. Given complete data, parameters and structure can be estimated efficiently in closed-form. However, if data is incomplete, the latent states of the CTBN have to be estimated by laboriously simulating the intractable dynamics of the assumed CTBN. This is a problem, especially for structure learning tasks, where this has to be done for each element of a super-exponentially growing set of possible structures. In order to circumvent this notorious bottleneck, we develop a novel gradient-based approach to structure learning. Instead of sampling and scoring all possible structures individually, we assume the generator of the CTBN to be composed as a mixture of generators stemming from different structures. In this framework, structure learning can be performed via a gradient-based optimization of mixture weights. We combine this approach with a new variational method that allows for a closed-form calculation of this mixture marginal likelihood. We show the scalability of our method by learning structures of previously inaccessible sizes from synthetic and real-world data., Comment: Accepted at NeurIPS2019

We propose moment-based variational inference as a flexible framework for approximate smoothing of latent Markov jump processes. The main ingredient of our approach is to partition the set of all transitions of the latent process into classes. This allows to express the Kullback-Leibler divergence between the approximate and the exact posterior process in terms of a set of moment functions that arise naturally from the chosen partition. To illustrate possible choices of the partition, we consider special classes of jump processes that frequently occur in applications. We then extend the results to parameter inference and demonstrate the method on several examples., Comment: Accepted by the 36th International Conference on Machine Learning (ICML 2019)

Recent advances in quality adaptation algorithms leave adaptive bitrate (ABR) streaming architectures at a crossroads: When determining the sustainable video quality one may either rely on the information gathered at the client vantage point or on server and network assistance. The fundamental problem here is to determine how valuable either information is for the adaptation decision. This problem becomes particularly hard in future Internet settings such as Named Data Networking (NDN) where the notion of a network connection does not exist. In this paper, we provide a fresh view on ABR quality adaptation for QoE maximization, which we formalize as a decision problem under uncertainty, and for which we contribute a sparse Bayesian contextual bandit algorithm denoted CBA. This allows taking high-dimensional streaming context information, including client-measured variables and network assistance, to find online the most valuable information for the quality adaptation. Since sparse Bayesian estimation is computationally expensive, we develop a fast new inference scheme to support online video adaptation. We perform an extensive evaluation of our adaptation algorithm in the particularly challenging setting of NDN, where we use an emulation testbed to demonstrate the efficacy of CBA compared to state-of-the-art algorithms.

Due to missing IP multicast support on an Internet scale, over-the-top media streams are delivered with the help of overlays as used by content delivery networks and their peer-to-peer (P2P) extensions. In this context, mesh/pull-based swarming plays an important role either as pure streaming approach or in combination with tree/push mechanisms. However, the impact of realistic client populations with heterogeneous resources is not yet fully understood. In this technical report, we contribute to closing this gap by mathematically analysing the most basic scheduling mechanisms latest deadline first (LDF) and earliest deadline first (EDF) in a continuous time Markov chain framework and combining them into a simple, yet powerful, mixed strategy to leverage inherent differences in client resources. The main contributions are twofold: (1) a mathematical framework for swarming on random graphs is proposed with a focus on LDF and EDF strategies in heterogeneous scenarios; (2) a mixed strategy, named SchedMix, is proposed that leverages peer heterogeneity. The proposed strategy, SchedMix is shown to outperform the other two strategies using different abstractions: a mean-field theoretic analysis of buffer probabilities, simulations of a stochastic model on random graphs, and a full-stack implementation of a P2P streaming system., Comment: Technical report and supplementary material to http://ieeexplore.ieee.org/document/7497234/

Synthetic biology aims at designing modular genetic circuits that can be assembled according to the desired function. When embedded in a cell, a circuit module becomes a small subnetwork within a larger environmental network, and its dynamics is therefore affected by potentially unknown interactions with the environment. It is well-known that the presence of the environment not only causes extrinsic noise but also memory effects, which means that the dynamics of the subnetwork is affected by its past states via a memory function that is characteristic of the environment. We study several generic scenarios for the coupling between a small module and a larger environment, with the environment consisting of a chain of mono-molecular reactions. By mapping the dynamics of this coupled system onto random walks, we are able to give exact analytical expressions for the arising memory functions. Hence, our results give insights into the possible types of memory functions and thereby help to better predict subnetwork dynamics., Comment: 14 pages, 6 figures Accepted Version

Continuous-time Bayesian networks (CTBNs) constitute a general and powerful framework for modeling continuous-time stochastic processes on networks. This makes them particularly attractive for learning the directed structures among interacting entities. However, if the available data is incomplete, one needs to simulate the prohibitively complex CTBN dynamics. Existing approximation techniques, such as sampling and low-order variational methods, either scale unfavorably in system size, or are unsatisfactory in terms of accuracy. Inspired by recent advances in statistical physics, we present a new approximation scheme based on cluster-variational methods significantly improving upon existing variational approximations. We can analytically marginalize the parameters of the approximate CTBN, as these are of secondary importance for structure learning. This recovers a scalable scheme for direct structure learning from incomplete and noisy time-series data. Our approach outperforms existing methods in terms of scalability., Comment: Accepted at NIPS 2018

Group factor analysis (GFA) methods have been widely used to infer the common structure and the group-specific signals from multiple related datasets in various fields including systems biology and neuroimaging. To date, most available GFA models require Gibbs sampling or slice sampling to perform inference, which prevents the practical application of GFA to large-scale data. In this paper we present an efficient collapsed variational inference (CVI) algorithm for the nonparametric Bayesian group factor analysis (NGFA) model built upon an hierarchical beta Bernoulli process. Our CVI algorithm proceeds by marginalizing out the group-specific beta process parameters, and then approximating the true posterior in the collapsed space using mean field methods. Experimental results on both synthetic and real-world data demonstrate the effectiveness of our CVI algorithm for the NGFA compared with state-of-the-art GFA methods.

Cellular reactions have multi-scale nature in the sense that the abundance of molecular species and the magnitude of reaction rates can vary in a wide range. This diversity leads to hybrid models that combine deterministic and stochastic modeling approaches. To reveal this multi-scale nature, we proposed jump-diffusion approximation in a previous study. The key idea behind the model was to partition reactions into fast and slow groups, and then to combine Markov chain updating scheme for the slow set with diffusion (Langevin) approach updating scheme for the fast set. Then, the state vector of the model was defined as the summation of the random time change model and the solution of the Langevin equation. In this study, we have proved that the joint probability density function of the jump-diffusion approximation over the reaction counting process satisfies the hybrid master equation, which is the summation of the chemical master equation and the Fokker-Planck equation. To solve the hybrid master equation, we propose an algorithm using the moments of reaction counters of fast reactions given the reaction counters of slow reactions. Then, we solve a constrained optimization problem for each conditional probability density at the time point of interest utilizing the maximum entropy approach. Based on the multiplication rule for joint probability density functions, we construct the solution of the hybrid master equation. To show the efficiency of the method, we implement it to a canonical model of gene regulation.

In this short technical note, we extend a recently published result [Liao2017] on the Perron root (or the spectral radius) of non-negative matrices to real-valued non-negative kernels on an arbitrary measurable space $(\mathrm{E}, \mathcal{E})$. To be precise, for any real-valued non-negative kernel $K : \mathrm{E}\times \mathcal{E} \rightarrow \mathbb{R}$, we prove that the spectral radius $\rho(K)$ of $K$ satisfies $$ \inf_{x \in \mathrm{E} } \frac{ \mathcal{R} K \cdotp L (x) }{ \mathcal{R} L (x) } \le \rho(K) \le \sup_{x \in \mathrm{E} } \frac{ \mathcal{R} K\cdotp L (x) }{ \mathcal{R} L (x) }, $$ where $L$ is an arbitrary Kernel on $(\mathrm{E}, \mathcal{E})$, which is integrable with respect to the left eigenmeasure of $K$ and satisfies $ \mathcal{R} L (x) >0 $ for all $x \in \mathrm{E}$, and the operator $\mathcal{R}$ is defined by $\mathcal{R}L (x) :=\int_{\mathrm{E}} L(x, \mathrm{d}y) $., Comment: 7 pages, no figures, technical note

We present a probabilistic model for learning from dynamic relational data, wherein the observed interactions among networked nodes are modeled via the Bernoulli Poisson link function, and the underlying network structure are characterized by nonnegative latent node-group memberships, which are assumed to be gamma distributed. The latent memberships evolve according to Markov processes. The optimal number of latent groups can be determined by data itself. The computational complexity of our method scales with the number of non-zero links, which makes it scalable to large sparse dynamic relational data. We present batch and online Gibbs sampling algorithms to perform model inference. Finally, we demonstrate the model's performance on both synthetic and real-world datasets compared to state-of-the-art methods.

Markov jump process models have many applications across science. Often, these models are defined on a state-space of product form and only one of the components of the process is of direct interest. In this paper, we extend the marginal process framework, which provides a marginal description of the component of interest, to the case of fully coupled processes. We use entropic matching to obtain a finite-dimensional approximation of the filtering equation, which governs the transition rates of the marginal process. The resulting equations can be seen as a combination of two projection operations applied to the full master equation, so that we obtain a principled model reduction framework. We demonstrate the resulting reduced description on the totally asymmetric exclusion process. An important class of Markov jump processes are stochastic reaction networks, which have applications in chemical and biomolecular kinetics, ecological models and models of social networks. We obtain a particularly simple instantiation of the marginal process framework for mass-action systems by using product-Poisson distributions for the approximate solution of the filtering equation. We investigate the resulting approximate marginal process analytically and numerically., Comment: 16 pages, 5 figures; accepted for publication in Physical Review E

We study a Markovian agent-based model (MABM) in this paper. Each agent is endowed with a local state that changes over time as the agent interacts with its neighbours. The neighbourhood structure is given by a graph. In a recent paper [Simon et al. 2011], the authors used the automorphisms of the underlying graph to generate a lumpable partition of the joint state space ensuring Markovianness of the lumped process for binary dynamics. However, many large random graphs tend to become asymmetric rendering the automorphism-based lumping approach ineffective as a tool of model reduction. In order to mitigate this problem, we propose a lumping method based on a notion of local symmetry, which compares only local neighbourhoods of vertices. Since local symmetry only ensures approximate lumpability, we quantify the approximation error by means of Kullback-Leibler divergence rate between the original Markov chain and a lifted Markov chain. We prove the approximation error decreases monotonically. The connections to fibrations of graphs are also discussed., Comment: 28 pages, 4 figures

Advances in the field of inverse reinforcement learning (IRL) have led to sophisticated inference frameworks that relax the original modeling assumption of observing an agent behavior that reflects only a single intention. Instead of learning a global behavioral model, recent IRL methods divide the demonstration data into parts, to account for the fact that different trajectories may correspond to different intentions, e.g., because they were generated by different domain experts. In this work, we go one step further: using the intuitive concept of subgoals, we build upon the premise that even a single trajectory can be explained more efficiently locally within a certain context than globally, enabling a more compact representation of the observed behavior. Based on this assumption, we build an implicit intentional model of the agent's goals to forecast its behavior in unobserved situations. The result is an integrated Bayesian prediction framework that significantly outperforms existing IRL solutions and provides smooth policy estimates consistent with the expert's plan. Most notably, our framework naturally handles situations where the intentions of the agent change over time and classical IRL algorithms fail. In addition, due to its probabilistic nature, the model can be straightforwardly applied in active learning scenarios to guide the demonstration process of the expert., Comment: 45 pages, 14 figures; ### Version 3 ### published in the Journal of Machine Learning Research

Collaborative uploading describes a type of crowdsourcing scenario in networked environments where a device utilizes multiple paths over neighboring devices to upload content to a centralized processing entity such as a cloud service. Intermediate devices may aggregate and preprocess this data stream. Such scenarios arise in the composition and aggregation of information, e.g., from smartphones or sensors. We use a queuing theoretic description of the collaborative uploading scenario, capturing the ability to split data into chunks that are then transmitted over multiple paths, and finally merged at the destination. We analyze replication and allocation strategies that control the mapping of data to paths and provide closed-form expressions that pinpoint the optimal strategy given a description of the paths' service distributions. Finally, we provide an online path-aware adaptation of the allocation strategy that uses statistical inference to sequentially minimize the expected waiting time for the uploaded data. Numerical results show the effectiveness of the adaptive approach compared to the proportional allocation and a variant of the join-the-shortest-queue allocation, especially for bursty path conditions., Comment: 15 pages, 11 figures, extended version of a conference paper accepted for publication in the Proceedings of the IEEE International Conference on Computer Communications (INFOCOM), 2018

In this paper we derive several quasi steady-state approximations (QSSAs) to the stochastic reaction network describing the Michaelis-Menten enzyme kinetics. We show how the different assumptions about chemical species abundance and reaction rates lead to the standard QSSA (sQSSA), the total QSSA (tQSSA), and the reverse QSSA (rQSSA) approximations. These three QSSAs have been widely studied in the literature in deterministic ordinary differential equation (ODE) settings and several sets of conditions for their validity have been proposed. By using multiscaling techniques introduced in Kang (and Kurtz 2013) and Ball et al. (2006) we show that these conditions for deterministic QSSAs largely agree with the ones for QSSAs in the large volume limits of the underlying stochastic enzyme kinetic network.

Genome-scale gene networks contain regulatory genes called hubs that have many interaction partners. These genes usually play an essential role in gene regulation and cellular processes. Despite recent advancements in high-throughput technology, inferring gene networks with hub genes from high-dimensional data still remains a challenging problem. Novel statistical network inference methods are needed for efficient and accurate reconstruction of hub networks from high-dimensional data. To address this challenge we propose DW-Lasso, a degree weighted Lasso (least absolute shrinkage and selection operator) method which infers gene networks with hubs efficiently under the low sample size setting. Our network reconstruction approach is formulated as a two stage procedure: first, the degree of networks is estimated iteratively, and second, the gene regulatory network is reconstructed using degree information. A useful property of the proposed method is that it naturally favors the accumulation of neighbors around hub genes and thereby helps in accurate modeling of the high-throughput data under the assumption that the underlying network exhibits hub structure. In a simulation study, we demonstrate good predictive performance of the proposed method in comparison to traditional Lasso type methods in inferring hub and scale-free graphs. We show the effectiveness of our method in an application to microarray data of \textit{E.coli} and RNA sequencing data of Kidney Clear Cell Carcinoma from The Cancer Genome Atlas datasets., Comment: 15 pages, 8 figures. Submitted to Bioinformatics

Approximate solutions of the chemical master equation and the chemical Fokker-Planck equation are an important tool in the analysis of biomolecular reaction networks. Previous studies have highlighted a number of problems with the moment-closure approach used to obtain such approximations, calling it an ad-hoc method. In this article, we give a new variational derivation of moment-closure equations which provides us with an intuitive understanding of their properties and failure modes and allows us to correct some of these problems. We use mixtures of product-Poisson distributions to obtain a flexible parametric family which solves the commonly observed problem of divergences at low system sizes. We also extend the recently introduced entropic matching approach to arbitrary ansatz distributions and Markov processes, demonstrating that it is a special case of variational moment closure. This provides us with a particularly principled approximation method. Finally, we extend the above approaches to cover the approximation of multi-time joint distributions, resulting in a viable alternative to process-level approximations which are often intractable., Comment: Minor changes and clarifications; corrected some typos

We study a stochastic compartmental susceptible-infected (SI) epidemic process on a configuration model random graph with a given degree distribution over a finite time interval $[0,T],$ for some $ T>0$. In this setting, we split the population of graph nodes into two compartments, namely, $S$ and $I$, denoting the susceptible and infected nodes, respectively. In addition to the sizes of these two compartments, we study counts of $SI$-edges (those connecting a susceptible and an infected node) and $SS$-edges (those connecting two susceptible nodes). We describe the dynamical process in terms of these counts and present a functional central limit theorem (FCLT) for them, a scaling limit of the dynamical process as $n$, the number of nodes in the random graph, grows to infinity. To be precise, we show that these counts, when appropriately scaled, converge weakly to a continuous Gaussian vector martingale process the usual Skorohod space of real 3-dimensional vector-valued \cadlag\, functions on $[0,T]$ endowed with the Skorohod topology. We assume certain technical requirements for this purpose. We discuss applications of our FCLT in percolation theory (from a non-equilibrium statistical mechanics point of view), and in computer science in the context of spread of computer viruses. We also provide simulation results for some common degree distributions., Comment: 52 pages, 6 figures

Frameworks, such as MapReduce and Hadoop are abundant nowadays. They seek to reap benefits of parallelization, albeit subject to a synchronization constraint at the output. Fork-Join (FJ) queuing models are used to analyze such systems. Arriving jobs are split into tasks each of which is mapped to exactly one server. A job leaves the system when all of its tasks are executed. As a metric of performance, we consider waiting times for both work-conserving and non-work conserving server systems under a mathematical set-up general enough to take into account possible phase-type behavior of the servers, and as suggested by recent evidences, bursty arrivals. To this end, we present a Markov-additive process framework for an FJ system and provide computable bounds on tail probabilities of steady-state waiting times, for both types of servers separately. We apply our results to three scenarios, namely, non-renewal (Markov-modulated) arrivals, servers showing phase-type behavior, and Markov-modulated arrivals and services. We compare our bounds against estimates obtained through simulations and also provide a theoretical conceptualization of provisions in FJ systems. Finally, we calibrate our model with real data traces, and illustrate how our bounds can be used to devise provisions.

Fork-Join (FJ) queueing models capture the dynamics of system parallelization under synchronization constraints, for example, for applications such as MapReduce, multipath transmission and RAID systems. Arriving jobs are first split into tasks and mapped to servers for execution, such that a job can only leave the system when all of its tasks are executed. In this paper, we provide computable stochastic bounds for the waiting and response time distributions for heterogeneous FJ systems under general parallelization benefit. Our main contribution is a generalized mathematical framework for probabilistic server scheduling strategies that are essentially characterized by a probability distribution over the number of utilized servers, and the optimization thereof. We highlight the trade-off between the scaling benefit due to parallelization and the FJ inherent synchronization penalty. Further, we provide optimal scheduling strategies for arbitrary scaling regimes that map to different levels of parallelization benefit. One notable insight obtained from our results is that different applications with varying parallelization benefits result in different optimal strategies. Finally, we complement our analytical results by applying them to various applications showing the optimality of the proposed scheduling strategies., Comment: 16 pages, 8 figures

Learning from demonstration (LfD) is the process of building behavioral models of a task from demonstrations provided by an expert. These models can be used e.g. for system control by generalizing the expert demonstrations to previously unencountered situations. Most LfD methods, however, make strong assumptions about the expert behavior, e.g. they assume the existence of a deterministic optimal ground truth policy or require direct monitoring of the expert's controls, which limits their practical use as part of a general system identification framework. In this work, we consider the LfD problem in a more general setting where we allow for arbitrary stochastic expert policies, without reasoning about the optimality of the demonstrations. Following a Bayesian methodology, we model the full posterior distribution of possible expert controllers that explain the provided demonstration data. Moreover, we show that our methodology can be applied in a nonparametric context to infer the complexity of the state representation used by the expert, and to learn task-appropriate partitionings of the system state space., Comment: 17 pages, 8 figures; ### Version 4 ### to appear in IEEE Transactions on Pattern Analysis and Machine Intelligence

Inverse reinforcement learning (IRL) has become a useful tool for learning behavioral models from demonstration data. However, IRL remains mostly unexplored for multi-agent systems. In this paper, we show how the principle of IRL can be extended to homogeneous large-scale problems, inspired by the collective swarming behavior of natural systems. In particular, we make the following contributions to the field: 1) We introduce the swarMDP framework, a sub-class of decentralized partially observable Markov decision processes endowed with a swarm characterization. 2) Exploiting the inherent homogeneity of this framework, we reduce the resulting multi-agent IRL problem to a single-agent one by proving that the agent-specific value functions in this model coincide. 3) To solve the corresponding control problem, we propose a novel heterogeneous learning scheme that is particularly tailored to the swarm setting. Results on two example systems demonstrate that our framework is able to produce meaningful local reward models from which we can replicate the observed global system dynamics., Comment: 9 pages, 8 figures; ### Version 2 ### version accepted at AAMAS 2017

The synchronization of self-propelled particles (SPPs) is a fascinating instance of emergent behavior in living and man-made systems, such as colonies of bacteria, flocks of birds, robot ensembles, and many others. The recent discovery of chimera states in coupled oscillators opens up new perspectives and indicates that other emergent behaviors may exist for SPPs. Indeed, for a minimal extension of the classical Vicsek model we show the existence of chimera states for SPPs, i.e., one group of particles synchronizes while others wander around chaotically. Compared to chimeras in coupled oscillators where the site position is fixed, SPPs give rise to new distinctive forms of chimeric behavior. We emphasize that the found behavior is directly implied by the structure of the deterministic equation of motion and is not caused by exogenous stochastic excitation. In the scaling limit of infinitely many particles, we show that the chimeric state persists. Our findings provide the starting point for the search or elicitation of chimeric states in real world SPP systems.