Search

One of the key objectives in many fields in machine learning is to discover causal relationships among a set of variables from observational data. In linear non-Gaussian acyclic models (LiNGAM), it can be shown that the true underlying causal structure can be identified uniquely from merely observational data. The DirectLiNGAM algorithm is a well-known solution to learn the true causal structure in a high dimensional setting. DirectLiNGAM algorithm executes in a sequence of iterations and it performs a set of comparisons between pairs of variables in each iteration. Unfortunately, the runtime of this algorithm grows significantly as the number of variables increases. In this paper, we propose a parallel algorithm, called ParaLiNGAM, to learn casual structures based on DirectLiNGAM algorithm. We propose a threshold mechanism that can reduce the number of comparisons remarkably compared with the sequential solution. Moreover, in order to further reduce runtime, we employ a messaging mechanism between workers. We also present an implementation of ParaLiNGAM on GPU, considering hardware constraints. Experimental results on synthetic and real data show that our proposed solution outperforms DirectLiNGAM by a factor up to 4788X, and by a median of 2344X. • A scalable algorithm for learning causal structures in linear non-Gaussian models. • Significant speedup by selecting the necessary computations judiciously. • Speedup ratio of up to 4788 on GPUs in real-world and synthetic datasets. [ABSTRACT FROM AUTHOR]

We study experiment design for unique identification of the causal graph of a simple SCM, where the graph may contain cycles. The presence of cycles in the structure introduces major challenges for experiment design as, unlike acyclic graphs, learning the skeleton of causal graphs with cycles may not be possible from merely the observational distribution. Furthermore, intervening on a variable in such graphs does not necessarily lead to orienting all the edges incident to it. In this paper, we propose an experiment design approach that can learn both cyclic and acyclic graphs and hence, unifies the task of experiment design for both types of graphs. We provide a lower bound on the number of experiments required to guarantee the unique identification of the causal graph in the worst case, showing that the proposed approach is order-optimal in terms of the number of experiments up to an additive logarithmic term. Moreover, we extend our result to the setting where the size of each experiment is bounded by a constant. For this case, we show that our approach is optimal in terms of the size of the largest experiment required for uniquely identifying the causal graph in the worst case. [ABSTRACT FROM AUTHOR]

We study the problem of experiment design to learn causal structures from interventional data. We consider an active learning setting in which the experimenter decides to intervene on one of the variables in the system in each step and uses the results of the intervention to recover further causal relationships among the variables. The goal is to fully identify the causal structures with minimum number of interventions. We present the first deep reinforcement learning based solution for the problem of experiment design. In the proposed method, we embed input graphs to vectors using a graph neural network and feed them to another neural network which outputs a variable for performing intervention in each step. Both networks are trained jointly via a Q-iteration algorithm. Experimental results show that the proposed method achieves competitive performance in recovering causal structures with respect to previous works, while significantly reducing execution time in dense graphs. • Active learning of causal structures by training an agent with a deep reinforcement learning algorithm. • The agent is trained by jointly optimizing an embedding network and a score network. • The proposed solution achieves competitive performance compared with previous work while reducing execution time. [ABSTRACT FROM AUTHOR]

We propose two nonlinear regression methods, namely, Adversarial Orthogonal Regression (AdOR) for additive noise models and Adversarial Orthogonal Structural Equation Model (AdOSE) for the general case of structural equation models. Both methods try to make the residual of regression independent from regressors, while putting no assumption on noise distribution. In both methods, two adversarial networks are trained simultaneously where a regression network outputs predictions and a loss network that estimates mutual information (in AdOR) and KL-divergence (in AdOSE). These methods can be formulated as a minimax two-player game; at equilibrium, AdOR finds a deterministic map between inputs and output and estimates mutual information between residual and inputs, while AdOSE estimates a conditional probability distribution of output given inputs. The proposed methods can be used as subroutines to address several learning problems in causality, such as causal direction determination (or more generally, causal structure learning) and causal model estimation. Experimental results on both synthetic and real-world data demonstrate that the proposed methods have remarkable performance with respect to previous solutions. • Two adversarial regression methods making the residual statistically independent of regressors in models with unknown distributions. • Both methods can be utilized in several causal inference tasks by measuring the dependence between residual and regressors. • Proposed regression methods obtain the best fitted curves compared to previous methods in terms of Integral Squared Error (ISE). [ABSTRACT FROM AUTHOR]

Given a social network modeled as a weighted graph $G$ G , the influence maximization problem seeks $k$ k vertices to become initially influenced, to maximize the expected number of influenced nodes under a particular diffusion model. The influence maximization problem has been proven to be NP-hard, and most proposed solutions to the problem are approximate greedy algorithms, which can guarantee a tunable approximation ratio for their results with respect to the optimal solution. The state-of-the-art algorithms are based on Reverse Influence Sampling (RIS) technique, which can offer both computational efficiency and non-trivial $(1-{1}/{e}-\epsilon)$ (1 - 1 / e - ε) -approximation ratio guarantee for any $\epsilon >0$ ε > 0 . RIS-based algorithms, despite their lower computational cost compared to other methods, still require long running times to solve the problem in large-scale graphs with low values of $\epsilon$ ε . In this article, we present a novel and efficient parallel implementation of a RIS-based algorithm, namely IMM, on GPU. The proposed GPU-accelerated influence maximization algorithm, named gIM, can significantly reduce the running time on large-scale graphs with low values of $\epsilon$ ε . Furthermore, we show that gIM algorithm can solve other variations of the IM problem, only by applying minor modifications. Experimental results show that the proposed solution reduces the runtime by a factor up to 220 ×. The source code of gIM is publicly available online. [ABSTRACT FROM AUTHOR]

We consider distributed statistical optimization in one-shot setting, where there are m machines each observing n i.i.d. samples. Based on its observed samples, each machine sends a B-bit-long message to a server. The server then collects messages from all machines, and estimates a parameter that minimizes an expected convex loss function. We investigate the impact of communication constraint, B, on the expected error and derive a tight lower bound on the error achievable by any algorithm. We then propose an estimator, which we call Multi-Resolution Estimator (MRE), whose expected error (when B = d logmn where d is the dimension of parameter) meets the aforementioned lower bound up to a poly-logarithmic factor in mn. The expected error of MRE, unlike existing algorithms, tends to zero as the number of machines (m) goes to infinity, even when the number of samples per machine (n) remains upper bounded by a constant. We also address the problem of learning under tiny communication budget, and present lower and upper error bounds for the case that the budget B is a constant. [ABSTRACT FROM AUTHOR]

We consider the problem of learning causal models from observational data generated by linear non-Gaussian acyclic causal models with latent variables. Without considering the effect of latent variables, the inferred causal relationships among the observed variables are often wrong. Under faithfulness assumption, we propose a method to check whether there exists a causal path between any two observed variables. From this information, we can obtain the causal order among the observed variables. The next question is whether the causal effects can be uniquely identified as well. We show that causal effects among observed variables cannot be identified uniquely under mere assumptions of faithfulness and non-Gaussianity of exogenous noises. However, we are able to propose an efficient method that identifies the set of all possible causal effects that are compatible with the observational data. We present additional structural conditions on the causal graph under which causal effects among observed variables can be determined uniquely. Furthermore, we provide necessary and sufficient graphical conditions for unique identification of the number of variables in the system. Experiments on synthetic data and real-world data show the effectiveness of our proposed algorithm for learning causal models. [ABSTRACT FROM AUTHOR]

In distributed function computation, each node has an initial value and the goal is to compute a function of these values in a distributed manner. In this paper, we propose a novel token-based approach to compute a wide class of target functions to which we refer as “token-based function computation with memory” (TCM) algorithm. In this approach, node values are attached to tokens and travel across the network. Each pair of travelling tokens would coalesce when they meet, forming a token with a new value as a function of the original token values. In contrast to the coalescing random walk (CRW) algorithm, where token movement is governed by random walk, meeting of tokens in our scheme is accelerated by adopting a novel chasing mechanism. We proved that, compared to the CRW algorithm, the TCM algorithm results in a reduction of time complexity by a factor of at least \sqrtn/\log (n) in Erdös-Renyi and complete graphs, and by a factor of $\log (n)/\log (\log (n))$ in torus networks. Simulation results show that there is at least a constant factor improvement in the message complexity of TCM algorithm in all considered topologies. Robustness of the CRW and TCM algorithms in the presence of node failure is analyzed. We show that their robustness can be improved by running multiple instances of the algorithms in parallel. [ABSTRACT FROM PUBLISHER]

Abstract: In this paper, we study the impact of sensing error and channel fading on the decision process of a multiple secondary user network in a primary network whose channel occupancy states are modelled as a Bernoulli process. We present a randomized access strategy to maximize total secondary network throughput. The proposed method guarantees that the probability of collision between primary and secondary users in each channel is less than the predefined value of P c = ξ. To find the optimal access strategy, we formulate secondary network throughput as an optimization problem. Then, using the KKT method to find the solution, we break the original problem into multiple sub-problems. Then, we employ an iterative algorithm and a sifting technique to reduce the number of sub-problems and find the final solution. The solution contains both the SUs access strategies and their detector operating points. In other words, we present a solution for the cross layer design in cognitive radio networks. At the end, we compare our results with two other heuristic strategies (1) equi-probable strategy and (2) weighted strategy and show that the proposed method outperforms them in terms of total throughput. [Copyright &y& Elsevier]

• Proposing two distributed algorithms for multi-choice majority voting in the beep model. • Showing the correctness of the algorithms and analyzing their time, message, and space complexities. • Comparing with the baseline methods analytically. • Evaluating the proposed methods experimentally on different network topologies and initial distributions of votes. We consider the problem of distributed multi-choice voting in a setting that each node can communicate with its neighbors merely by sending beep signals. Given its simplicity, the beep communication model is of practical importance in different applications such as system biology and wireless sensor networks. Yet, the distributed majority voting has not been resolved in this setting. In this paper, we propose two algorithms, named Distributed Voting with Beeps, to resolve this problem. In the first proposed algorithm, the adjacent nodes having the same value form a set called spot. Afterwards, the spots with majority value try to corrode the spots with non-majority values. The second proposed algorithm is based on pairwise interactions between nodes. The proposed algorithms have a termination detection procedure to check whether voting is achieved. We establish theoretical guarantees for the convergence of these algorithms. In particular, we show that the success probability of the first algorithm tends to one as the percentage of the initial votes in majority increases. For the second algorithm, we show that it returns the correct output with high probability. Our experiments show that the algorithms are fairly invariant to the network topology, initial distribution of values, and network size. [ABSTRACT FROM AUTHOR]

The main goal in many fields in the empirical sciences is to discover causal relationships among a set of variables from observational data. PC algorithm is one of the promising solutions to learn underlying causal structure by performing a number of conditional independence tests. In this paper, we propose a novel GPU-based parallel algorithm, called cuPC, to execute an order-independent version of PC. The proposed solution has two variants, cuPC-E and cuPC-S, which parallelize PC in two different ways for multivariate normal distribution. Experimental results show the scalability of the proposed algorithms with respect to the number of variables, the number of samples, and different graph densities. For instance, in one of the most challenging datasets, the runtime is reduced from more than 11 hours to about 4 seconds. On average, cuPC-E and cuPC-S achieve 500X and 1300X speedup, respectively, compared to serial implementation on CPU. [ABSTRACT FROM AUTHOR]