Search

Causality is essential for understanding complex systems, such as the economy, the brain, and the climate. Constructing causal graphs often relies on either data-driven or expert-driven approaches, both fraught with challenges. The former methods, like the celebrated PC algorithm, face issues with data requirements and assumptions of causal sufficiency, while the latter demand substantial time and domain knowledge. This work explores the capabilities of Large Language Models (LLMs) as an alternative to domain experts for causal graph generation. We frame conditional independence queries as prompts to LLMs and employ the PC algorithm with the answers. The performance of the LLM-based conditional independence oracle on systems with known causal graphs shows a high degree of variability. We improve the performance through a proposed statistical-inspired voting schema that allows some control over false-positive and false-negative rates. Inspecting the chain-of-thought argumentation, we find causal reasoning to justify its answer to a probabilistic query. We show evidence that knowledge-based CIT could eventually become a complementary tool for data-driven causal discovery.

Detecting latent confounders from proxy variables is an essential problem in causal effect estimation. Previous approaches are limited to low-dimensional proxies, sorted proxies, and binary treatments. We remove these assumptions and present a novel Proxy Confounder Factorization (PCF) framework for continuous treatment effect estimation when latent confounders manifest through high-dimensional, mixed proxy variables. For specific sample sizes, our two-step PCF implementation, using Independent Component Analysis (ICA-PCF), and the end-to-end implementation, using Gradient Descent (GD-PCF), achieve high correlation with the latent confounder and low absolute error in causal effect estimation with synthetic datasets in the high sample size regime. Even when faced with climate data, ICA-PCF recovers four components that explain $75.9\%$ of the variance in the North Atlantic Oscillation, a known confounder of precipitation patterns in Europe. Code for our PCF implementations and experiments can be found here: https://github.com/IPL-UV/confound_it. The proposed methodology constitutes a stepping stone towards discovering latent confounders and can be applied to many problems in disciplines dealing with high-dimensional observed proxies, e.g., spatiotemporal fields.

Physics is a field of science that has traditionally used the scientific method to answer questions about why natural phenomena occur and to make testable models that explain the phenomena. Discovering equations, laws and principles that are invariant, robust and causal explanations of the world has been fundamental in physical sciences throughout the centuries. Discoveries emerge from observing the world and, when possible, performing interventional studies in the system under study. With the advent of big data and the use of data-driven methods, causal and equation discovery fields have grown and made progress in computer science, physics, statistics, philosophy, and many applied fields. All these domains are intertwined and can be used to discover causal relations, physical laws, and equations from observational data. This paper reviews the concepts, methods, and relevant works on causal and equation discovery in the broad field of Physics and outlines the most important challenges and promising future lines of research. We also provide a taxonomy for observational causal and equation discovery, point out connections, and showcase a complete set of case studies in Earth and climate sciences, fluid dynamics and mechanics, and the neurosciences. This review demonstrates that discovering fundamental laws and causal relations by observing natural phenomena is being revolutionised with the efficient exploitation of observational data, modern machine learning algorithms and the interaction with domain knowledge. Exciting times are ahead with many challenges and opportunities to improve our understanding of complex systems., Comment: 137 pages

This paper introduces a novel statistical regression framework that allows the incorporation of consistency constraints. A linear and nonlinear (kernel-based) formulation are introduced, and both imply closed-form analytical solutions. The models exploit all the information from a set of drivers while being maximally independent of a set of auxiliary, protected variables. We successfully illustrate the performance in the estimation of chlorophyll content., Comment: arXiv admin note: substantial text overlap with arXiv:1710.05578

Certain theoretical aspects of vector autoregression (VAR) as tools to model economic time series are revised, in particular their capacity to include both short term and long term information. The VAR model, in its error correction form, is derived and the permanent-transitory decomposition of factors proposed by Gonzalo and Granger (1995) studied. An introductory exposition of estimation theory for reduced rank models, necessary to estimate the error correction model, is given. Cointegration analysis using the VAR model is carried out for government bond interest rates (short, medium and long term) of the United States, Mexico and Canada, with the objective of finding the long-term common factors that drive the system. The error correction model of this system is estimated using Johansen's method. Using this estimation the permanent-transitory decomposition of the system is calculated. Hypothesis tests are carried out on permanent factors to determine which of the nine rates studied drive the system., Comment: thesis, in Spanish, Instituto Technol\'ogico aut\'onomo de M\'exico, 2016

Deforestation detection using satellite images can make an important contribution to forest management. Current approaches can be broadly divided into those that compare two images taken at similar periods of the year and those that monitor changes by using multiple images taken during the growing season. The CMFDA algorithm described in Zhu et al. (2012) is an algorithm that builds on the latter category by implementing a year-long, continuous, time-series based approach to monitoring images. This algorithm was developed for 30m resolution, 16-day frequency reflectance data from the Landsat satellite. In this work we adapt the algorithm to 1km, 16-day frequency reflectance data from the modis sensor aboard the Terra satellite. The CMFDA algorithm is composed of two submodels which are fitted on a pixel-by-pixel basis. The first estimates the amount of surface reflectance as a function of the day of the year. The second estimates the occurrence of a deforestation event by comparing the last few predicted and real reflectance values. For this comparison, the reflectance observations for six different bands are first combined into a forest index. Real and predicted values of the forest index are then compared and high absolute differences for consecutive observation dates are flagged as deforestation events. Our adapted algorithm also uses the two model framework. However, since the modis 13A2 dataset used, includes reflectance data for different spectral bands than those included in the Landsat dataset, we cannot construct the forest index. Instead we propose two contrasting approaches: a multivariate and an index approach similar to that of CMFDA.

Gene expression depends on thousands of factors and we usually only have access to tens or hundreds of observations of gene expression levels meaning we are in a high-dimensional setting. Additionally we don't always observe or care about all the factors. However, many different gene expression levels depend on a set of common factors. By observing the joint variance of the gene expression levels together with the observed primary variables (those we care about) Surrogate Variable Analysis (SVA) seeks to estimate the remaining unobserved factors. The ultimate goal is to assess whether the primary variable (or vector) has a significant effect on the different gene expression levels, but without estimating unobserved factors first the various regression models and hypothesis tests are dependent which complicates significance analysis. In this work we define a class of additive gene expression structural equation models (SEMs) which are convenient for modeling gene expression data and which provides a useful framework to understand the various steps of the SVA methodology. We justify the use of this class from a modeling viewpoint but also from a causality viewpoint by exploring the independence and causality properties of this class and comparing to the biologically driven data assumptions. For this we use some of the theory that has been developed elsewhere on graphical models and causality. We then give a detailed description of the SVA methodology and its implementation in the R package sva referring each step to different parts of the additive gene expression SEM defined previously.

Sparse feature selection is necessary when we fit statistical models, we have access to a large group of features, don't know which are relevant, but assume that most are not. Alternatively, when the number of features is larger than the available data the model becomes over parametrized and the sparse feature selection task involves selecting the most informative variables for the model. When the model is a simple location model and the number of relevant features does not grow with the total number of features, sparse feature selection corresponds to sparse mean estimation. We deal with a simplified mean estimation problem consisting of an additive model with gaussian noise and mean that is in a restricted, finite hypothesis space. This restriction simplifies the mean estimation problem into a selection problem of combinatorial nature. Although the hypothesis space is finite, its size is exponential in the dimension of the mean. In limited data settings and when the size of the hypothesis space depends on the amount of data or on the dimension of the data, choosing an approximation set of hypotheses is a desirable approach. Choosing a set of hypotheses instead of a single one implies replacing the bias-variance trade off with a resolution-stability trade off. Generalization capacity provides a resolution selection criterion based on allowing the learning algorithm to communicate the largest amount of information in the data to the learner without error. In this work the theory of approximation set coding and generalization capacity is explored in order to understand this approach. We then apply the generalization capacity criterion to the simplified sparse mean estimation problem and detail an importance sampling algorithm which at once solves the difficulty posed by large hypothesis spaces and the slow convergence of uniform sampling algorithms.

The method proposed by Bernardo and Smith [2000] to approximate reference priors by simulation was analyzed with the objective of improving the procedure in order to obtain consistent estimators and to allow the estimation of asymptotic probability intervals. In this sense, the variance of Bernardo's estimator was derived and was used to construct probability intervals that permitted the expression of the estimation error as a function of sample size. Additionally a variance reduction technique (common random numbers) were explored as a means to obtain more precise estimations with smaller sample sizes. These technique was found to considerably reduce estimation error for some of the examples explored. In other cases the use of the technique resulted in zero estimation error given that the estimator does not depend on the sample., Comment: in Spanish