Search

Data Attribution (DA) methods quantify the influence of individual training data points on model outputs and have broad applications such as explainability, data selection, and noisy label identification. However, existing DA methods are often computationally intensive, limiting their applicability to large-scale machine learning models. To address this challenge, we introduce the Generalized Group Data Attribution (GGDA) framework, which computationally simplifies DA by attributing to groups of training points instead of individual ones. GGDA is a general framework that subsumes existing attribution methods and can be applied to new DA techniques as they emerge. It allows users to optimize the trade-off between efficiency and fidelity based on their needs. Our empirical results demonstrate that GGDA applied to popular DA methods such as Influence Functions, TracIn, and TRAK results in upto 10x-50x speedups over standard DA methods while gracefully trading off attribution fidelity. For downstream applications such as dataset pruning and noisy label identification, we demonstrate that GGDA significantly improves computational efficiency and maintains effectiveness, enabling practical applications in large-scale machine learning scenarios that were previously infeasible.

In this work, three fundamentally different machine learning models are combined to create a new, joint model for forecasting the UEFA EURO 2024. Therefore, a generalized linear model, a random forest model, and a extreme gradient boosting model are used to predict the number of goals a team scores in a match. The three models are trained on the match results of the UEFA EUROs 2004-2020, with additional covariates characterizing the teams for each tournament as well as three enhanced variables derived from different ranking methods for football teams. The first enhanced variable is based on historic match data from national teams, the second is based on the bookmakers' tournament winning odds of all participating teams, and the third is based on historic match data of individual players both for club and international matches, resulting in player ratings. Then, based on current covariate information of the participating teams, the final trained model is used to predict the UEFA EURO 2024. For this purpose, the tournament is simulated 100.000 times, based on the estimated expected number of goals for all possible matches, from which probabilities across the different tournament stages are derived. Our combined model identifies France as the clear favourite with a winning probability of 19.2%, followed by England (16.7%) and host Germany (13.7%).

We obtain the comparison principle for discontinuous viscosity sub- and supersolutions of nonlocal Hamilton-Jacobi equations, with superlinear and coercive gradient terms. The nonlocal terms are integro-differential operators in L\'evy form, with general measures: $x$-dependent, possibly degenerate and without any restriction on the order. The measures must satisfy a combined Wasserstein/Total Variation-continuity assumption, which is one of the weakest conditions used in the context of viscosity approach for this type of integro-differential PDEs. The proof relies on a regularizing effect due to the gradient growth. We present several examples of applications to PDEs with different types of nonlocal operators (measures with density, operators of variable order, L\'evy-It\^o operators).

The article demonstrates the procedure how to compute the Zariski closure of an orbit by an algebraic action of finitely generated group on the affine plane. First half of the algorithm is about deciding whether given finitely generated group is contained in an algebraic group. For the next half, we compute the totality of the invariant subvarieties for a single triangular automorphism. Then, the computation for the individual generators is applied to compute the orbit Zariski closure for a finitely generated group., Comment: 37 pages. Comments are welcome

As Large Language Models (LLMs) are increasingly being employed in real-world applications in critical domains such as healthcare, it is important to ensure that the Chain-of-Thought (CoT) reasoning generated by these models faithfully captures their underlying behavior. While LLMs are known to generate CoT reasoning that is appealing to humans, prior studies have shown that these explanations do not accurately reflect the actual behavior of the underlying LLMs. In this work, we explore the promise of three broad approaches commonly employed to steer the behavior of LLMs to enhance the faithfulness of the CoT reasoning generated by LLMs: in-context learning, fine-tuning, and activation editing. Specifically, we introduce novel strategies for in-context learning, fine-tuning, and activation editing aimed at improving the faithfulness of the CoT reasoning. We then carry out extensive empirical analyses with multiple benchmark datasets to explore the promise of these strategies. Our analyses indicate that these strategies offer limited success in improving the faithfulness of the CoT reasoning, with only slight performance enhancements in controlled scenarios. Activation editing demonstrated minimal success, while fine-tuning and in-context learning achieved marginal improvements that failed to generalize across diverse reasoning and truthful question-answering benchmarks. In summary, our work underscores the inherent difficulty in eliciting faithful CoT reasoning from LLMs, suggesting that the current array of approaches may not be sufficient to address this complex challenge.

Quantile regression is a powerful tool for detecting exposure-outcome associations given covariates across different parts of the outcome's distribution, but has two major limitations when the aim is to infer the effect of an exposure. Firstly, the exposure coefficient estimator may not converge to a meaningful quantity when the model is misspecified, and secondly, variable selection methods may induce bias and excess uncertainty, rendering inferences biased and overly optimistic. In this paper, we address these issues via partially linear quantile regression models which parametrize the conditional association of interest, but do not restrict the association with other covariates in the model. We propose consistent estimators for the unknown model parameter by mapping it onto a nonparametric main effect estimand that captures the (conditional) association of interest even when the quantile model is misspecified. This estimand is estimated using the efficient influence function under the nonparametric model, allowing for the incorporation of data-adaptive procedures such as variable selection and machine learning. Our approach provides a flexible and reliable method for detecting associations that is robust to model misspecification and excess uncertainty induced by variable selection methods. The proposal is illustrated using simulation studies and data on annual health care costs associated with excess body weight.

The thermodynamics and transport properties of polymeric materials are essential for the design of reactors and for the development of polymer deconstruction processes. Existing property prediction tools such as correlations based on entropy scaling, kinetic gas theory, and free-volume model are inadequate for polymers. In this paper, we introduce a data-driven model for polyolefins based on data from molecular dynamics simulations that can accurately predict the transport properties of polyethylenes and their binary mixtures with higher n-alkanes across a range of temperatures, pressures, concentrations, and oligomer molecular weights.

There is considerable interest in designing new polymeric materials with built-in mechanisms for recycling. In this work, we present a systematic exploration of several polyethylene-based polymers that contain strategically incorporated cleavable bonds that are susceptible to chemical degradation. We consider ten distinct telechelic functionalities across varying chain lengths: ester, aromatic ester, anhydride, carbonate, urethane, siloxane, oxalate, acetal, urea, and amide linkages. Our study is focused on the effects of cleavable bonds on density, diffusion, enthalpy, and melting temperatures. We also examine the crystallization process. These are key properties for polymer processing and, by comparing them to those of linear polyethylene, we establish a platform that will help guide future synthetic efforts towards the design of sustainable plastics with performance comparable to that of traditional, high-performance polyolefin-based materials.

The inverse spin Hall effect used for detection of spin currents was observed by voltage measurements in bilayers of normal metal (NM)/ferromagnetic metal (FM), using Nb, Mo and Bi as normal metal and Permalloy (Py, Ni$_{81}$Fe$_{19}$) as ferromagnetic metal. The spin current was generated by the spin pumping effect with ferromagnetic resonance. The samples were deposited by dc magnetron sputtering at room temperature on Si (001) substrates. The three bilayers of Nb/Py, Mo/Py and Bi/Py had a spin-orbit coupling large enough to observe the voltage generation by spin Hall effect, Comment: in Spanish language

In this work we show that several problems naturally modeled as Nonlinear Absolute Value Equations (NAVE), can be restated as Nonlinear Complementarity Problems (NCP) and solved efficiently using smoothing regularizing techniques under mild assumptions. Applications include ridge optimization and resolution of nonlinear ordinary differential equations.

This work introduces a new general approach for the numerical analysis of stable equilibria to second order mean field games systems in cases where the uniqueness of solutions may fail. For the sake of simplicity, we focus on a simple stationary case. We propose an abstract framework to study these solutions by reformulating the mean field game system as an abstract equation in a Banach space. In this context, stable equilibria turn out to be regular solutions to this equation, meaning that the linearized system is well-posed. We provide three applications of this property: we study the sensitivity analysis of stable solutions, establish error estimates for their finite element approximations, and prove the local converge of Newton's method in infinite dimensions.

In this paper, we propose a new flexible distribution for data on the three-dimensional torus which we call a trivariate wrapped Cauchy copula. Our trivariate copula has several attractive properties. It has a simple form of density and is unimodal. its parameters are interpretable and allow adjustable degree of dependence between every pair of variables and these can be easily estimated. The conditional distributions of the model are well studied bivariate wrapped Cauchy distributions. Furthermore, the distribution can be easily simulated. Parameter estimation via maximum likelihood for the distribution is given and we highlight the simple implementation procedure to obtain these estimates. We compare our model to its competitors for analysing trivariate data and provide some evidence of its advantages. Another interesting feature of this model is that it can be extended to cylindrical copula as we describe this new cylindrical copula and then gives its properties. We illustrate our trivariate wrapped Cauchy copula on data from protein bioinformatics of conformational angles, and our cylindrical copula using climate data related to buoy in the Adriatic Sea. The paper is motivated by these real trivariate datasets, but we indicate how the model can be extended to multivariate copulas., Comment: 38 pages, 11 figures

As the size of quantum hardware progressively increases, the conjectured computational advantages of quantum technologies tend to be threatened by noise, which randomly corrupts the design of quantum logical gates. Several methods already exist to reduce the impacts of noise on that matter. However, a reliable and user-friendly one to reduce the noise impact has not been presented yet. Addressing this issue, this paper proposes a relevant method based on evolutionary optimisation and modulation of the gate design. This method consists of two parts : a model of quantum gate design with time-dependent noise terms, parameterised by a vector of laser phases, and an evolutionary optimisation platform aimed at satisfying a trade-off between the gate fidelity and a pulse duration-related metric of the time consuming simulation model. This feature is the main novelty of this work. Another advantage is the ability to treat any noise spectrum, regardless of its characteristics (e.g., variance, frequency range, etc). A thorough validation of the method is presented, which is based on empirical averaging of random gate trajectories. It is shown that evolutionary based method is successfully applied for noise mitigation. It is expected that the proposed method will help designing more and more noise-resisting quantum gates.

Recent advancements in Large Language Models (LLMs) have demonstrated exceptional capabilities in complex tasks like machine translation, commonsense reasoning, and language understanding. One of the primary reasons for the adaptability of LLMs in such diverse tasks is their in-context learning (ICL) capability, which allows them to perform well on new tasks by simply using a few task samples in the prompt. Despite their effectiveness in enhancing the performance of LLMs on diverse language and tabular tasks, these methods have not been thoroughly explored for their potential to generate post hoc explanations. In this work, we carry out one of the first explorations to analyze the effectiveness of LLMs in explaining other complex predictive models using ICL. To this end, we propose a novel framework, In-Context Explainers, comprising of three novel approaches that exploit the ICL capabilities of LLMs to explain the predictions made by other predictive models. We conduct extensive analysis with these approaches on real-world tabular and text datasets and demonstrate that LLMs are capable of explaining other predictive models similar to state-of-the-art post hoc explainers, opening up promising avenues for future research into LLM-based post hoc explanations of complex predictive models.

Wheelchair basketball, regulated by the International Wheelchair Basketball Federation, is a sport designed for individuals with physical disabilities. This paper presents a data-driven tool that effectively determines optimal team line-ups based on past performance data and metrics for player effectiveness. Our proposed methodology involves combining a Bayesian longitudinal model with an integer linear problem to optimise the line-up of a wheelchair basketball team. To illustrate our approach, we use real data from a team competing in the Rollstuhlbasketball Bundesliga, namely the Doneck Dolphins Trier. We consider three distinct performance metrics for each player and incorporate uncertainty from the posterior predictive distribution of the longitudinal model into the optimisation process. The results demonstrate the tool's ability to select the most suitable team compositions and calculate posterior probabilities of compatibility or incompatibility among players on the court.

We introduce the asynchronous graph generator (AGG), a novel graph attention network for imputation and prediction of multi-channel time series. Free from recurrent components or assumptions about temporal/spatial regularity, AGG encodes measurements, timestamps and channel-specific features directly in the nodes via learnable embeddings. Through an attention mechanism, these embeddings allow for discovering expressive relationships among the variables of interest in the form of a homogeneous graph. Once trained, AGG performs imputation by \emph{conditional attention generation}, i.e., by creating a new node conditioned on given timestamps and channel specification. The proposed AGG is compared to related methods in the literature and its performance is analysed from a data augmentation perspective. Our experiments reveal that AGG achieved state-of-the-art results in time series imputation, classification and prediction for the benchmark datasets \emph{Beijing Air Quality}, \emph{PhysioNet ICU 2012} and \emph{UCI localisation}, outperforming other recent attention-based networks.

Electron cyclotron waves (whistlers), are commonly observed in plasmas near Earth and the solar wind. In the presence of nonlinear mirror modes, bursts of whistlers, usually called lion roars, have been observed within low magnetic field regions associated to these modes. In the intracluster medium (ICM) of galaxy clusters, the excitation of the mirror instability is expected, but it is not yet clear whether electron and ion cyclotron waves can also be present under conditions where gas pressure dominates over magnetic pressure (high $\beta$). In this work, we perform fully kinetic particle-in-cell (PIC) simulations of a plasma subject to a continuous amplification of the mean magnetic field $\textbf{B}(t)$ to study the nonlinear stages of the mirror instability and the ensuing excitation of whistler and ion cyclotron (IC) waves under ICM conditions. Once mirror modes reach nonlinear amplitudes, both whistler and IC waves start to emerge simultaneously, with sub-dominant amplitudes, propagating in low-$\textbf{B}$ regions, and quasi-parallel to $\textbf{B}(t)$. We show that the underlying source of excitation is the pressure anisotropy of electrons and ions trapped in mirror modes with loss-cone type distributions. We also observe that IC waves play an essential role in regulating the ion pressure anisotropy at nonlinear stages. We argue that whistler and IC waves are a concomitant feature at late stages of the mirror instability even at high-$\beta$, and therefore expected to be present in astrophysical environments like the ICM. We discuss the implications of our results for collisionless heating and dissipation of turbulence in the ICM., Comment: 22 pages, 17 figures. Submitted to ApJ. Comments are welcome

Band engineering stands as an efficient route to induce strongly correlated quantum many-body phenomena. Besides inspiring analogies among diverse physical fields, tuning on demand the group velocity is highly attractive in photonics because it allows unconventional flows of light. $\Lambda$-schemes offer a route to control the propagation of light in a lattice-free configurations, enabling exotic phases such as slow-light and allowing for highly optical non-linear systems. Here, we realize room-temperature intercavity Frenkel polaritons excited across two strongly coupled cavities. We demonstrate the formation of a tuneable heavy-polariton, akin to slow light, appearing in the absence of a periodic in-plane potential. Our photonic architecture based on a simple three-level scheme enables the unique spatial segregation of photons and excitons in different cavities and maintains a balanced degree of mixing between them. This unveils a dynamical competition between many-body scattering processes and the underlying polariton nature which leads to an increased fluorescence lifetime. The intercavity polariton features are further revealed under appropriate resonant pumping, where we observe suppression of the polariton fluorescence intensity., Comment: 8+ 4 pages

We propose a Statistically Enhanced Learning (aka. SEL) model to predict handball games. Our Machine Learning model augmented with SEL features outperforms state-of-the-art models with an accuracy beyond 80%. In this work, we show how we construct the data set to train Machine Learning models on past female club matches. We then compare different models and evaluate them to assess their performance capabilities. Finally, explainability methods allow us to change the scope of our tool from a purely predictive solution to a highly insightful analytical tool. This can become a valuable asset for handball teams' coaches providing valuable statistical and predictive insights to prepare future competitions.

Feature engineering is of critical importance in the field of Data Science. While any data scientist knows the importance of rigorously preparing data to obtain good performing models, only scarce literature formalizes its benefits. In this work, we will present the method of Statistically Enhanced Learning (SEL), a formalization framework of existing feature engineering and extraction tasks in Machine Learning (ML). The difference compared to classical ML consists in the fact that certain predictors are not directly observed but obtained as statistical estimators. Our goal is to study SEL, aiming to establish a formalized framework and illustrate its improved performance by means of simulations as well as applications on real life use cases.

The Right to Explanation is an important regulatory principle that allows individuals to request actionable explanations for algorithmic decisions. However, several technical challenges arise when providing such actionable explanations in practice. For instance, models are periodically retrained to handle dataset shifts. This process may invalidate some of the previously prescribed explanations, thus rendering them unactionable. But, it is unclear if and when such invalidations occur, and what factors determine explanation stability i.e., if an explanation remains unchanged amidst model retraining due to dataset shifts. In this paper, we address the aforementioned gaps and provide one of the first theoretical and empirical characterizations of the factors influencing explanation stability. To this end, we conduct rigorous theoretical analysis to demonstrate that model curvature, weight decay parameters while training, and the magnitude of the dataset shift are key factors that determine the extent of explanation (in)stability. Extensive experimentation with real-world datasets not only validates our theoretical results, but also demonstrates that the aforementioned factors dramatically impact the stability of explanations produced by various state-of-the-art methods., Comment: 30 pages, 19 figures. To be published at UAI 2023

This work addresses the challenge of providing consistent explanations for predictive models in the presence of model indeterminacy, which arises due to the existence of multiple (nearly) equally well-performing models for a given dataset and task. Despite their similar performance, such models often exhibit inconsistent or even contradictory explanations for their predictions, posing challenges to end users who rely on these models to make critical decisions. Recognizing this issue, we introduce ensemble methods as an approach to enhance the consistency of the explanations provided in these scenarios. Leveraging insights from recent work on neural network loss landscapes and mode connectivity, we devise ensemble strategies to efficiently explore the underspecification set -- the set of models with performance variations resulting solely from changes in the random seed during training. Experiments on five benchmark financial datasets reveal that ensembling can yield significant improvements when it comes to explanation similarity, and demonstrate the potential of existing ensemble methods to explore the underspecification set efficiently. Our findings highlight the importance of considering model indeterminacy when interpreting explanations and showcase the effectiveness of ensembles in enhancing the reliability of explanations in machine learning.

It is well-known that modern computer vision systems often exhibit behaviors misaligned with those of humans: from adversarial attacks to image corruptions, deep learning vision models suffer in a variety of settings that humans capably handle. In light of these phenomena, here we introduce another, orthogonal perspective studying the human-machine vision gap. We revisit the task of recovering images under degradation, first introduced over 30 years ago in the Recognition-by-Components theory of human vision. Specifically, we study the performance and behavior of neural networks on the seemingly simple task of classifying regular polygons at varying orders of degradation along their perimeters. To this end, we implement the Automated Shape Recoverability Test for rapidly generating large-scale datasets of perimeter-degraded regular polygons, modernizing the historically manual creation of image recoverability experiments. We then investigate the capacity of neural networks to recognize and recover such degraded shapes when initialized with different priors. Ultimately, we find that neural networks' behavior on this simple task conflicts with human behavior, raising a fundamental question of the robustness and learning capabilities of modern computer vision models., Comment: Accepted as a conference paper to NeurIPS 2023 (Datasets & Benchmarks Track)

Counterfactual explanations have been widely studied in explainability, with a range of application dependent methods prominent in fairness, recourse and model understanding. The major shortcoming associated with these methods, however, is their inability to provide explanations beyond the local or instance-level. While many works touch upon the notion of a global explanation, typically suggesting to aggregate masses of local explanations in the hope of ascertaining global properties, few provide frameworks that are both reliable and computationally tractable. Meanwhile, practitioners are requesting more efficient and interactive explainability tools. We take this opportunity to propose Global & Efficient Counterfactual Explanations (GLOBE-CE), a flexible framework that tackles the reliability and scalability issues associated with current state-of-the-art, particularly on higher dimensional datasets and in the presence of continuous features. Furthermore, we provide a unique mathematical analysis of categorical feature translations, utilising it in our method. Experimental evaluation with publicly available datasets and user studies demonstrate that GLOBE-CE performs significantly better than the current state-of-the-art across multiple metrics (e.g., speed, reliability)., Comment: Published as a conference paper at ICML 2023 (9 page main text, 3 page references, 16 page appendix)