Search

Instruction-tuned LLMs are able to provide an explanation about their output to users by generating self-explanations that do not require gradient computations or the application of possibly complex XAI methods. In this paper, we analyse whether this ability results in a good explanation by evaluating self-explanations in the form of input rationales with respect to their plausibility to humans as well as their faithfulness to models. For this, we apply two text classification tasks: sentiment classification and forced labour detection. Next to English, we further include Danish and Italian translations of the sentiment classification task and compare self-explanations to human annotations for all samples. To allow for direct comparisons, we also compute post-hoc feature attribution, i.e., layer-wise relevance propagation (LRP) and apply this pipeline to 4 LLMs (Llama2, Llama3, Mistral and Mixtral). Our results show that self-explanations align more closely with human annotations compared to LRP, while maintaining a comparable level of faithfulness., Comment: preprint

We derive a linearized version of the monotonicity method for shape reconstruction using time harmonic elastic waves. The linearized method provides an efficient version of the method, drastically reducing computation time. Here we show that the linearized method has some additional advantages. The linearized method can in particular be used to obtain additional information on the material parameters, and is able to partially separate and identify the supports of the Lam\'e parameters.

We present and analyze a control variate strategy based on couplings to reduce the variance of finite difference estimators of sensitivity coefficients, called transport coefficients in the physics literature. We study the bias and variance of a sticky-coupling and a synchronous-coupling based estimator as the finite difference parameter $\eta$ goes to zero. For diffusions with elliptic additive noise, we show that when the drift is contractive outside a compact the bias of a sticky-coupling based estimator is bounded as $\eta \to 0$ and its variance behaves like $\eta^{-1}$, compared to the standard estimator whose bias and variance behave like $\eta^{-1}$ and $\eta^{-2}$, respectively. Under the stronger assumption that the drift is contractive everywhere, we additionally show that the bias and variance of the synchronous-coupling based estimator are both bounded as $\eta \to 0$. Our hypotheses include overdamped Langevin dynamics with many physically relevant non-convex potentials. We illustrate our theoretical results with numerical examples, including overdamped Langevin dynamics with a highly non-convex Lennard-Jones potential to demonstrate both failure of synchronous coupling and the effectiveness of sticky coupling in the not globally contractive setting., Comment: 57 pages, 11 figures

Many advances in astronomy and astrophysics originate from accurate images of the sky emission across multiple wavelengths. This often requires reconstructing spatially and spectrally correlated signals detected from multiple instruments. To facilitate the high-fidelity imaging of these signals, we introduce the universal Bayesian imaging kit (UBIK). Specifically, we present J-UBIK, a flexible and modular implementation leveraging the JAX-accelerated NIFTy.re software as its backend. J-UBIK streamlines the implementation of the key Bayesian inference components, providing for all the necessary steps of Bayesian imaging pipelines. First, it provides adaptable prior models for different sky realizations. Second, it includes likelihood models tailored to specific instruments. So far, the package includes three instruments: Chandra and eROSITA for X-ray observations, and the James Webb Space Telescope (JWST) for the near- and mid-infrared. The aim is to expand this set in the future. Third, these models can be integrated with various inference and optimization schemes, such as maximum a posteriori estimation and variational inference. Explicit demos show how to integrate the individual modules into a full analysis pipeline. Overall, J-UBIK enables efficient generation of high-fidelity images via Bayesian pipelines that can be tailored to specific research objectives.

We consider the Demand Strip Packing problem (DSP), in which we are given a set of jobs, each specified by a processing time and a demand. The task is to schedule all jobs such that they are finished before some deadline $D$ while minimizing the peak demand, i.e., the maximum total demand of tasks executed at any point in time. DSP is closely related to the Strip Packing problem (SP), in which we are given a set of axis-aligned rectangles that must be packed into a strip of fixed width while minimizing the maximum height. DSP and SP are known to be NP-hard to approximate to within a factor below $\frac{3}{2}$. To achieve the essentially best possible approximation guarantee, we prove a structural result. Any instance admits a solution with peak demand at most $\big(\frac32+\varepsilon\big)OPT$ satisfying one of two properties. Either (i) the solution leaves a gap for a job with demand $OPT$ and processing time $\mathcal O(\varepsilon D)$ or (ii) all jobs with demand greater than $\frac{OPT}2$ appear sorted by demand in immediate succession. We then provide two efficient algorithms that find a solution with maximum demand at most $\big(\frac32+\varepsilon\big)OPT$ in the respective case. A central observation, which sets our approach apart from previous ones for DSP, is that the properties (i) and (ii) need not be efficiently decidable: We can simply run both algorithms and use whichever solution is the better one.

We consider a new scheduling problem on parallel identical machines in which the number of machines is initially not known, but it follows a given probability distribution. Only after all jobs are assigned to a given number of bags, the actual number of machines is revealed. Subsequently, the jobs need to be assigned to the machines without splitting the bags. This is the stochastic version of a related problem introduced by Stein and Zhong [SODA 2018, TALG 2020] and it is, for example, motivated by bundling jobs that need to be scheduled by data centers. We present two PTASs for the stochastic setting, computing job-to-bag assignments that (i) minimize the expected maximum machine load and (ii) maximize the expected minimum machine load (like in the Santa Claus problem), respectively. The former result follows by careful enumeration combined with known PTASs. For the latter result, we introduce an intricate dynamic program that we apply to a suitably rounded instance.

Recent sequence modeling approaches using Selective State Space Sequence Models, referred to as Mamba models, have seen a surge of interest. These models allow efficient processing of long sequences in linear time and are rapidly being adopted in a wide range of applications such as language modeling, demonstrating promising performance. To foster their reliable use in real-world scenarios, it is crucial to augment their transparency. Our work bridges this critical gap by bringing explainability, particularly Layer-wise Relevance Propagation (LRP), to the Mamba architecture. Guided by the axiom of relevance conservation, we identify specific components in the Mamba architecture, which cause unfaithful explanations. To remedy this issue, we propose MambaLRP, a novel algorithm within the LRP framework, which ensures a more stable and reliable relevance propagation through these components. Our proposed method is theoretically sound and excels in achieving state-of-the-art explanation performance across a diverse range of models and datasets. Moreover, MambaLRP facilitates a deeper inspection of Mamba architectures, uncovering various biases and evaluating their significance. It also enables the analysis of previous speculations regarding the long-range capabilities of Mamba models.

Multiple instance learning (MIL) is an effective and widely used approach for weakly supervised machine learning. In histopathology, MIL models have achieved remarkable success in tasks like tumor detection, biomarker prediction, and outcome prognostication. However, MIL explanation methods are still lagging behind, as they are limited to small bag sizes or disregard instance interactions. We revisit MIL through the lens of explainable AI (XAI) and introduce xMIL, a refined framework with more general assumptions. We demonstrate how to obtain improved MIL explanations using layer-wise relevance propagation (LRP) and conduct extensive evaluation experiments on three toy settings and four real-world histopathology datasets. Our approach consistently outperforms previous explanation attempts with particularly improved faithfulness scores on challenging biomarker prediction tasks. Finally, we showcase how xMIL explanations enable pathologists to extract insights from MIL models, representing a significant advance for knowledge discovery and model debugging in digital histopathology.

This survey paper presents a comprehensive and conceptual overview of anomaly detection using dynamic graphs. We focus on existing graph-based anomaly detection (AD) techniques and their applications to dynamic networks. The contributions of this survey paper include the following: i) a comparative study of existing surveys on anomaly detection; ii) a Dynamic Graph-based Anomaly Detection (DGAD) review framework in which approaches for detecting anomalies in dynamic graphs are grouped based on traditional machine-learning models, matrix transformations, probabilistic approaches, and deep-learning approaches; iii) a discussion of graphically representing both discrete and dynamic networks; and iv) a discussion of the advantages of graph-based techniques for capturing the relational structure and complex interactions in dynamic graph data. Finally, this work identifies the potential challenges and future directions for detecting anomalies in dynamic networks. This DGAD survey approach aims to provide a valuable resource for researchers and practitioners by summarizing the strengths and limitations of each approach, highlighting current research trends, and identifying open challenges. In doing so, it can guide future research efforts and promote advancements in anomaly detection in dynamic graphs. Keywords: Graphs, Anomaly Detection, dynamic networks,Graph Neural Networks (GNN), Node anomaly, Graph mining., Comment: 32 pages (double column), 4 figures, and the manuscript has just been accepted in ACM Journals of Transactions on Knowledge Discovery from Data (TKDD)

As Transformers have become state-of-the-art models for natural language processing (NLP) tasks, the need to understand and explain their predictions is increasingly apparent. Especially in unsupervised applications, such as information retrieval tasks, similarity models built on top of foundation model representations have been widely applied. However, their inner prediction mechanisms have mostly remained opaque. Recent advances in explainable AI have made it possible to mitigate these limitations by leveraging improved explanations for Transformers through layer-wise relevance propagation (LRP). Using BiLRP, an extension developed for computing second-order explanations in bilinear similarity models, we investigate which feature interactions drive similarity in NLP models. We validate the resulting explanations and demonstrate their utility in three corpus-level use cases, analyzing grammatical interactions, multilingual semantics, and biomedical text retrieval. Our findings contribute to a deeper understanding of different semantic similarity tasks and models, highlighting how novel explainable AI methods enable in-depth analyses and corpus-level insights., Comment: Accepted to NAACL 2024

Real-world graphs are complex to process for performing effective analysis, such as anomaly detection. However, recently, there have been several research efforts addressing the issues surrounding graph-based anomaly detection. In this paper, we discuss a comprehensive overview of anomaly detection techniques on graph data. We also discuss the various application domains which use those anomaly detection techniques. We present a new taxonomy that categorizes the different state-of-the-art anomaly detection methods based on assumptions and techniques. Within each category, we discuss the fundamental research ideas that have been done to improve anomaly detection. We further discuss the advantages and disadvantages of current anomaly detection techniques. Finally, we present potential future research directions in anomaly detection on graph-structured data.

Rationales in the form of manually annotated input spans usually serve as ground truth when evaluating explainability methods in NLP. They are, however, time-consuming and often biased by the annotation process. In this paper, we debate whether human gaze, in the form of webcam-based eye-tracking recordings, poses a valid alternative when evaluating importance scores. We evaluate the additional information provided by gaze data, such as total reading times, gaze entropy, and decoding accuracy with respect to human rationale annotations. We compare WebQAmGaze, a multilingual dataset for information-seeking QA, with attention and explainability-based importance scores for 4 different multilingual Transformer-based language models (mBERT, distil-mBERT, XLMR, and XLMR-L) and 3 languages (English, Spanish, and German). Our pipeline can easily be applied to other tasks and languages. Our findings suggest that gaze data offers valuable linguistic insights that could be leveraged to infer task difficulty and further show a comparable ranking of explainability methods to that of human rationales., Comment: Accepted to LREC-COLING 2024

Imaging is the process of transforming noisy, incomplete data into a space that humans can interpret. NIFTy is a Bayesian framework for imaging and has already successfully been applied to many fields in astrophysics. Previous design decisions held the performance and the development of methods in NIFTy back. We present a rewrite of NIFTy, coined NIFTy.re, which reworks the modeling principle, extends the inference strategies, and outsources much of the heavy lifting to JAX. The rewrite dramatically accelerates models written in NIFTy, lays the foundation for new types of inference machineries, improves maintainability, and enables interoperability between NIFTy and the JAX machine learning ecosystem., Comment: 10 pages, 2 figures, published in Journal of Open Source Software (JOSS)

We propose a new concept of lifts of reversible diffusion processes and show that various well-known non-reversible Markov processes arising in applications are lifts in this sense of simple reversible diffusions. Furthermore, we introduce a concept of non-asymptotic relaxation times and show that these can at most be reduced by a square root through lifting, generalising a related result in discrete time. Finally, we demonstrate how the recently developed approach to quantitative hypocoercivity based on space-time Poincar\'e inequalities can be rephrased and simplified in the language of lifts and how it can be applied to find optimal lifts., Comment: Fixed error in Section 5. 30 pages

Querying complex models for precise information (e.g. traffic models, database systems, large ML models) often entails intense computations and results in long response times. Thus, weaker models which give imprecise results quickly can be advantageous, provided inaccuracies can be resolved using few queries to a stronger model. In the fundamental problem of computing a maximum-weight basis of a matroid, a well-known generalization of many combinatorial optimization problems, algorithms have access to a clean oracle to query matroid information. We additionally equip algorithms with a fast but dirty oracle modelling an unknown, potentially different matroid. We design and analyze practical algorithms which only use few clean queries w.r.t. the quality of the dirty oracle, while maintaining robustness against arbitrarily poor dirty matroids, approaching the performance of classic algorithms for the given problem. Notably, we prove that our algorithms are, in many respects, best-possible. Further, we outline extensions to other matroid oracle types, non-free dirty oracles and other matroid problems.

In recent years, the diagnosis of gliomas has become increasingly complex. Analysis of glioma histopathology images using artificial intelligence (AI) offers new opportunities to support diagnosis and outcome prediction. To give an overview of the current state of research, this review examines 83 publicly available research studies that have proposed AI-based methods for whole-slide histopathology images of human gliomas, covering the diagnostic tasks of subtyping (23/83), grading (27/83), molecular marker prediction (20/83), and survival prediction (29/83). All studies were reviewed with regard to methodological aspects as well as clinical applicability. It was found that the focus of current research is the assessment of hematoxylin and eosin-stained tissue sections of adult-type diffuse gliomas. The majority of studies (52/83) are based on the publicly available glioblastoma and low-grade glioma datasets from The Cancer Genome Atlas (TCGA) and only a few studies employed other datasets in isolation (16/83) or in addition to the TCGA datasets (15/83). Current approaches mostly rely on convolutional neural networks (63/83) for analyzing tissue at 20x magnification (35/83). A new field of research is the integration of clinical data, omics data, or magnetic resonance imaging (29/83). So far, AI-based methods have achieved promising results, but are not yet used in real clinical settings. Future work should focus on the independent validation of methods on larger, multi-site datasets with high-quality and up-to-date clinical and molecular pathology annotations to demonstrate routine applicability.

Assurance Cases (ACs) are an established approach in safety engineering to argue quality claims in a structured way. In the context of quality assurance for Machine Learning (ML)-based software components, ACs are also being discussed and appear promising. Tools for operationalizing ACs do exist, yet mainly focus on supporting safety engineers on the system level. However, assuring the quality of an ML component within the system is commonly the responsibility of data scientists, who are usually less familiar with these tools. To address this gap, we propose a framework to support the operationalization of ACs for ML components based on technologies that data scientists use on a daily basis: Python and Jupyter Notebook. Our aim is to make the process of creating ML-related evidence in ACs more effective. Results from the application of the framework, documented through notebooks, can be integrated into existing AC tools. We illustrate the application of the framework on an example excerpt concerned with the quality of the test data., Comment: Accepted for publication at International Conference on Product-Focused Software Process Improvement (Profes 2023), https://conf.researchr.org/home/profes-2023

We study a fundamental online scheduling problem where jobs with processing times, weights, and deadlines arrive online over time at their release dates. The task is to preemptively schedule these jobs on a single or multiple (possibly unrelated) machines with the objective to maximize the weighted throughput, the total weight of jobs that complete before their deadline. To overcome known lower bounds for the competitive analysis, we assume that each job arrives with some slack $\varepsilon > 0$; that is, the time window for processing job $j$ on any machine $i$ on which it can be executed has length at least $(1+\varepsilon)$ times $j$'s processing time on machine $i$. Our contribution is a best possible online algorithm for weighted throughput maximization on unrelated machines: Our algorithm is $O\big(\frac1\varepsilon\big)$-competitive, which matches the lower bound for unweighted throughput maximization on a single machine. Even for a single machine, it was not known whether the problem with weighted jobs is "harder" than the problem with unweighted jobs. Thus, we answer this question and close weighted throughput maximization on a single machine with a best possible competitive ratio $\Theta\big(\frac1\varepsilon\big)$. While we focus on non-migratory schedules, our algorithm achieves the same (up to constants) performance guarantee when compared to an optimal migratory schedule.

Contrastive explanations, where one decision is explained in contrast to another, are supposed to be closer to how humans explain a decision than non-contrastive explanations, where the decision is not necessarily referenced to an alternative. This claim has never been empirically validated. We analyze four English text-classification datasets (SST2, DynaSent, BIOS and DBpedia-Animals). We fine-tune and extract explanations from three different models (RoBERTa, GTP-2, and T5), each in three different sizes and apply three post-hoc explainability methods (LRP, GradientxInput, GradNorm). We furthermore collect and release human rationale annotations for a subset of 100 samples from the BIOS dataset for contrastive and non-contrastive settings. A cross-comparison between model-based rationales and human annotations, both in contrastive and non-contrastive settings, yields a high agreement between the two settings for models as well as for humans. Moreover, model-based explanations computed in both settings align equally well with human rationales. Thus, we empirically find that humans do not necessarily explain in a contrastive manner.9 pages, long paper at ACL 2022 proceedings., Comment: 9 pages, long paper at EMNLP 2023 proceedings

Historical materials are abundant. Yet, piecing together how human knowledge has evolved and spread both diachronically and synchronically remains a challenge that can so far only be very selectively addressed. The vast volume of materials precludes comprehensive studies, given the restricted number of human specialists. However, as large amounts of historical materials are now available in digital form there is a promising opportunity for AI-assisted historical analysis. In this work, we take a pivotal step towards analyzing vast historical corpora by employing innovative machine learning (ML) techniques, enabling in-depth historical insights on a grand scale. Our study centers on the evolution of knowledge within the `Sacrobosco Collection' -- a digitized collection of 359 early modern printed editions of textbooks on astronomy used at European universities between 1472 and 1650 -- roughly 76,000 pages, many of which contain astronomic, computational tables. An ML based analysis of these tables helps to unveil important facets of the spatio-temporal evolution of knowledge and innovation in the field of mathematical astronomy in the period, as taught at European universities.

We consider the problem of reconstructing inhomogeneities in an isotropic elastic body using time harmonic waves. Here we extend the so called monotonicity method for inclusion detection and show how to determine certain types of inhomogeneities in the Lam\'e parameters and the density. We also included some numerical tests of the method.

This work considers Bayesian experimental design for the inverse boundary value problem of linear elasticity in a two-dimensional setting. The aim is to optimize the positions of compactly supported pressure activations on the boundary of the examined body in order to maximize the value of the resulting boundary deformations as data for the inverse problem of reconstructing the Lam\'e parameters inside the object. We resort to a linearized measurement model and adopt the framework of Bayesian experimental design, under the assumption that the prior and measurement noise distributions are mutually independent Gaussians. This enables the use of the standard Bayesian A-optimality criterion for deducing optimal positions for the pressure activations. The (second) derivatives of the boundary measurements with respect to the Lam\'e parameters and the positions of the boundary pressure activations are deduced to allow minimizing the corresponding objective function, i.e., the trace of the covariance matrix of the posterior distribution, by a gradient-based optimization algorithm. Two-dimensional numerical experiments are performed to demonstrate the functionality of our approach., Comment: 23 pages, 11 figures

Supernovae are an important source of energy in the interstellar medium. Young remnants of supernovae have a peak emission in the X-ray region, making them interesting objects for X-ray observations. In particular, the supernova remnant SN1006 is of great interest due to its historical record, proximity and brightness. It has therefore been studied by several X-ray telescopes. Improving the X-ray imaging of this and other remnants is important but challenging as it requires to address a spatially varying instrument response in order to achieve a high signal-to-noise ratio. Here, we use Chandra observations to demonstrate the capabilities of Bayesian image reconstruction using information field theory. Our objective is to reconstruct denoised, deconvolved and spatio-spectral resolved images from X-ray observations and to decompose the emission into different morphologies, namely diffuse and point-like. Further, we aim to fuse data from different detectors and pointings into a mosaic and quantify the uncertainty of our result. Utilizing prior knowledge on the spatial and spectral correlation structure of the two components, diffuse emission and point sources, the presented method allows the effective decomposition of the signal into these. In order to accelerate the imaging process, we introduce a multi-step approach, in which the spatial reconstruction obtained for a single energy range is used to derive an informed starting point for the full spatio-spectral reconstruction. The method is applied to 11 Chandra observations of SN1006 from 2008 and 2012, providing a detailed, denoised and decomposed view of the remnant. In particular, the separated view of the diffuse emission should provide new insights into its complex small-scale structures in the center of the remnant and at the shock front profiles.

For the obstacle problem with a nonlinear operator, we characterize the space of global solutions with compact contact sets. This is achieved by constructing a bijection onto a class of quadratic polynomials describing the asymptotic behavior of solutions., Comment: 10 pages

Ecological momentary assessment (EMA) data have a broad base of application in the study of time trends and relations. In EMA studies, there are a number of design considerations which influence the analysis of the data. One general modeling framework is particularly well-suited for these analyses: state-space modeling. Here, we present the state-space modeling framework with recommendations for the considerations that go into modeling EMA data. These recommendations can account for the issues that come up in EMA data analysis such as idiographic versus nomothetic modeling, missing data, and stationary versus non-stationary data. In addition, we suggest R packages in order to implement these recommendations in practice. Overall, well-designed EMA studies offer opportunities for researchers to handle the momentary minutiae in their assessment of psychological phenomena.

We study global solutions to the thin obstacle problem with at most quadratic growth at infinity. We show that every ellipsoid can be realized as the contact set of such a solution. On the other hand, if such a solution has a compact contact set, we show that it must be an ellipsoid., Comment: A uniqueness result is added to Theorem 1.1, thanks to the suggestion of one referee. Funding information is added

Optimization of accelerator performance parameters is limited by numerous trade-offs and finding the appropriate balance between optimization goals for an unknown system is challenging to achieve. Here we show that multi-objective Bayesian optimization can map the solution space of a laser wakefield accelerator in a very sample-efficient way. Using a Gaussian mixture model, we isolate contributions related to an electron bunch at a certain energy and we observe that there exists a wide range of Pareto-optimal solutions that trade beam energy versus charge at similar laser-to-beam efficiency. However, many applications such as light sources require particle beams at a certain target energy. Once such a constraint is introduced we observe a direct trade-off between energy spread and accelerator efficiency. We furthermore demonstrate how specific solutions can be exploited using \emph{a posteriori} scalarization of the objectives, thereby efficiently splitting the exploration and exploitation phases.

Electrically controlled rotation of spins in a semiconducting channel is a prerequisite for the successful realization of many spintronic devices, like, e.g., the spin field effect transistor (sFET). To date, there have been only a few reports on electrically controlled spin precession in sFET-like devices. These devices operated in the ballistic regime, as postulated in the original sFET proposal, and hence need high SOC channel materials in practice. Here, we demonstrate gate-controlled precession of spins in a non-ballistic sFET using an array of narrow diffusive wires as a channel between a spin source and a spin drain. Our study shows that spins traveling in a semiconducting channel can be coherently rotated on a distance far exceeding the electrons mean free path, and spin-transistor functionality can be thus achieved in non-ballistic channels with relatively low SOC, relaxing two major constraints of the original sFET proposal., Comment: Main text: 21 pages, 4 figures. Supplementary Information:11 pages, 4 figures

Laser-plasma physics has developed rapidly over the past few decades as high-power lasers have become both increasingly powerful and more widely available. Early experimental and numerical research in this field was restricted to single-shot experiments with limited parameter exploration. However, recent technological improvements make it possible to gather an increasing amount of data, both in experiments and simulations. This has sparked interest in using advanced techniques from mathematics, statistics and computer science to deal with, and benefit from, big data. At the same time, sophisticated modeling techniques also provide new ways for researchers to effectively deal with situations in which still only sparse amounts of data are available. This paper aims to present an overview of relevant machine learning methods with focus on applicability to laser-plasma physics, including its important sub-fields of laser-plasma acceleration and inertial confinement fusion.