Search

As AI systems become more capable, widely deployed, and increasingly autonomous in critical areas such as cybersecurity, biological research, and healthcare, ensuring their safety and alignment with human values is paramount. Machine unlearning -- the ability to selectively forget or suppress specific types of knowledge -- has shown promise for privacy and data removal tasks, which has been the primary focus of existing research. More recently, its potential application to AI safety has gained attention. In this paper, we identify key limitations that prevent unlearning from serving as a comprehensive solution for AI safety, particularly in managing dual-use knowledge in sensitive domains like cybersecurity and chemical, biological, radiological, and nuclear (CBRN) safety. In these contexts, information can be both beneficial and harmful, and models may combine seemingly harmless information for harmful purposes -- unlearning this information could strongly affect beneficial uses. We provide an overview of inherent constraints and open problems, including the broader side effects of unlearning dangerous knowledge, as well as previously unexplored tensions between unlearning and existing safety mechanisms. Finally, we investigate challenges related to evaluation, robustness, and the preservation of safety features during unlearning. By mapping these limitations and open challenges, we aim to guide future research toward realistic applications of unlearning within a broader AI safety framework, acknowledging its limitations and highlighting areas where alternative approaches may be required.

In deep-inelastic positron-proton scattering, the lepton-jet azimuthal angular asymmetry is measured using data collected with the H1 detector at HERA. When the average transverse momentum of the lepton-jet system, $\lvert \vec{P}_\perp \rvert $, is much larger than the total transverse momentum of the system, $\lvert \vec{q}_\perp \rvert$, the asymmetry between parallel and antiparallel configurations, $\vec{P}_\perp$ and $\vec{q}_\perp$, is expected to be generated by initial and final state soft gluon radiation and can be predicted using perturbation theory. Quantifying the angular properties of the asymmetry therefore provides an additional test of the strong force. Studying the asymmetry is important for future measurements of intrinsic asymmetries generated by the proton's constituents through Transverse Momentum Dependent (TMD) Parton Distribution Functions (PDFs), where this asymmetry constitutes a dominant background. Moments of the azimuthal asymmetries are measured using a machine learning method for unfolding that does not require binning., Comment: 15 pages, 4 figures, 2 tables, updated affiliations and acknowledgements

The variance of a linearly combined forecast distribution (or linear pool) consists of two components: The average variance of the component distributions (`average uncertainty'), and the average squared difference between the components' means and the pool's mean (`disagreement'). This paper shows that similar decompositions hold for a class of uncertainty measures that can be constructed as entropy functions of kernel scores. The latter are a rich family of scoring rules that covers point and distribution forecasts for univariate and multivariate, discrete and continuous settings. The results in this paper are useful for two reasons. First, they provide a generic description of the uncertainty implicit in the linear pool. Second, they suggest principled measures of forecast disagreement in a wide range of applied settings.

Modern robotic perception is highly dependent on neural networks. It is well known that neural network-based perception can be unreliable in real-world deployment, especially in difficult imaging conditions. Out-of-distribution detection is commonly proposed as a solution for ensuring reliability in real-world deployment. Previous work has shown that normalizing flow models can be used for out-of-distribution detection to improve reliability of robotic perception tasks. Specifically, camera parameters can be optimized with respect to the likelihood output from a normalizing flow, which allows a perception system to adapt to difficult vision scenarios. With this work we propose to use the absolute gradient values from a normalizing flow, which allows the perception system to optimize local regions rather than the whole image. By setting up a table top picking experiment with exceptionally difficult lighting conditions, we show that our method achieves a 60% higher success rate for an object detection task compared to previous methods.

The visible orientation of human eyes creates some transparency about people's spatial attention and other mental states. This leads to a dual role for the eyes as a means of sensing and communication. Accordingly, artificial eye models are being explored as communication media in human-machine interaction scenarios. One challenge in the use of eye models for communication consists of resolving spatial reference ambiguities, especially for screen-based models. Here, we introduce an approach for overcoming this challenge through the introduction of reflection-like features that are contingent on artificial eye movements. We conducted a user study with 30 participants in which participants had to use spatial references provided by dynamic eye models to advance in a fast-paced group interaction task. Compared to a non-reflective eye model and a pure reflection mode, their combination in the new approach resulted in a higher identification accuracy and user experience, suggesting a synergistic benefit., Comment: 15 pages, 8 figures, 4 tables

In 1966, Erd\H{o}s, Goodman, and P\'{o}sa showed that if $G$ is an $n$-vertex graph, then at most $\lfloor n^2/4 \rfloor$ cliques of $G$ are needed to cover the edges of $G$, and the bound is best possible as witnessed by the balanced complete bipartite graph. This was generalized independently by Gy\H{o}ri--Kostochka, Kahn, and Chung, who showed that every $n$-vertex graph admits an edge-decomposition into cliques of total `cost' at most $2 \lfloor n^2/4 \rfloor$, where an $i$-vertex clique has cost $i$. Erd\H{o}s suggested the following strengthening: every $n$-vertex graph admits an edge-decomposition into cliques of total cost at most $\lfloor n^2/4 \rfloor$, where now an $i$-vertex clique has cost $i-1$. We prove fractional relaxations and asymptotically optimal versions of both this conjecture and a conjecture of Dau, Milenkovic, and Puleo on covering the $t$-vertex cliques of a graph instead of the edges. Our proofs introduce a general framework for these problems using Zykov symmetrization, the Frankl-R\"odl nibble method, and the Szemer\'edi Regularity Lemma.

HD 60435 is a well-known rapidly oscillating (roAp) Ap star with a series of alternating even and odd degree modes, making it a prime asteroseismic target. It is also an oblique pulsator with rotational inclination, $i$, and magnetic/pulsation obliquity, $\beta$, such that both magnetic/pulsation poles are viewed over the rotation period, $P_{\rm rot} = 7.679696$ d, determined from rotational light variations. While some roAp stars have stable pulsation mode amplitudes over decades, HD 60435 is known to have amplitude variations on time scales as short as 1 d. We show from 5 yr of {\it TESS} observations that there is strong amplitude modulation on this short time scale with possible mode interactions. Most remarkably, HD 60435 stopped pulsating during the time span of the {\it TESS} observations. This is the first time that any pulsating star has been observed to cease pulsating entirely. That has implications for mode interaction, excitation and damping, and is relevant to the problem of why only some stars in many pulsation instability strips pulsate, while others do not. During a 24.45-d time span of the {\it TESS} data when there was mode stability for a dipole mode and a quadrupole mode, the oblique pulsator model constrained $i$ and $\beta$, which we used to model those modes with a magnetic pulsation model from which we determined a polar field strength of 4 kG, in good agreement with a known magnetic measurement. We modelled the frequency separations showing that they can constrain the global metallicity, something that is not possible from spectroscopy of the highly peculiar Ap atmosphere., Comment: 19 pages plus an appendix. 13 figures. Accepted for publication in MNRAS

We analyze heat transfer and Casimir forces involving a nonreciprocal nanoparticle. By dissecting the resulting expressions into reciprocal and nonreciprocal contributions, we find that the particle's self emission contains $++$ and $--$ terms, i.e., the particle's reciprocal ($+$) and nonreciprocal ($-$) parts couple to the respective parts of its surrounding. In contrast, the heat transfer to the nanoparticle from the surrounding contains $-+$ and $+-$ contributions, which we find to persist at equal temperatures. For two nanoparticles, such persistent transfer is found to require one particle to be nonreciprocal and the other to be anisotropic. The propulsion force for the nanoparticle, for which our results agree with previous work, is dominated by $\pm\mp$ terms, making it distinct from forces found for reciprocal particles. The amplitude of the propulsion force can be orders of magnitude larger than gravitational forces. Despite being distinct, we find the $\pm\mp$ terms to be bound by $\pm\pm$ terms, a consequence of passivity of the objects. For the force, this bound limits the efficiency in a heat engine setup, as observed for parallel plates before., Comment: 14 pages, 5 figures

Machine unlearning is gaining increasing attention as a way to remove adversarial data poisoning attacks from already trained models and to comply with privacy and AI regulations. The objective is to unlearn the effect of undesired data from a trained model while maintaining performance on the remaining data. This paper introduces HyperForget, a novel machine unlearning framework that leverages hypernetworks - neural networks that generate parameters for other networks - to dynamically sample models that lack knowledge of targeted data while preserving essential capabilities. Leveraging diffusion models, we implement two Diffusion HyperForget Networks and used them to sample unlearned models in Proof-of-Concept experiments. The unlearned models obtained zero accuracy on the forget set, while preserving good accuracy on the retain sets, highlighting the potential of HyperForget for dynamic targeted data removal and a promising direction for developing adaptive machine unlearning algorithms., Comment: AdvML-Frontiers'24: The 3rd Workshop on New Frontiers in Adversarial Machine Learning@NeurIPS'24, Vancouver, CA

In this article, we provide an explicit description of the Lipschitz saturation $A^*_{B,R}$ of a subalgebra $A\subseteq R[[t^{\gamma_1},\ldots, t^{\gamma_n}]]$ over a ring $R$ in terms of the numerical semigroup $\Gamma = \langle\gamma_{1},\ldots,\gamma_{n}\rangle$ with mild conditions imposed on $R$.

The probability current is a vital quantity in the Fokker-Planck description of stochastic processes. It characterizes nonequilibrium stationary states, appears in linear response calculation, and has been related to the entropy production and the heat flux. We recover and review the probability current in the Onsager-Machlup functional approach to Markov processes. We derive a self contained expression for the stationary probability current and the non-equilibrium fluctuation-dissipation theorem using field theoretical methods. The derived formulas are explicitly evaluated in the Ornstein-Uhlenbeck process of a harmonically bound particle in shear flow as exemplary analytic expressions. Our work closes a gap since it removes a missing link, i.e.~the probability current, in the supposed equivalence of the Fokker-Planck and the path-integral approach., Comment: 17 pages

The Novikov-Shubin invariant associated to a graph provides information about the accumulation of eigenvalues of the corresponding adjacency matrix close to the origin. For a directed graph these eigenvalues lie in the complex plane and having a finite value for the planar Novikov-Shubin invariant indicates a polynomial behaviour of the eigenvalue density as a function of the distance to zero. We provide a complete description of these invariants for dense random digraphs with constant batch sizes, i.e. for the directed stochastic block model. The invariants depend only on which batches in the graph are connected by non-zero edge densities. We present an explicit finite step algorithm for their computation. For the proof we identify the asymptotic spectral density with the distribution of a $\mathbb{C}^K$-valued circular element in operator-valued free probability theory. We determine the spectral density in the bulk regime by solving the associated Dyson equation and infer the singular behaviour of this density close to the origin by determining the exponents associated to the power law with which the resolvent entries of the adjacency matrix that corresponds to the individual batches diverge to infinity or converge to zero.

Developing physics-based models for molecular simulation requires fitting many unknown parameters to experimental datasets of diverse properties. Fitting such diverse data is typically carried out piecemeal, resulting in a "zoo" of disconnected but highly related models. Here, we leverage recently developed methods for computing low-variance gradient estimates with automatic differentiation to facilitate the extensible, reproducible, and transparent fitting of the physics-based DNA model oxDNA. We demonstrate simultaneous fitting to structural, mechanical, and thermodynamic data using a variety of techniques, including enhanced sampling, external forcing, cluster-move Monte Carlo schemes, and micro- and millisecond timescales, and fit oxDNA to a range of target properties, such as twist-stretch couplings and duplex melting temperatures. We also demonstrate how gradients can serve as an efficient means of sensitivity analysis through which we uncover the surprising importance of the cross-stacking interaction in oxDNA. Though grounded in oxDNA, this work provides a framework for extensible, community-driven force field development more generally that will yield rapidly improved models and subsequent physical insights for systems ranging from RNA to protein to their interactions with inorganic materials.

A key objective of interpretability research on large language models (LLMs) is to develop methods for robustly steering models toward desired behaviors. To this end, two distinct approaches to interpretability -- ``bottom-up" and ``top-down" -- have been presented, but there has been little quantitative comparison between them. We present a case study comparing the effectiveness of representative vector steering methods from each branch: function vectors (FV; arXiv:2310.15213), as a bottom-up method, and in-context vectors (ICV; arXiv:2311.06668) as a top-down method. While both aim to capture compact representations of broad in-context learning tasks, we find they are effective only on specific types of tasks: ICVs outperform FVs in behavioral shifting, whereas FVs excel in tasks requiring more precision. We discuss the implications for future evaluations of steering methods and for further research into top-down and bottom-up steering given these findings.

Photoelectron diffraction (PED) is a powerful spectroscopic technique that combines elemental resolution with a high sensitivity to the local atomic arrangement at crystal surfaces, thus providing unique fingerprints of selected atomic sites in matter. Stimulated by the rapid innovation in the development of various analysis methods for probing the atomic and electronic structures of van der Waals (vdW) heterostructures of two-dimensional materials, we present a theoretical assessment of the capacity of PED for extracting structural properties such as stacking, twist angles and interlayer distances. We provide a complete description of PED for the benchmark vdW heterostructure bilayer graphene (BLG), by calculating and analyzing the PED of BLG in Bernal and AA-stacking as well as twisted BLG for a wide range of the twist angle., Comment: 10 pages, 7 figures

Transformers have demonstrated remarkable in-context learning capabilities across various domains, including statistical learning tasks. While previous work has shown that transformers can implement common learning algorithms, the adversarial robustness of these learned algorithms remains unexplored. This work investigates the vulnerability of in-context learning in transformers to \textit{hijacking attacks} focusing on the setting of linear regression tasks. Hijacking attacks are prompt-manipulation attacks in which the adversary's goal is to manipulate the prompt to force the transformer to generate a specific output. We first prove that single-layer linear transformers, known to implement gradient descent in-context, are non-robust and can be manipulated to output arbitrary predictions by perturbing a single example in the in-context training set. While our experiments show these attacks succeed on linear transformers, we find they do not transfer to more complex transformers with GPT-2 architectures. Nonetheless, we show that these transformers can be hijacked using gradient-based adversarial attacks. We then demonstrate that adversarial training enhances transformers' robustness against hijacking attacks, even when just applied during finetuning. Additionally, we find that in some settings, adversarial training against a weaker attack model can lead to robustness to a stronger attack model. Lastly, we investigate the transferability of hijacking attacks across transformers of varying scales and initialization seeds, as well as between transformers and ordinary least squares (OLS). We find that while attacks transfer effectively between small-scale transformers, they show poor transferability in other scenarios (small-to-large scale, large-to-large scale, and between transformers and OLS).

Zero-shot coordination (ZSC) is a popular setting for studying the ability of reinforcement learning (RL) agents to coordinate with novel partners. Prior ZSC formulations assume the $\textit{problem setting}$ is common knowledge: each agent knows the underlying Dec-POMDP, knows others have this knowledge, and so on ad infinitum. However, this assumption rarely holds in complex real-world settings, which are often difficult to fully and correctly specify. Hence, in settings where this common knowledge assumption is invalid, agents trained using ZSC methods may not be able to coordinate well. To address this limitation, we formulate the $\textit{noisy zero-shot coordination}$ (NZSC) problem. In NZSC, agents observe different noisy versions of the ground truth Dec-POMDP, which are assumed to be distributed according to a fixed noise model. Only the distribution of ground truth Dec-POMDPs and the noise model are common knowledge. We show that a NZSC problem can be reduced to a ZSC problem by designing a meta-Dec-POMDP with an augmented state space consisting of all the ground-truth Dec-POMDPs. For solving NZSC problems, we propose a simple and flexible meta-learning method called NZSC training, in which the agents are trained across a distribution of coordination problems - which they only get to observe noisy versions of. We show that with NZSC training, RL agents can be trained to coordinate well with novel partners even when the (exact) problem setting of the coordination is not common knowledge.

We investigate shear and interface modes excited in neutron stars with an elastic crust in the full general relativistic framework and compare them to the results obtained within the relativistic Cowling approximation. We observe that the Cowling approximation has virtually no impact on the frequencies or the eigenfunctions of the shear modes; in contrast, the interface modes that arise due to the discontinuities of the shear modulus experience a shift in frequency by several percent when applying the Cowling approximation. Furthermore, we extend a scheme based on the properties of Breit-Wigner resonances, which allows us to estimate the damping times of slowly damped modes; our extension can provide an estimation of the damping time even if the features of the Breit-Wigner resonance are incomplete or if some of them violate the underlying linearity assumption. The proposed scheme is also computationally less expensive and numerically more robust and we provide accurate estimates as well as lower bounds for damping times of shear and interface modes., Comment: 16 pages, 8 figures

One core aspect of immersive visualization labs is to develop and provide powerful tools and applications that allow for efficient analysis and exploration of scientific data. As the requirements for such applications are often diverse and complex, the same applies to the development process. This has led to a myriad of different tools, frameworks, and approaches that grew and developed over time. The steady advance of commercial off-the-shelf game engines such as Unreal Engine has made them a valuable option for development in immersive visualization labs. In this work, we share our experience of migrating to Unreal Engine as a primary developing environment for immersive visualization applications. We share our considerations on requirements, present use cases developed in our lab to communicate advantages and challenges experienced, discuss implications on our research and development environments, and aim to provide guidance for others within our community facing similar challenges., Comment: 11 + 3 pages, 9 figures, 1 table, published in Presence: Virtual and Augmented Reality

In this paper, we propose an approach that combines Vision Language Models (VLMs) and Behavior Trees (BTs) to address failures in robotics. Current robotic systems can handle known failures with pre-existing recovery strategies, but they are often ill-equipped to manage unknown failures or anomalies. We introduce VLMs as a monitoring tool to detect and identify failures during task execution. Additionally, VLMs generate missing conditions or skill templates that are then incorporated into the BT, ensuring the system can autonomously address similar failures in future tasks. We validate our approach through simulations in several failure scenarios.

Sparse Autoencoders (SAEs) have shown promise in improving the interpretability of neural network activations, but can learn features that are not features of the input, limiting their effectiveness. We propose \textsc{Mutual Feature Regularization} \textbf{(MFR)}, a regularization technique for improving feature learning by encouraging SAEs trained in parallel to learn similar features. We motivate \textsc{MFR} by showing that features learned by multiple SAEs are more likely to correlate with features of the input. By training on synthetic data with known features of the input, we show that \textsc{MFR} can help SAEs learn those features, as we can directly compare the features learned by the SAE with the input features for the synthetic data. We then scale \textsc{MFR} to SAEs that are trained to denoise electroencephalography (EEG) data and SAEs that are trained to reconstruct GPT-2 Small activations. We show that \textsc{MFR} can improve the reconstruction loss of SAEs by up to 21.21\% on GPT-2 Small, and 6.67\% on EEG data. Our results suggest that the similarity between features learned by different SAEs can be leveraged to improve SAE training, thereby enhancing performance and the usefulness of SAEs for model interpretability.

As reinforcement learning agents become increasingly deployed in real-world scenarios, predicting future agent actions and events during deployment is important for facilitating better human-agent interaction and preventing catastrophic outcomes. This paper experimentally evaluates and compares the effectiveness of future action and event prediction for three types of RL agents: explicitly planning, implicitly planning, and non-planning. We employ two approaches: the inner state approach, which involves predicting based on the inner computations of the agents (e.g., plans or neuron activations), and a simulation-based approach, which involves unrolling the agent in a learned world model. Our results show that the plans of explicitly planning agents are significantly more informative for prediction than the neuron activations of the other types. Furthermore, using internal plans proves more robust to model quality compared to simulation-based approaches when predicting actions, while the results for event prediction are more mixed. These findings highlight the benefits of leveraging inner states and simulations to predict future agent actions and events, thereby improving interaction and safety in real-world deployments., Comment: 16 pages, 8 figures

We present the results of the "Fast Calorimeter Simulation Challenge 2022" - the CaloChallenge. We study state-of-the-art generative models on four calorimeter shower datasets of increasing dimensionality, ranging from a few hundred voxels to a few tens of thousand voxels. The 31 individual submissions span a wide range of current popular generative architectures, including Variational AutoEncoders (VAEs), Generative Adversarial Networks (GANs), Normalizing Flows, Diffusion models, and models based on Conditional Flow Matching. We compare all submissions in terms of quality of generated calorimeter showers, as well as shower generation time and model size. To assess the quality we use a broad range of different metrics including differences in 1-dimensional histograms of observables, KPD/FPD scores, AUCs of binary classifiers, and the log-posterior of a multiclass classifier. The results of the CaloChallenge provide the most complete and comprehensive survey of cutting-edge approaches to calorimeter fast simulation to date. In addition, our work provides a uniquely detailed perspective on the important problem of how to evaluate generative models. As such, the results presented here should be applicable for other domains that use generative AI and require fast and faithful generation of samples in a large phase space., Comment: 204 pages, 100+ figures, 30+ tables

Deep neural networks have proven to be extremely powerful, however, they are also vulnerable to adversarial attacks which can cause hazardous incorrect predictions in safety-critical applications. Certified robustness via randomized smoothing gives a probabilistic guarantee that the smoothed classifier's predictions will not change within an $\ell_2$-ball around a given input. On the other hand (uncertainty) score-based rejection is a technique often applied in practice to defend models against adversarial attacks. In this work, we fuse these two approaches by integrating a classifier that abstains from predicting when uncertainty is high into the certified robustness framework. This allows us to derive two novel robustness guarantees for uncertainty aware classifiers, namely (i) the radius of an $\ell_2$-ball around the input in which the same label is predicted and uncertainty remains low and (ii) the $\ell_2$-radius of a ball in which the predictions will either not change or be uncertain. While the former provides robustness guarantees with respect to attacks aiming at increased uncertainty, the latter informs about the amount of input perturbation necessary to lead the uncertainty aware model into a wrong prediction. Notably, this is on CIFAR10 up to 20.93% larger than for models not allowing for uncertainty based rejection. We demonstrate, that the novel framework allows for a systematic robustness evaluation of different network architectures and uncertainty measures and to identify desired properties of uncertainty quantification techniques. Moreover, we show that leveraging uncertainty in a smoothed classifier helps out-of-distribution detection.

The increasing integration of machine learning across various domains has underscored the necessity for accessible systems that non-experts can utilize effectively. To address this need, the field of automated machine learning (AutoML) has developed tools to simplify the construction and optimization of ML pipelines. However, existing AutoML solutions often lack efficiency in creating online pipelines and ease of use for Human-Computer Interaction (HCI) applications. Therefore, in this paper, we introduce AdaptoML-UX, an adaptive framework that incorporates automated feature engineering, machine learning, and incremental learning to assist non-AI experts in developing robust, user-centered ML models. Our toolkit demonstrates the capability to adapt efficiently to diverse problem domains and datasets, particularly in HCI, thereby reducing the necessity for manual experimentation and conserving time and resources. Furthermore, it supports model personalization through incremental learning, customizing models to individual user behaviors. HCI researchers can employ AdaptoML-UX (\url{https://github.com/MichaelSargious/AdaptoML_UX}) without requiring specialized expertise, as it automates the selection of algorithms, feature engineering, and hyperparameter tuning based on the unique characteristics of the data.

Representation engineering methods have recently shown promise for enabling efficient steering of model behavior. However, evaluation pipelines for these methods have primarily relied on subjective demonstrations, instead of quantitative, objective metrics. We aim to take a step towards addressing this issue by advocating for four properties missing from current evaluations: (i) contexts sufficiently similar to downstream tasks should be used for assessing intervention quality; (ii) model likelihoods should be accounted for; (iii) evaluations should allow for standardized comparisons across different target behaviors; and (iv) baseline comparisons should be offered. We introduce an evaluation pipeline grounded in these criteria, offering both a quantitative and visual analysis of how effectively a given method works. We use this pipeline to evaluate two representation engineering methods on how effectively they can steer behaviors such as truthfulness and corrigibility, finding that some interventions are less effective than previously reported., Comment: Accepted to the NeurIPS 2024 - Workshop on Foundation Model Interventions

This study investigates the risks of exposing confidential chemical structures when machine learning models trained on these structures are made publicly available. We use membership inference attacks, a common method to assess privacy that is largely unexplored in the context of drug discovery, to examine neural networks for molecular property prediction in a black-box setting. Our results reveal significant privacy risks across all evaluated datasets and neural network architectures. Combining multiple attacks increases these risks. Molecules from minority classes, often the most valuable in drug discovery, are particularly vulnerable. We also found that representing molecules as graphs and using message-passing neural networks may mitigate these risks. We provide a framework to assess privacy risks of classification models and molecular representations. Our findings highlight the need for careful consideration when sharing neural networks trained on proprietary chemical structures, informing organisations and researchers about the trade-offs between data confidentiality and model openness.

The real-time dynamics of local magnetic moments exchange coupled to a metallic system of conduction electrons is subject to dissipative friction even in the absence of spin-orbit coupling. Phenomenologically, this is usually described by a local Gilbert damping constant. Here, we use both linear response theory and adiabatic response theory to derive the spin friction microscopically for a generic single-band tight-binding model of the electronic structure. The resulting Gilbert damping is time-dependent and nonlocal. For a one-dimensional model, we compare the emergent relaxation dynamics as obtained from LRT and ART against each other and against the full solution of the microscopic equations of motion and demonstrate the importance of nonlocality, while the time dependence turns out to be irrelevant. In two dimensions and for a few magnetic moments in different geometries, it is found that the inclusion of nonlocal Gilbert damping can counterintuitively lead to longer relaxation times. Besides the distance dependence, the directional dependence of the nonlocal Gilbert damping turns out as very important. Our results are based on an expression relating the nonlocal Gilbert damping to the nonlocal tight-binding density of states close to the Fermi energy. This is exact in case of noninteracting electrons. Effects due to electronic correlations are studied within the random-phase approximation. For the Hubbard model at half filling and with increasing interaction strength, we find a strong enhancement of the nonlocality of spin friction., Comment: 21 pages, 17 figures, v2 with minor changes

Diffusion models have led to significant advancements in generative modelling. Yet their widespread adoption poses challenges regarding data attribution and interpretability. In this paper, we aim to help address such challenges in diffusion models by developing an influence functions framework. Influence function-based data attribution methods approximate how a model's output would have changed if some training data were removed. In supervised learning, this is usually used for predicting how the loss on a particular example would change. For diffusion models, we focus on predicting the change in the probability of generating a particular example via several proxy measurements. We show how to formulate influence functions for such quantities and how previously proposed methods can be interpreted as particular design choices in our framework. To ensure scalability of the Hessian computations in influence functions, we systematically develop K-FAC approximations based on generalised Gauss-Newton matrices specifically tailored to diffusion models. We recast previously proposed methods as specific design choices in our framework and show that our recommended method outperforms previous data attribution approaches on common evaluations, such as the Linear Data-modelling Score (LDS) or retraining without top influences, without the need for method-specific hyperparameter tuning.

Autoencoders have been used for finding interpretable and disentangled features underlying neural network representations in both image and text domains. While the efficacy and pitfalls of such methods are well-studied in vision, there is a lack of corresponding results, both qualitative and quantitative, for the text domain. We aim to address this gap by training sparse autoencoders (SAEs) on a synthetic testbed of formal languages. Specifically, we train SAEs on the hidden representations of models trained on formal languages (Dyck-2, Expr, and English PCFG) under a wide variety of hyperparameter settings, finding interpretable latents often emerge in the features learned by our SAEs. However, similar to vision, we find performance turns out to be highly sensitive to inductive biases of the training pipeline. Moreover, we show latents correlating to certain features of the input do not always induce a causal impact on model's computation. We thus argue that causality has to become a central target in SAE training: learning of causal features should be incentivized from the ground-up. Motivated by this, we propose and perform preliminary investigations for an approach that promotes learning of causally relevant features in our formal language setting., Comment: NeurIPS workshop on Foundation Model Interventions (Awarded best paper)

We present a proposal for a future light dark matter search experiment at the Electron Stretcher Accelerator ELSA in Bonn: Lohengrin. It employs the fixed-target missing momentum based technique for searching for dark-sector particles. The Lohengrin experiment uses a high intensity electron beam that is shot onto a thin target to produce mainly SM bremsstrahlung and - in rare occasions - possibly new particles coupling feebly to the electron. A well motivated candidate for such a new particle is the dark photon, a new massive gauge boson arising from a new gauge interaction in a dark sector and mixing kinetically with the standard model photon. The Lohengrin experiment is estimated to reach sensitivity to couplings small enough to explain the relic abundance of dark matter in various models for dark photon masses between approximately 1 MeV and approximately 100 MeV.

Machine learning has revolutionized the modeling of clinical timeseries data. Using machine learning, a Deep Neural Network (DNN) can be automatically trained to learn a complex mapping of its input features for a desired task. This is particularly valuable in Electronic Health Record (EHR) databases, where patients often spend extended periods in intensive care units (ICUs). Machine learning serves as an efficient method for extract meaningful information. However, many state-of-the-art (SOTA) methods for training DNNs demand substantial volumes of labeled data, posing significant challenges for clinics in terms of cost and time. Self-supervised learning offers an alternative by allowing practitioners to extract valuable insights from data without the need for costly labels. Yet, current SOTA methods often necessitate large data batches to achieve optimal performance, increasing computational demands. This presents a challenge when working with long clinical timeseries data. To address this, we propose an efficient method of contrastive pretraining tailored for long clinical timeseries data. Our approach utilizes an estimator for negative pair comparison, enabling effective feature extraction. We assess the efficacy of our pretraining using standard self-supervised tasks such as linear evaluation and semi-supervised learning. Additionally, our model demonstrates the ability to impute missing measurements, providing clinicians with deeper insights into patient conditions. We demonstrate that our pretraining is capable of achieving better performance as both the size of the model and the size of the measurement vocabulary scale. Finally, we externally validate our model, trained on the MIMIC-III dataset, using the eICU dataset. We demonstrate that our model is capable of learning robust clinical information that is transferable to other clinics.

Probing coherent quantum dynamics in light-matter interactions at the microscopic level requires high-repetition-rate isolated attosecond pulses (IAPs) in pump-probe experiments. To date, the generation of IAPs has been mainly limited to the kilohertz regime. In this work, we experimentally achieve attosecond control of extreme-ultraviolet (XUV) high harmonics in the wide-bandgap dielectric MgO, driven by a synthesized field of two femtosecond pulses at 800nm and 2000nm with relative phase stability. The resulting quasi-continuous harmonic plateau with ~ 9 eV spectral width centered around 16.5 eV photon energy can be tuned by the two-color phase and supports the generation of an IAP (~ 730 attoseconds), confirmed by numerical simulation based on three-band semiconductor Bloch equations. Leveraging the high-repetition-rate driver laser and the moderate intensity requirements of solid-state high-harmonic generation, we achieve IAP production at an unprecedented megahertz repetition rate, paving the way for all-solid compact XUV sources for IAP generation., Comment: 28 pages, 10 figures

Preference learning is a central component for aligning current LLMs, but this process can be vulnerable to data poisoning attacks. To address this concern, we introduce PoisonBench, a benchmark for evaluating large language models' susceptibility to data poisoning during preference learning. Data poisoning attacks can manipulate large language model responses to include hidden malicious content or biases, potentially causing the model to generate harmful or unintended outputs while appearing to function normally. We deploy two distinct attack types across eight realistic scenarios, assessing 21 widely-used models. Our findings reveal concerning trends: (1) Scaling up parameter size does not inherently enhance resilience against poisoning attacks; (2) There exists a log-linear relationship between the effects of the attack and the data poison ratio; (3) The effect of data poisoning can generalize to extrapolated triggers that are not included in the poisoned data. These results expose weaknesses in current preference learning techniques, highlighting the urgent need for more robust defenses against malicious models and data manipulation., Comment: Tingchen Fu and Fazl Barez are core research contributors

During the COVID-19 pandemic, it became evident that the effectiveness of applying intervention measures is significantly influenced by societal acceptance, which, in turn, is affected by the processes of opinion formation. This article explores one among the many possibilities of a coupled opinion-epidemic system. The findings reveal either intricate periodic patterns or chaotic dynamics, leading to substantial fluctuations in opinion distribution and, consequently, significant variations in the total number of infections over time. Interestingly, the model is exhibiting the protective pattern., Comment: 18 pages, 9 figures

Despite tremendous progress in developing deep-learning-based weather forecasting systems, their design space, including the impact of different design choices, is yet to be well understood. This paper aims to fill this knowledge gap by systematically analyzing these choices including architecture, problem formulation, pretraining scheme, use of image-based pretrained models, loss functions, noise injection, multi-step inputs, additional static masks, multi-step finetuning (including larger stride models), as well as training on a larger dataset. We study fixed-grid architectures such as UNet, fully convolutional architectures, and transformer-based models, along with grid-invariant architectures, including graph-based and operator-based models. Our results show that fixed-grid architectures outperform grid-invariant architectures, indicating a need for further architectural developments in grid-invariant models such as neural operators. We therefore propose a hybrid system that combines the strong performance of fixed-grid models with the flexibility of grid-invariant architectures. We further show that multi-step fine-tuning is essential for most deep-learning models to work well in practice, which has been a common practice in the past. Pretraining objectives degrade performance in comparison to supervised training, while image-based pretrained models provide useful inductive biases in some cases in comparison to training the model from scratch. Interestingly, we see a strong positive effect of using a larger dataset when training a smaller model as compared to training on a smaller dataset for longer. Larger models, on the other hand, primarily benefit from just an increase in the computational budget. We believe that these results will aid in the design of better weather forecasting systems in the future.

Vision-Language Models (VLMs) are powerful tools for processing and understanding text and images. We study the processing of visual tokens in the language model component of LLaVA, a prominent VLM. Our approach focuses on analyzing the localization of object information, the evolution of visual token representations across layers, and the mechanism of integrating visual information for predictions. Through ablation studies, we demonstrated that object identification accuracy drops by over 70\% when object-specific tokens are removed. We observed that visual token representations become increasingly interpretable in the vocabulary space across layers, suggesting an alignment with textual tokens corresponding to image content. Finally, we found that the model extracts object information from these refined representations at the last token position for prediction, mirroring the process in text-only language models for factual association tasks. These findings provide crucial insights into how VLMs process and integrate visual information, bridging the gap between our understanding of language and vision models, and paving the way for more interpretable and controllable multimodal systems.

We investigate feature universality in large language models (LLMs), a research field that aims to understand how different models similarly represent concepts in the latent spaces of their intermediate layers. Demonstrating feature universality allows discoveries about latent representations to generalize across several models. However, comparing features across LLMs is challenging due to polysemanticity, in which individual neurons often correspond to multiple features rather than distinct ones. This makes it difficult to disentangle and match features across different models. To address this issue, we employ a method known as dictionary learning by using sparse autoencoders (SAEs) to transform LLM activations into more interpretable spaces spanned by neurons corresponding to individual features. After matching feature neurons across models via activation correlation, we apply representational space similarity metrics like Singular Value Canonical Correlation Analysis to analyze these SAE features across different LLMs. Our experiments reveal significant similarities in SAE feature spaces across various LLMs, providing new evidence for feature universality.

Large Language Models (LLMs) are rapidly becoming commodity components of larger software systems. This poses natural security and privacy problems: poisoned data retrieved from one component can change the model's behavior and compromise the entire system, including coercing the model to spread confidential data to untrusted components. One promising approach is to tackle this problem at the system level via dynamic information flow (aka taint) tracking. Unfortunately, the traditional approach of propagating the most restrictive input label to the output is too conservative for applications where LLMs operate on inputs retrieved from diverse sources. In this paper, we propose a novel, more permissive approach to propagate information flow labels through LLM queries. The key idea behind our approach is to propagate only the labels of the samples that were influential in generating the model output and to eliminate the labels of unnecessary input. We implement and investigate the effectiveness of two variations of this approach, based on (i) prompt-based retrieval augmentation, and (ii) a $k$-nearest-neighbors language model. We compare these with the baseline of an introspection-based influence estimator that directly asks the language model to predict the output label. The results obtained highlight the superiority of our prompt-based label propagator, which improves the label in more than 85% of the cases in an LLM agent setting. These findings underscore the practicality of permissive label propagation for retrieval augmentation., Comment: 16 pages, 11 figures

Medical image analysis tasks often focus on regions or structures located in a particular location within the patient's body. Often large parts of the image may not be of interest for the image analysis task. When using deep-learning based approaches, this causes an unnecessary increases the computational burden during inference and raises the chance of errors. In this paper, we introduce CTARR, a novel generic method for CT Anatomical Region Recognition. The method serves as a pre-processing step for any deep learning-based CT image analysis pipeline by automatically identifying the pre-defined anatomical region that is relevant for the follow-up task and removing the rest. It can be used in (i) image segmentation to prevent false positives in anatomically implausible regions and speeding up the inference, (ii) image classification to produce image crops that are consistent in their anatomical context, and (iii) image registration by serving as a fast pre-registration step. Our proposed method is based on atlas registration and provides a fast and robust way to crop any anatomical region encoded as one or multiple bounding box(es) from any unlabeled CT scan of the brain, chest, abdomen and/or pelvis. We demonstrate the utility and robustness of the proposed method in the context of medical image segmentation by evaluating it on six datasets of public segmentation challenges. The foreground voxels in the regions of interest are preserved in the vast majority of cases and tasks (97.45-100%) while taking only fractions of a seconds to compute (0.1-0.21s) on a deep learning workstation and greatly reducing the segmentation runtime (2.0-12.7x). Our code is available at https://github.com/ThomasBudd/ctarr.

This paper addresses the problem of human-based driver support. Nowadays, driver support systems help users to operate safely in many driving situations. Nevertheless, these systems do not fully use the rich information that is available from sensing the human driver. In this paper, we therefore present a human-based risk model that uses driver information for improved driver support. In contrast to state of the art, our proposed risk model combines a) the current driver perception based on driver errors, such as the driver overlooking another vehicle (i.e., notice error), and b) driver personalization, such as the driver being defensive or confident. In extensive simulations of multiple interactive driving scenarios, we show that our novel human-based risk model achieves earlier warning times and reduced warning errors compared to a baseline risk model not using human driver information.

Orbital angular momentum (OAM) light beams for high-order harmonic generation (HHG) provide an additional degree of freedom to study the light-matter interaction at ultrafast timescales. A more sophisticated configuration is a perfect optical vortex (POV) beam, a light beam with a helical wavefront characterized by a phase singularity at its center and an azimuthal phase variation. POV beams are characterized by a radial profile which is independent of the OAM. Here we study the non-perturbative process of gas-phase HHG using a linearly polarized POV beam. We observe that the harmonics are emitted with similar divergence due the perfectness of the POV-driven harmonics. Furthermore, the topological charge upscaling is rigorously followed. We show that a POV beam is more advantageous than that of the Laguerre-Gaussian beam for cases where a large topological charge with a small core size is required. Our research establishes a pathway for producing bright structured extreme ultraviolet (XUV) coherent radiation sources--a pivotal tool with multifaceted applications across various technological domains.

We consider the Brown measure of $a+\mathfrak{c}$, where $a$ lies in a commutative tracial von Neumann algebra $\mathcal{B}$ and $\mathfrak{c}$ is a $\cal{B}$-valued circular element. Under certain regularity conditions on $a$ and the covariance of $\mathfrak{c}$ this Brown measure has a density with respect to the Lebesgue measure on the complex plane which is real analytic apart from jump discontinuities at the boundary of its support. With the exception of finitely many singularities this one-dimensional spectral edge is real analytic. We provide a full description of all possible edge singularities as well as all points in the interior, where the density vanishes. The edge singularities are classified in terms of their local edge shape while internal zeros of the density are classified in terms of the shape of the density locally around these points. We also show that each of these countably infinitely many singularity types occurs for an appropriate choice of $a$ when $\mathfrak{c}$ is a standard circular element. The Brown measure of $a+\mathfrak{c}$ arises as the empirical spectral distribution of a diagonally deformed non-Hermitian random matrix with independent entries when its dimension tends to infinity., Comment: 50 pages, 5 figures. Minor changes

Magnetotaxis is a well known phenomenon in swimming microorganisms which sense magnetic fields e.g. by incorporating crystalline magnetosomes. In designing artificial active matter with tunable dynamics, external magnetic fields can provide a versatile method for guidance. Here, it is the question what material properties are necessary to elicit a significant response. In this working paper, we document in experiments on self-propelling nematic microdroplets that the weak diamagnetic torques exerted by a sub-Tesla magnetic field are already sufficient to significantly affect the dynamics of these swimmers. We find a rich and nontrivial variety of dynamic modes by varying droplet size, state of confinement and magnetic field strength.

We explore near-field thermal radiation transport in nanoparticles embedded within a multilayer slab structure, focusing on dynamic modulation of heat flux via cavity interactions. Our findings reveal that by tuning the distance between reflectors and nanoparticles, thermal transport can be significantly suppressed or enhanced, driven by selective excitation of surface modes within the cavity. By precisely adjusting inter-slab gaps, we achieve multi-order control over thermal flux while maintaining stability across a broad range of configurations. Notably, internal slab arrangement plays a pivotal role, with compact designs yielding the most pronounced effects. This work unveils a novel mechanism for manipulating near-field heat transfer, with exciting potential for nanoscale thermal management and thermal sensing technologies.

Vision Based Navigation consists in utilizing cameras as precision sensors for GNC after extracting information from images. To enable the adoption of machine learning for space applications, one of obstacles is the demonstration that available training datasets are adequate to validate the algorithms. The objective of the study is to generate datasets of images and metadata suitable for training machine learning algorithms. Two use cases were selected and a robust methodology was developed to validate the datasets including the ground truth. The first use case is in-orbit rendezvous with a man-made object: a mockup of satellite ENVISAT. The second use case is a Lunar landing scenario. Datasets were produced from archival datasets (Chang'e 3), from the laboratory at DLR TRON facility and at Airbus Robotic laboratory, from SurRender software high fidelity image simulator using Model Capture and from Generative Adversarial Networks. The use case definition included the selection of algorithms as benchmark: an AI-based pose estimation algorithm and a dense optical flow algorithm were selected. Eventually it is demonstrated that datasets produced with SurRender and selected laboratory facilities are adequate to train machine learning algorithms., Comment: 6 pages, 4 figures, preprint of the proceedings of ESA SPAICE conference 2024

Let $\mathrm{ex}(n,H,\mathcal{F})$ be the maximum number of copies of $H$ in an $n$-vertex graph which contains no copy of a graph from $\mathcal{F}$. Thinking of $H$ and $\mathcal{F}$ as fixed, we study the asymptotics of $\mathrm{ex}(n,H,\mathcal{F})$ in $n$. We say that a rational number $r$ is \emph{realizable for $H$} if there exists a finite family $\mathcal{F}$ such that $\mathrm{ex}(n,H,\mathcal{F}) = \Theta(n^r)$. Using randomized algebraic constructions, Bukh and Conlon showed that every rational between $1$ and $2$ is realizable for $K_2$. We generalize their result to show that every rational between $1$ and $t$ is realizable for $K_t$, for all $t \geq 2$. We also determine the realizable rationals for stars and note the connection to a related Sidorenko-type supersaturation problem., Comment: 28 pages, 8 figures

We extend the concept of loss landscape mode connectivity to the input space of deep neural networks. Mode connectivity was originally studied within parameter space, where it describes the existence of low-loss paths between different solutions (loss minimizers) obtained through gradient descent. We present theoretical and empirical evidence of its presence in the input space of deep networks, thereby highlighting the broader nature of the phenomenon. We observe that different input images with similar predictions are generally connected, and for trained models, the path tends to be simple, with only a small deviation from being a linear path. Our methodology utilizes real, interpolated, and synthetic inputs created using the input optimization technique for feature visualization. We conjecture that input space mode connectivity in high-dimensional spaces is a geometric effect that takes place even in untrained models and can be explained through percolation theory. We exploit mode connectivity to obtain new insights about adversarial examples and demonstrate its potential for adversarial detection. Additionally, we discuss applications for the interpretability of deep networks.

Accurate 3D object detection is vital for automated driving. While lidar sensors are well suited for this task, they are expensive and have limitations in adverse weather conditions. 3+1D imaging radar sensors offer a cost-effective, robust alternative but face challenges due to their low resolution and high measurement noise. Existing 3+1D imaging radar datasets include radar and lidar data, enabling cross-modal model improvements. Although lidar should not be used during inference, it can aid the training of radar-only object detectors. We explore two strategies to transfer knowledge from the lidar to the radar domain and radar-only object detectors: 1. multi-stage training with sequential lidar point cloud thin-out, and 2. cross-modal knowledge distillation. In the multi-stage process, three thin-out methods are examined. Our results show significant performance gains of up to 4.2 percentage points in mean Average Precision with multi-stage training and up to 3.9 percentage points with knowledge distillation by initializing the student with the teacher's weights. The main benefit of these approaches is their applicability to other 3D object detection networks without altering their architecture, as we show by analyzing it on two different object detectors. Our code is available at https://github.com/rst-tu-dortmund/lerojd, Comment: Accepted for publication as ECCV 2024

Given a connected finite graph $G$, an integer-valued function $f$ on $V(G)$ is called $M$-Lipschitz if the value of $f$ changes by at most $M$ along the edges of $G$. In 2013, Peled, Samotij, and Yehudayoff showed that random $M$-Lipschitz functions on graphs with sufficiently good expansion typically exhibit small fluctuations, giving sharp bounds on the typical range of such functions, assuming $M$ is not too large. We prove that the same conclusion holds under a relaxed expansion condition and for larger $M$, (partially) answering questions of Peled et al. Our techniques involve a combination of Sapozhenko's graph container methods and entropy methods from information theory., Comment: 24 pages