Search

This paper presents a comprehensive overview of the data preparation pipeline developed for the OpenGPT-X project, a large-scale initiative aimed at creating open and high-performance multilingual large language models (LLMs). The project goal is to deliver models that cover all major European languages, with a particular focus on real-world applications within the European Union. We explain all data processing steps, starting with the data selection and requirement definition to the preparation of the final datasets for model training. We distinguish between curated data and web data, as each of these categories is handled by distinct pipelines, with curated data undergoing minimal filtering and web data requiring extensive filtering and deduplication. This distinction guided the development of specialized algorithmic solutions for both pipelines. In addition to describing the processing methodologies, we provide an in-depth analysis of the datasets, increasing transparency and alignment with European data regulations. Finally, we share key insights and challenges faced during the project, offering recommendations for future endeavors in large-scale multilingual data preparation for LLMs.

A standard method to reduce the linewidth of semiconductor lasers involves the use of external optical feedback (EOF). However, feedback powers less than 1 % usually trigger coherence collapse (CC), leading to chaotic laser dynamics and linewidth broadening. This paper explores a method to mitigate CC through precise tuning of the feedback polarization depending on the feedback power. We report a semiconductor laser with a sub-kHz linewidth, achieved via EOF. The laser features a U-shaped cavity with two sampled grating distributed Bragg reflectors (SG-DBRs), enabling broad tunability across a 42 nm wavelength range (1513-1555 nm). By injecting optical feedback into both sides of the laser cavity via an external fiber-based cavity, we reduce the linewidth by more than three orders of magnitude, from MHz to sub-kHz across the laser's tuning range. Our approach achieves significant linewidth reduction while maintaining coherence at high feedback levels, marking an improvement over prior studies where CC limited performance. These results pave the way for ultra-narrow linewidth diode lasers with wide tunability, which would benefit fields like coherent optical communications and spectroscopy.

The High Energy Modular Array (HEMA) is one of three instruments that compose the STROBE-X mission concept. The HEMA is a large-area, high-throughput non-imaging pointed instrument based on the Large Area Detector developed as part of the LOFT mission concept. It is designed for spectral timing measurements of a broad range of sources and provides a transformative increase in sensitivity to X-rays in the energy range of 2--30 keV compared to previous instruments, with an effective area of 3.4 m$^{2}$ at 8.5 keV and an energy resolution of better than 300 eV at 6 keV in its nominal field of regard., Comment: 16 pages, 10 figures

We present multi-epoch optical spectropolarimetric and imaging polarimetric observations of the nearby Type II supernova (SN) 2023ixf discovered in M101 at a distance of 6.85 Mpc. The first imaging polarimetric observations were taken +2.33 days (60085.08 MJD) after the explosion, while the last imaging polarimetric data points (+73.19 and +76.19 days) were acquired after the fall from the light curve plateau. At +2.33 days there is strong evidence of circumstellar material (CSM) interaction in the spectra and the light curve. A significant level of polarization $P_r = 0.88\pm 0.06 \% $ seen during this phase indicates that this CSM is aspherical. We find that the polarization evolves with time toward the interstellar polarization level ($0.35\%$) during the photospheric phase, which suggests that the recombination photosphere is spherically symmetric. There is a jump in polarization ($P_r =0.65 \pm 0.08 \% $) at +73.19 days when the light curve falls from the plateau. This is a phase where polarimetric data is sensitive to non-spherical inner ejecta or a decrease in optical depth into the single scattering regime. We also present spectropolarimetric data that reveal line (de)polarization during most of the observed epochs. In addition, at +14.50 days we see an "inverse P Cygn" profile in the H and He line polarization, which clearly indicates the presence of asymmetrically distributed material overlying the photosphere. The overall temporal evolution of polarization is typical for Type II SNe, but the high level of polarization during the rising phase has only been observed in SN 2023ixf., Comment: 14 pages, 7 figures, submitted to ApJL, comments welcome

We prove that if $G$ is a noncompact connected real reductive linear Lie group, then any discrete subgroup of $G$ acting properly discontinuously and cocompactly on some homogeneous space $G/H$ of $G$ is quasi-isometrically embedded and sharp for $G/H$, i.e. satisfies a strong, quantitative form of proper discontinuity. For noncompact reductive $H$, this was known as the Sharpness Conjecture, with applications to spectral analysis on pseudo-Riemannian locally symmetric spaces developed in arXiv:1209.4075. For $G/H$ rational of real corank one, we use sharpness to fully characterize properly discontinuous and cocompact actions on $G/H$ in terms of Anosov representations. This enables us to show that in real corank one, acting properly discontinuously and cocompactly on $G/H$ is an open property, and also to prove that a number of homogeneous spaces do not admit compact quotients, such as $\mathrm{SL}(n+1,\mathbb{K})/\mathrm{SL}(n,\mathbb{K})$ for $n>1$ and $\mathbb{K}=\mathbb{R}$, $\mathbb{C}$, or the quaternions., Comment: 45 pages

In a recently introduced coset guessing game, Alice plays against Bob and Charlie, aiming to meet a joint winning condition. Bob and Charlie can only communicate before the game starts to devise a joint strategy. The game we consider begins with Alice preparing a 2m-qubit quantum state based on a random selection of three parameters. She sends the first m qubits to Bob and the rest to Charlie and then reveals to them her choice for one of the parameters. Bob is supposed to guess one of the hidden parameters, Charlie the other, and they win if both guesses are correct. From previous work, we know that the probability of Bob's and Charlie's guesses being simultaneously correct goes to zero exponentially as m increases. We derive a tight upper bound on this probability and show how Bob and Charlie can achieve it. While developing the optimal strategy, we devised an encoding circuit using only CNOT and Hadamard gates, which could be relevant for building efficient CSS-coded systems. We found that the role of quantum information that Alice communicates to Bob and Charlie is to make their responses correlated rather than improve their individual (marginal) correct guessing rates.

Path planning is an essential component of autonomous driving. A global planner is responsible for the high-level planning. It basically performs a shortest-path search on a known map, thereby defining waypoints used to control the local (low-level) planner. Local planning is a runtime verification method which is repeatedly run on the vehicle itself in real-time, so as to find the optimal short-horizon path which leads to the desired waypoint in a way which is both efficient and safe. The challenge is that the local planner has to take into account repeatedly incoming updates about the information available of the environment. In addition, it performs a complex task, as it has to take into account a large variety of requirements, originating from the necessity of collision avoidance with obstacles, respecting traffic rules, sticking to regulatory requirements, and lastly to reach the next waypoint efficiently. In this paper, we describe a logic-based specification mechanism which fulfills all these requirements., Comment: 10 pages, 5 Figures, 1 Table, Accepted as a short paper at 27th Brazilian Symposium on Formal Methods (SBMF 2024)

With increasing amounts of data in astronomy, automated analysis methods have become crucial. Synthetic data are required for developing and testing such methods. Current simulations often suffer from insufficient detail or inaccurate representation of source type occurrences. To overcome those deficiencies, we implemented a deep generative model trained on observations to generate realistic radio galaxy images with full control over the flux and source morphology. We used a diffusion model, trained with continuous time steps to reduce sampling time without quality impairments. Two models were trained on two different datasets, respectively. One set was a selection of images from the second data release of the LOFAR Two-Metre Sky Survey (LoTSS). The model is conditioned on peak flux values to preserve signal intensity information after re-scaling image pixel values. The other, smaller set was obtained from the VLA survey of Faint Images of the Radio Sky at Twenty-Centimeters (FIRST). In that set, every image was provided with a morphological class label the corresponding model was conditioned on. Conditioned sampling is realized with classifier-free diffusion guidance. We evaluated the quality of generated images by comparing distributions of different quantities over the real and generated data, including results from the standard source-finding algorithms. The class conditioning was evaluated by training a classifier and comparing its performance on both real and generated data. We were able to generate realistic images of high quality using 25 sampling steps, which is unprecedented in the field of radio astronomy. The generated images are visually indistinguishable from the training data and the distributions of different image metrics were replicated. The classifier performs equally well for real and generated images, indicating strong sampling control over morphological properties.

Orthogonal nonnegative matrix factorization (ONMF) has become a standard approach for clustering. As far as we know, most works on ONMF rely on the Frobenius norm to assess the quality of the approximation. This paper presents a new model and algorithm for ONMF that minimizes the Kullback-Leibler (KL) divergence. As opposed to the Frobenius norm which assumes Gaussian noise, the KL divergence is the maximum likelihood estimator for Poisson-distributed data, which can model better sparse vectors of word counts in document data sets and photo counting processes in imaging. We develop an algorithm based on alternating optimization, KL-ONMF, and show that it performs favorably with the Frobenius-norm based ONMF for document classification and hyperspectral image unmixing., Comment: 10 pages, corrected some typos

Single metallocene molecules act as sensitive spin detectors when decorating the probe of a scanning tunneling microscope (STM). However, the impact of the atomic-scale electrode details on the molecular spin state has remained elusive to date. Here, a nickelocene (Nc) STM junction is manipulated in an atom-wise manner showing clearly the dependence of the spin excitation spectrum on the anchoring of Nc to Cu(111), a Cu monomer and trimer. Moreover, while the spin state of the same Nc tip is a triplet with tunable spin excitation energies upon contacting the surface, it transitions to a Kondo-screened doublet on a Cu atom. Notably, the non-trivial magnetic exchange interaction of the molecular spin with the electron continuum of the substrate determines the spectral line shape of the spin excitations.

Semiconductor quantum dots in microcavities are an excellent platform for the efficient generation of indistinguishable single photons. However, their use in a wide range of quantum technologies requires their controlled fabrication and integration in compact closed-cycle cryocoolers, with a key challenge being the efficient and stable extraction of the single photons into a single-mode fiber. Here we report on a novel method for fiber-pigtailing of deterministically fabricated single-photon sources. Our technique allows for nanometer-scale alignment accuracy between the source and a fiber, alignment that persists all the way from room temperature to 2.4 K. We demonstrate high performance of the device under near-resonant optical excitation with g$^{(2)}$(0) = 1.3 %, a photon indistinguishability of 97.5 % and a fibered brightness of 20.8 %. We show that the indistinguishability and single-photon rate are stable for over ten hours of continuous operation in a single cooldown. We further confirm that the device performance is not degraded by nine successive cooldown-warmup cycles., Comment: 7 pages, 5 figures

We demonstrate dual-frequency stabilization of a hybrid-integrated multi-channel laser to an integrated high Q-factor silicon nitride (SiN) coil resonator with more than 40 dB frequency noise suppression. The frequency locked channels are utilized for a proof-of-concept fiber sensing experiment., Comment: 4 pages, 3 figures, European Conference on Optical Communication (ECOC)

Dataset ownership verification, the process of determining if a dataset is used in a model's training data, is necessary for detecting unauthorized data usage and data contamination. Existing approaches, such as backdoor watermarking, rely on inducing a detectable behavior into the trained model on a part of the data distribution. However, these approaches have limitations, as they can be harmful to the model's performances or require unpractical access to the model's internals. Most importantly, previous approaches lack guarantee against false positives. This paper introduces data taggants, a novel non-backdoor dataset ownership verification technique. Our method uses pairs of out-of-distribution samples and random labels as secret keys, and leverages clean-label targeted data poisoning to subtly alter a dataset, so that models trained on it respond to the key samples with the corresponding key labels. The keys are built as to allow for statistical certificates with black-box access only to the model. We validate our approach through comprehensive and realistic experiments on ImageNet1k using ViT and ResNet models with state-of-the-art training recipes. Our findings demonstrate that data taggants can reliably make models trained on the protected dataset detectable with high confidence, without compromising validation accuracy, and demonstrates superiority over backdoor watermarking. Moreover, our method shows to be stealthy and robust against various defense mechanisms., Comment: 16 pages, 7 figures

We live in a world that is experiencing an unprecedented boom of AI applications that increasingly penetrate and enhance all sectors of private and public life, from education, media, medicine, and mobility to the industrial and professional workspace, and -- potentially particularly consequentially -- robotics. As this world is simultaneously grappling with climate change, the climate and environmental implications of the development and use of AI have become an important subject of public and academic debate. In this paper, we aim to provide guidance on the climate-related regulation for data centers and AI specifically, and discuss how to operationalize these requirements. We also highlight challenges and room for improvement, and make a number of policy proposals to this end. In particular, we propose a specific interpretation of the AI Act to bring reporting on the previously unadressed energy consumption from AI inferences back into the scope. We also find that the AI Act fails to address indirect greenhouse gas emissions from AI applications. Furthermore, for the purpose of energy consumption reporting, we compare levels of measurement within data centers and recommend measurement at the cumulative server level. We also argue for an interpretation of the AI Act that includes environmental concerns in the mandatory risk assessment (sustainability risk assessment, SIA), and provide guidance on its operationalization. The EU data center regulation proves to be a good first step but requires further development by including binding renewable energy and efficiency targets for data centers. Overall, we make twelve concrete policy proposals, in four main areas: Energy and Environmental Reporting Obligations; Legal and Regulatory Clarifications; Transparency and Accountability Mechanisms; and Future Far-Reaching Measures beyond Transparency., Comment: 18 pages, 1 figure, preprint

Inverse design (ID) is a computational method that systematically explores a design space to find optimal device geometries based on specific performance criteria. In silicon photonics, ID often leads to devices with design features that degrade significantly due to the fabrication process, limiting the applicability of these devices in scalable silicon photonic fabrication. We demonstrate a solution to this performance degradation through fabrication-aware inverse design (FAID), integrating lithography models for deep-ultraviolet (DUV) lithography and electron beam lithography (EBL) into the shape optimization approach of ID. A Y-branch and an SWG-to-strip converter were generated and fabricated with this new approach. Simulated and measured results verify that the FAID yields devices with up to 0.6 dB lower insertion loss per device. The modified workflow enables designers to use ID to generate devices that adjust for process bias predicted by lithography models., Comment: 4 pages

Research on non-verbal behavior generation for social interactive agents focuses mainly on the believability and synchronization of non-verbal cues with speech. However, existing models, predominantly based on deep learning architectures, often perpetuate biases inherent in the training data. This raises ethical concerns, depending on the intended application of these agents. This paper addresses these issues by first examining the influence of gender on facial non-verbal behaviors. We concentrate on gaze, head movements, and facial expressions. We introduce a classifier capable of discerning the gender of a speaker from their non-verbal cues. This classifier achieves high accuracy on both real behavior data, extracted using state-of-the-art tools, and synthetic data, generated from a model developed in previous work.Building upon this work, we present a new model, FairGenderGen, which integrates a gender discriminator and a gradient reversal layer into our previous behavior generation model. This new model generates facial non-verbal behaviors from speech features, mitigating gender sensitivity in the generated behaviors. Our experiments demonstrate that the classifier, developed in the initial phase, is no longer effective in distinguishing the gender of the speaker from the generated non-verbal behaviors.

In 1949, Zykov proposed the first explicit construction of triangle-free graphs with arbitrarily large chromatic number. We define a Zykov graph as any induced subgraph of a graph created using Zykov's construction. We give a structural characterization of Zykov graphs based on a specific type of stable set, that we call splitting stable set. It implies that recognizing this class is NP-complete, while being FPT in the treewidth of the input graph. We provide similar results for the Blanche Descartes construction.

We prove an interpolation theorem for nonlinear functionals defined on scales of Banach spaces that generalize Besov spaces. It applies to functionals defined only locally, requiring only some weak Lipschitz conditions, extending those introduced by Lions and Peetre. Our analysis is self-contained and independent of any previous results about interpolation theory. It depends solely on the concepts of Friedrichs' mollifiers, seen through the formalism introduced by Hamilton, combined with the frequency envelopes introduced by Tao and used recently by two of the authors and others to study the Cauchy problem for various quasilinear evolutions in partial differential equations. Inspired by this latter work, our main application states that, for an abstract flow map of a quasilinear problem, both the continuity of the flow as a function of time and the continuity of the data to solution map follow automatically from the estimates that are usually proven when establishing the existence of solutions: propagation of regularity via tame a priori estimates for higher regularities and contraction for weaker norms., Comment: Fixed typo in the title

Modeling based on differentiable programming holds great promise for astronomy, as it can employ techniques such as Hamiltonian Monte Carlo, gradient-based optimization, and other machine learning techniques. This new programming paradigm has motivated us to develop the first auto-differentiable spectrum model of exoplanets and brown dwarfs, ExoJAX (Kawahara et al. 2022). ExoJAX is designed to directly calculate cross-sections as functions of temperature and pressure, rather than interpolating tabulated data, to minimize errors in high-dispersion spectra modeling. However, its application was primarily proof-of-concept and limited to narrowband high-dispersion emission spectroscopy. In this paper, we have enhanced the differentiable opacity calculation using a new fast and memory-efficient algorithm, and have developed differentiable radiative transfer schemes, including emission, transmission, and reflection spectroscopy. These enhancements significantly expand the range of applications, as demonstrated through actual atmospheric retrievals: high-dispersion emission spectra of the brown dwarf GL229 B, medium-dispersion transmission spectra of the hot Saturn WASP-39 b from JWST, and high-dispersion reflection spectra of Jupiter. We obtained a C/O ratio for GL229 B consistent with its host star, constrained WASP-39 b's radial velocity from molecular fine structures at original resolution ($R \sim 3,000$), and estimated Jupiter's metallicity consistent with previous studies., Comment: 32 pages, 12 figures, submitted. ExoJAX is available at https://github.com/HajimeKawahara/exojax. ExoJAX2 is scheduled to be released after the review process

This paper addresses the problem of long-context linear system identification, where the state $x_t$ of a dynamical system at time $t$ depends linearly on previous states $x_s$ over a fixed context window of length $p$. We establish a sample complexity bound that matches the i.i.d. parametric rate up to logarithmic factors for a broad class of systems, extending previous works that considered only first-order dependencies. Our findings reveal a learning-without-mixing phenomenon, indicating that learning long-context linear autoregressive models is not hindered by slow mixing properties potentially associated with extended context windows. Additionally, we extend these results to (i) shared low-rank representations, where rank-regularized estimators improve rates with respect to dimensionality, and (ii) misspecified context lengths in strictly stable systems, where shorter contexts offer statistical advantages., Comment: 30 pages, 4 figures

In this note we analyze the performance of a simple root-finding algorithm in uniform attachment trees. The leaf-stripping algorithm recursively removes all leaves of the tree for a carefully chosen number of rounds. We show that, with probability $1 - \epsilon$, the set of remaining vertices contains the root and has a size only depending on $\epsilon$ but not on the size of the tree., Comment: 9 pages, 5 figures

Precise measurements of neutron star observables (such as mass and radius) allow one to constrain the equations of state for supranuclear matter and develop a stronger understanding of nuclear physics. The Neutron star Interior Composition ExploreR (NICER) tracks X-ray hotspots on rotating NSs and is able to infer precise information about the compactness of the star. Gravitational waves carry information about the tidal deformability (related to the tidal Love number) of neutron stars, which has been measured by the LIGO/Virgo/KAGRA collaboration. These two observables enjoy an approximately universal property between each other that is insensitive to the equations of state (the "universal Love-C relation"). In this paper, we focus on deriving two analytic expressions for the Love-C relations that are ready-to-use and improve upon previous analytic expressions. The first model is inspired by a Newtonian polytrope, whose perturbation to the gravitational potential can be found analytically. We extend this Newtonian model to the relativistic regime by providing a quadratic fit to the gravitational potential perturbation against stellar compactness. The second model makes use of the Tolman VII model and adopts a spectral expansion with Chebyshev polynomials, which converges faster than the Taylor expansions used in previous work. We find that the first model provides a more accurate description of the Love-C relation for realistic neutron stars than the second model, and it provides the best expression among all other analytic relations studied here in terms of describing the averaged numerical Love-C relation. These new models are not only useful in practice, but they also show the power and importance of analytic modeling of neutron stars., Comment: 11 pages, 9 figures

Effective image post-processing algorithms are vital for the successful direct imaging of exoplanets. Standard PSF subtraction methods use techniques based on a low-rank approximation to separate the rotating planet signal from the quasi-static speckles, and rely on signal-to-noise ratio maps to detect the planet. These steps do not interact or feed each other, leading to potential limitations in the accuracy and efficiency of exoplanet detection. We aim to develop a novel approach that iteratively finds the flux of the planet and the low-rank approximation of quasi-static signals, in an attempt to improve upon current PSF subtraction techniques. In this study, we extend the standard L2 norm minimization paradigm to an L1 norm minimization framework to better account for noise statistics in the high contrast images. Then, we propose a new method, referred to as Alternating Minimization Algorithm with Trajectory, that makes a more advanced use of estimating the low-rank approximation of the speckle field and the planet flux by alternating between them and utilizing both L1 and L2 norms. For the L1 norm minimization, we propose using L1 norm low-rank approximation, a low-rank approximation computed using an exact block-cyclic coordinate descent method, while we use randomized singular value decomposition for the L2 norm minimization. Additionally, we enhance the visibility of the planet signal using a likelihood ratio as a postprocessing step. Numerical experiments performed on a VLT/SPHERE-IRDIS dataset show the potential of AMAT to improve upon the existing approaches in terms of higher S/N, sensitivity limits, and ROC curves. Moreover, for a systematic comparison, we used datasets from the exoplanet data challenge to compare our algorithm to other algorithms in the challenge, and AMAT with likelihood ratio map performs better than most algorithms tested on the exoplanet data challenge., Comment: Submitted to A&A, 20 pages, 20 figures

We consider the discrete time infinite horizon average reward restless markovian bandit (RMAB) problem. We propose a \emph{model predictive control} based non-stationary policy with a rolling computational horizon $\tau$. At each time-slot, this policy solves a $\tau$ horizon linear program whose first control value is kept as a control for the RMAB. Our solution requires minimal assumptions and quantifies the loss in optimality in terms of $\tau$ and the number of arms, $N$. We show that its sub-optimality gap is $O(1/\sqrt{N})$ in general, and $\exp(-\Omega(N))$ under a local-stability condition. Our proof is based on a framework from dynamic control known as \emph{dissipativity}. Our solution easy to implement and performs very well in practice when compared to the state of the art. Further, both our solution and our proof methodology can easily be generalized to more general constrained MDP settings and should thus, be of great interest to the burgeoning RMAB community., Comment: Under review

The influence of Active Galactic Nuclei (AGN) on star formation within their host galaxies remains a topic of intense debate. One of the primary challenges in quantifying the star formation rate (SFR) within AGN hosts arises from the prevalent assumption in most methodologies, which attribute gas excitation to young stars alone. However, this assumption does not consider the contribution of the AGN to the ionization of the gas in their environment. To address this issue, we evaluate the use of strong optical emission lines to obtain the SFR surface density ($\Sigma{{\rm SFR_{AGN}}}$) in regions predominantly ionized by an AGN, using a sample of 293 AGN hosts from the MaNGA survey, with SFR measurements available through stellar population fitting. We propose calibrations involving the H$\alpha$ and [O\,{\sc iii}]$\lambda$5007 emission lines, which can be used to determine $\Sigma{{\rm SFR_{AGN}}}$, resulting in values consistent with those estimated through stellar population fitting., Comment: Accepted for publication in MNRAS

We study the robustness of parametrized tests of General Relativity (GR) with gravitational waves due to waveform inaccuracy. In particular, we determine the properties of the signal (signal-to-noise ratio (SNR) and source parameters) such that we are led to falsely identify a GR deviation due to neglecting spin precession or higher models in the recovery model. To characterize the statistical significance of the biases, we compute the Bayes factor between the non-GR and GR models, and the fitting factor of the non-GR model. For highly-precessing, edge-on signals, we find that mismodeling the signal leads to a significant systematic bias in the recovery of the non-GR parameter, even at an SNR of 30. However, these biased inferences are characterized by a significant loss of SNR and a weak preference for the non-GR model (over the GR model). At a higher SNR, the biased inferences display a strong preference for the non-GR model (over the GR model) and a significant loss of SNR. For edge-on signals containing asymmetric masses, at an SNR of 30, we find that excluding higher modes does not impact the ppE tests as much as excluding spin precession. Our analysis, therefore, identifies the spin-precessing and mass-asymmetric systems for which parametrized tests of GR are robust. With a toy model and using the linear signal approximation, we illustrate these regimes of bias and characterize them by obtaining bounds on the ratio of systematic to statistical error and the effective cycles incurred due to mismodeling. As a by-product of our analysis, we explicitly connect different measures and techniques commonly used to estimate systematic errors -- linear-signal approximation, Laplace approximation, fitting factor, effective cycles, and Bayes factor -- that are generally applicable to all studies of systematic uncertainties in gravitational wave parameter estimation., Comment: 51 pages, 22 figures, 5 tables

Supermassive black holes with masses $\gtrsim 10^9\,M_\odot$ have been discovered by JWST at high redshifts ($z\sim 7$). It is difficult to explain such objects as the result of accretive growth of stellar-mass seeds, as the rate at which baryons can be fed to the black hole is limited by the radiation pressure of the infalling matter. In this paper, we propose a new mechanism to create the early progenitors: the collapse of small dark matter halos through a dissipative cooling mechanism in the dark sector. These small black holes can then be efficiently fed with additional dissipative dark matter due to the comparatively weak interactions between the dark radiation and dark matter, which results in a very short Eddington time and high accretion rates., Comment: 11 pages, 3 figures

Head-to-head racing against opponents is a challenging and emerging topic in the domain of autonomous racing. We propose Predictive Spliner, a data-driven overtaking planner that learns the behavior of opponents through Gaussian Process (GP) regression, which is then leveraged to compute viable overtaking maneuvers in future sections of the racing track. Experimentally validated on a 1:10 scale autonomous racing platform using Light Detection and Ranging (LiDAR) information to perceive the opponent, Predictive Spliner outperforms State-of-the-Art (SotA) algorithms by overtaking opponents at up to 83.1% of its own speed, being on average 8.4% faster than the previous best-performing method. Additionally, it achieves an average success rate of 84.5%, which is 47.6% higher than the previous best-performing method. The method maintains computational efficiency with a Central Processing Unit (CPU) load of 22.79% and a computation time of 8.4 ms, evaluated on a Commercial off-the-Shelf (CotS) Intel i7-1165G7, making it suitable for real-time robotic applications. These results highlight the potential of Predictive Spliner to enhance the performance and safety of autonomous racing vehicles. The code for Predictive Spliner is available at: https://github.com/ForzaETH/predictive-spliner., Comment: Submitted to RA-L

Humans have the ability to learn new tasks by inferring high-level concepts from existing solution, then manipulating these concepts in lieu of the raw data. Can we automate this process by deriving latent semantic structures in a document collection using foundation models? We introduce fPLSA, a foundation-model-based Probabilistic Latent Semantic Analysis (PLSA) method that iteratively clusters and tags document segments based on document-level contexts. These tags can be used to model the structure of given documents and for hierarchical sampling of new texts. Our experiments on story writing, math, and multi-step reasoning datasets demonstrate that fPLSA tags help reconstruct the original texts better than existing tagging methods. Moreover, when used for hierarchical sampling, fPLSA produces more diverse outputs with a higher likelihood of hitting the correct answer than direct sampling and hierarchical sampling with existing tagging methods.

Does the choice of programming language affect energy consumption? Previous highly visible studies have established associations between certain programming languages and energy consumption. A causal misinterpretation of this work has led academics and industry leaders to use or support certain languages based on their claimed impact on energy consumption. This paper tackles this causal question directly. It first corrects and improves the measurement methodology used by prior work. It then develops a detailed causal model capturing the complex relationship between programming language choice and energy consumption. This model identifies and incorporates several critical but previously overlooked factors that affect energy usage. These factors, such as distinguishing programming languages from their implementations, the impact of the application implementations themselves, the number of active cores, and memory activity, can significantly skew energy consumption measurements if not accounted for. We show -- via empirical experiments, improved methodology, and careful examination of anomalies -- that when these factors are controlled for, notable discrepancies in prior work vanish. Our analysis suggests that the choice of programming language implementation has no significant impact on energy consumption beyond execution time., Comment: 18 pages

Large language models (LLMs) demonstrate remarkable emergent abilities to perform in-context learning across various tasks, including time series forecasting. This work investigates LLMs' ability to estimate probability density functions (PDFs) from data observed in-context; such density estimation (DE) is a fundamental task underlying many probabilistic modeling problems. We leverage the Intensive Principal Component Analysis (InPCA) to visualize and analyze the in-context learning dynamics of LLaMA-2 models. Our main finding is that these LLMs all follow similar learning trajectories in a low-dimensional InPCA space, which are distinct from those of traditional density estimation methods like histograms and Gaussian kernel density estimation (KDE). We interpret the LLaMA in-context DE process as a KDE with an adaptive kernel width and shape. This custom kernel model captures a significant portion of LLaMA's behavior despite having only two parameters. We further speculate on why LLaMA's kernel width and shape differs from classical algorithms, providing insights into the mechanism of in-context probabilistic reasoning in LLMs.

This study introduces a novel frequency-based data-driven controller for adaptive optics, using power spectral density for optimization while ensuring stability criteria. It addresses disturbance rejection, command amplitude constraints and system transfer functions through convex optimization to obtain an optimal control in an infinite input response filter form. Evaluated within the SAXO+ project, it demonstrates efficacy under diverse atmospheric conditions and operational scenarios. The proposed controller is tested in both standard and disentangled adaptive optics schemes, showcasing its adaptability and performance. Experimental validation is conducted using the COMPASS simulation tool, affirming the controller's promise for enhancing adaptive optics systems in real-world applications.

In this follow-up paper to [Local geometry of Equilibria and a Poincare-Bendixson-type Theorem for Holomorphic Flows, Nicolas Kainz, Dirk Lebiedz (2024)] we investigate the global topology and geometry of dynamical systems $\dot{x} = F(x)$ with entire vector field $F$ by exploiting and extending the local structure of finite elliptic decompositions. We prove topological properties of the basins of centers, nodes, and foci, while excluding isolated equilibria at the boundaries of the latter two. We propose a definition of global elliptic sectors and introduce the notion of a sector-forming orbit allowing global topological analysis. The structure of heteroclinic regions between two equilibria is characterized., Comment: 26 pages, 2 figures, to be published in Topology Proceedings

The isotopic compositions of samples returned from Cb-type asteroid Ryugu and Ivuna-type (CI) chondrites are distinct from other carbonaceous chondrites, which has led to the suggestion that Ryugu and CI chondrites formed in a different region of the accretion disk, possibly around the orbits of Uranus and Neptune. We show that, like for Fe, Ryugu and CI chondrites also have indistinguishable Ni isotope anomalies, which differ from those of other carbonaceous chondrites. We propose that this unique Fe and Ni isotopic composition reflects different accretion efficiencies of small FeNi metal grains among the carbonaceous chondrite parent bodies. The CI chondrites incorporated these grains more efficiently, possibly because they formed at the end of the disk's lifetime, when planetesimal formation was also triggered by photoevaporation of the disk. Isotopic variations among carbonaceous chondrites may thus reflect fractionation of distinct dust components from a common reservoir, implying CI chondrites and Ryugu may have formed in the same region of the accretion disk as other carbonaceous chondrites., Comment: Published open access in Science Advances

Existing preference optimization methods are mainly designed for directly learning from human feedback with the assumption that paired examples (preferred vs. dis-preferred) are available. In contrast, we propose a method that can leverage unpaired preferred or dis-preferred examples, and works even when only one type of feedback (positive or negative) is available. This flexibility allows us to apply it in scenarios with varying forms of feedback and models, including training generative language models based on human feedback as well as training policies for sequential decision-making problems, where learned (value) functions are available. Our approach builds upon the probabilistic framework introduced in (Dayan and Hinton, 1997), which proposes to use expectation-maximization (EM) to directly optimize the probability of preferred outcomes (as opposed to classic expected reward maximization). To obtain a practical algorithm, we identify and address a key limitation in current EM-based methods: when applied to preference optimization, they solely maximize the likelihood of preferred examples, while neglecting dis-preferred samples. We show how one can extend EM algorithms to explicitly incorporate dis-preferred outcomes, leading to a novel, theoretically grounded, preference optimization algorithm that offers an intuitive and versatile way to learn from both positive and negative feedback.

Spin fluctuations in two-dimensional (2D) ferromagnets in the presence of crystalline lattice dislocations are investigated. We show the existence of topologically protected non-propagative modes that localize at dislocations. These in-gap states, coined as {\it magnonic zero-modes}, are characterized by the $Z_2$ topological invariant that derives from parity symmetry broken induced by sublattice magnetic anisotropy. We uncover that bulk topology existing in the perfect crystal is robust under the influence of lattice defects, which is monitored by the real-space Bott index. It is also revealed that the topology of zero-modes remains unaffected when bulk topology becomes trivial and is remarkably resilient against magnetic disorder. Our findings point to the intriguing relationship between topological lattice defects and the spectrum of topological spin excitations., Comment: 5 pages, 4 figures

Procedural video representation learning is an active research area where the objective is to learn an agent which can anticipate and forecast the future given the present video input, typically in conjunction with textual annotations. Prior works often rely on large-scale pretraining of visual encoders and prediction models with language supervision. However, the necessity and effectiveness of extending compute intensive pretraining to learn video clip sequences with noisy text supervision have not yet been fully validated by previous works. In this work, we show that a strong off-the-shelf frozen pretrained visual encoder, along with a well designed prediction model, can achieve state-of-the-art (SoTA) performance in forecasting and procedural planning without the need for pretraining the prediction model, nor requiring additional supervision from language or ASR. Instead of learning representations from pixel space, our method utilizes the latent embedding space of publicly available vision encoders. By conditioning on frozen clip-level embeddings from observed steps to predict the actions of unseen steps, our prediction model is able to learn robust representations for forecasting through iterative denoising - leveraging the recent advances in diffusion transformers (Peebles & Xie, 2023). Empirical studies over a total of five procedural learning tasks across four datasets (NIV, CrossTask, COIN and Ego4D-v2) show that our model advances the strong baselines in long-horizon action anticipation (+2.6% in Verb ED@20, +3.1% in Noun ED@20), and significantly improves the SoTA in step forecasting (+5.0%), task classification (+3.8%), and procedure planning tasks (up to +2.28% in success rate, +3.39% in mAcc, and +0.90% in mIoU)., Comment: 10 pages

The potential effectiveness of counterspeech as a hate speech mitigation strategy is attracting increasing interest in the NLG research community, particularly towards the task of automatically producing it. However, automatically generated responses often lack the argumentative richness which characterises expert-produced counterspeech. In this work, we focus on two aspects of counterspeech generation to produce more cogent responses. First, by investigating the tension between helpfulness and harmlessness of LLMs, we test whether the presence of safety guardrails hinders the quality of the generations. Secondly, we assess whether attacking a specific component of the hate speech results in a more effective argumentative strategy to fight online hate. By conducting an extensive human and automatic evaluation, we show how the presence of safety guardrails can be detrimental also to a task that inherently aims at fostering positive social interactions. Moreover, our results show that attacking a specific component of the hate speech, and in particular its implicit negative stereotype and its hateful parts, leads to higher-quality generations., Comment: To appear in Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (long paper)

While laparoscopic liver resection is less prone to complications and maintains patient outcomes compared to traditional open surgery, its complexity hinders widespread adoption due to challenges in representing the liver's internal structure. Laparoscopic intraoperative ultrasound offers efficient, cost-effective and radiation-free guidance. Our objective is to aid physicians in identifying internal liver structures using laparoscopic intraoperative ultrasound. We propose a patient-specific approach using preoperative 3D ultrasound liver volume to train a deep learning model for real-time identification of portal tree and branch structures. Our personalized AI model, validated on ex vivo swine livers, achieved superior precision (0.95) and recall (0.93) compared to surgeons, laying groundwork for precise vessel identification in ultrasound-based liver resection. Its adaptability and potential clinical impact promise to advance surgical interventions and improve patient care.

Ontologies often lack explicit disjointness declarations between classes, despite their usefulness for sophisticated reasoning and consistency checking in Knowledge Graphs. In this study, we explore the potential of Large Language Models (LLMs) to enrich ontologies by identifying and asserting class disjointness axioms. Our approach aims at leveraging the implicit knowledge embedded in LLMs, using prompt engineering to elicit this knowledge for classifying ontological disjointness. We validate our methodology on the DBpedia ontology, focusing on open-source LLMs. Our findings suggest that LLMs, when guided by effective prompt strategies, can reliably identify disjoint class relationships, thus streamlining the process of ontology completion without extensive manual input. For comprehensive disjointness enrichment, we propose a process that takes logical relationships between disjointness and subclass statements into account in order to maintain satisfiability and reduce the number of calls to the LLM. This work provides a foundation for future applications of LLMs in automated ontology enhancement and offers insights into optimizing LLM performance through strategic prompt design. Our code is publicly available on GitHub at https://github.com/n28div/llm-disjointness., Comment: Accepted at KBC-LM'24 workshop at ISWC 2024

Large language models (LLMs) have proven to be remarkably efficient, both across a wide range of natural language processing tasks and well beyond them. However, a comprehensive theoretical analysis of the origins of their impressive performance remains elusive. In this paper, we approach this challenging task by drawing an equivalence between generic autoregressive language models with vocabulary of size $T$ and context window of size $K$ and Markov chains defined on a finite state space of size $\mathcal{O}(T^K)$. We derive several surprising findings related to the existence of a stationary distribution of Markov chains that capture the inference power of LLMs, their speed of convergence to it, and the influence of the temperature on the latter. We then prove pre-training and in-context generalization bounds and show how the drawn equivalence allows us to enrich their interpretation. Finally, we illustrate our theoretical guarantees with experiments on several recent LLMs to highlight how they capture the behavior observed in practice., Comment: 49 pages, 17 figures

Air-vented ionization chambers exposed to clinical radiation beams may suffer from recombination during the drift of the charge carriers towards the electrodes. Thus, dosimetry protocols recommend the use of a correction factor, usually denominated saturation factor ($k_{\rm sat}$), to correct the ionization chamber readout for the incomplete collection of charge. The two-voltage method is the recommended methodology for the calculation of the saturation factor, however, it is based on the early Boag model, which only takes into account the presence of positive and negative ions in the ionization chamber and does not account for the electric field screening or the free electron contribution to the signal. The numerical simulation shows a better agreement with the experimental data than the current analytical theories in terms of charge collection efficiency. The classical two-voltage method, systematically overestimates the saturation factor, with differences increasing with dose per pulse also present at low dose per pulse. These results may have implications for the dosimetry with ionization chambers in therapy modalities that use a dose per pulse higher than conventional radiotherapy such as intraoperative radiotherapy but also in conventional dose per pulse for ionization chambers that suffer from significant charge recombination.

Let $G$ be a countable group with no finitely generated subgroup of exponential growth. We show that every action of $G$ on a countable set preserving a linear (respectively, circular) order can be realised as the restriction of some action by $C^1$ diffeomorphisms on an interval (respectively, the circle) to an invariant subset. As a consequence, every action of $G$ by homeomorphisms on a compact connected one-manifold can be made $C^1$ upon passing to a semi-conjugate action. The proof is based on a functional characterisation of groups of local subexponential growth., Comment: 14 pages

The infall of the LMC into the Milky Way (MW) has dynamical implications throughout the MW's dark matter halo. We study the impact of this merger on the statistical properties of populations of simulated stellar streams. Specifically, we investigate the radial and on-sky angular dependence of stream perturbations caused by the direct effect of stream-LMC interactions and/or the response of the MW dark matter halo. We use a time-evolving MW--LMC simulation described by basis function expansions to simulate streams. We quantify the degree of perturbation using a set of stream property statistics including the misalignment of proper motions with the stream track. In the outer halo, direct stream--LMC interactions produce a statistically significant effect, boosting the fraction of misaligned proper motions by ~25% compared to the model with no LMC. Moreover, there is on-sky angular dependence of stream perturbations:~the highest fractions of perturbed streams coincide with the same on-sky quadrant as the present-day LMC location. In the inner halo, the MW halo dipole response primarily drives stream perturbations, but it remains uncertain whether this is a detectable signature distinct from the LMC's influence. For the fiducial MW--LMC model, we find agreement between the predicted fraction of streams with significantly misaligned proper motions, $\bar{\vartheta}>10^{\circ}$, and Dark Energy Survey data. Finally, we predict this fraction for the Vera Rubin Large Synoptic Survey Telescope (LSST) footprint. Using LSST data will improve our constraints on dark matter models and LMC properties as it is sensitive to both., Comment: ApJ submitted. 22 pages, 8 figures

Knowledge claims are abundant in the literature on large language models (LLMs); but can we say that GPT-4 truly "knows" the Earth is round? To address this question, we review standard definitions of knowledge in epistemology and we formalize interpretations applicable to LLMs. In doing so, we identify inconsistencies and gaps in how current NLP research conceptualizes knowledge with respect to epistemological frameworks. Additionally, we conduct a survey of 100 professional philosophers and computer scientists to compare their preferences in knowledge definitions and their views on whether LLMs can really be said to know. Finally, we suggest evaluation protocols for testing knowledge in accordance to the most relevant definitions., Comment: EMNLP 2024

Clinnova, a collaborative initiative involving France, Germany, Switzerland, and Luxembourg, is dedicated to unlocking the power of precision medicine through data federation, standardization, and interoperability. This European Greater Region initiative seeks to create an interoperable European standard using artificial intelligence (AI) and data science to enhance healthcare outcomes and efficiency. Key components include multidisciplinary research centers, a federated biobanking strategy, a digital health innovation platform, and a federated AI strategy. It targets inflammatory bowel disease, rheumatoid diseases, and multiple sclerosis (MS), emphasizing data quality to develop AI algorithms for personalized treatment and translational research. The IHU Strasbourg (Institute of Minimal-invasive Surgery) has the lead in this initiative to develop the federated learning (FL) proof of concept (POC) that will serve as a foundation for advancing AI in healthcare. At its core, Clinnova-MS aims to enhance MS patient care by using FL to develop more accurate models that detect disease progression, guide interventions, and validate digital biomarkers across multiple sites. This technical report presents insights and key takeaways from the first cross-border federated POC on MS segmentation of MRI images within the Clinnova framework. While our work marks a significant milestone in advancing MS segmentation through cross-border collaboration, it also underscores the importance of addressing technical, logistical, and ethical considerations to realize the full potential of FL in healthcare settings.

Nanostructured high-index dielectrics have shown great promise as low-loss photonic platforms for wavefront control and enhancing optical nonlinearities. However, their potential as optomechanical resonators has remained unexplored. In this work, we investigate the generation and detection of coherent acoustic phonons in individual crystalline gallium phosphide nanodisks on silica in a pump-probe configuration. By pumping the dielectric above its bandgap energy and probing over its transparent region, we observe the radial breathing mode of the disk with an oscillation frequency around 10 GHz. We analyze the performance of nanoantennas of various sizes in the 300-600 nm diameter range at fixed 125 nm height and find that the detection efficiency is maximum near the fundamental anapole state, in agreement with numerical simulations. By comparing with reference gold nanodisk and nanorod plasmonic resonators, we find that the dielectric nanoantennas display a modulation amplitude up to ~5 times larger. We further demonstrate the launching of acoustic waves through the underlaying substrate and the mechanical coupling between two nanostructures placed 3 {\mu}m apart, laying the basis for photonic-phononic signal processing using dielectric nanoantennas.

Topological materials, such as topological insulators or semimetals, usually not only reveal the nontrivial properties of their electronic wavefunctions through the appearance of stable boundary modes, but also through very specific electromagnetic responses. The anisotropic longitudinal magnetoresistance of Weyl semimetals, for instance, carries the signature of the chiral anomaly of Weyl fermions. However for topological nodal line semimetals -- materials where the valence and conduction bands cross each other on one-dimensional curves in the three-dimensional Brillouin zone -- such a characteristic has been lacking. Here we report the discovery of a peculiar charge transport effect generated by topological nodal lines: a dissipationless transverse signal in the presence of coplanar electric and magnetic fields, which originates from a Zeeman-induced conversion of topological nodal lines into Weyl nodes under infinitesimally small magnetic fields. We evidence this dissipationless topological response in trigonal \ce{PtBi2} persisting up to room temperature, and unveil the extensive topological nodal lines in the band structure of this non-magnetic material. These findings provide a new pathway to engineer Weyl nodes by arbitrary small magnetic fields and reveal that bulk topological nodal lines can exhibit non-dissipative transport properties.

Understanding generalization of overparametrized neural networks remains a fundamental challenge in machine learning. Most of the literature mostly studies generalization from an interpolation point of view, taking convergence of parameters towards a global minimum of the training loss for granted. While overparametrized architectures indeed interpolated the data for typical classification tasks, this interpolation paradigm does not seem valid anymore for more complex tasks such as in-context learning or diffusion. Instead for such tasks, it has been empirically observed that the trained models goes from global minima to spurious local minima of the training loss as the number of training samples becomes larger than some level we call optimization threshold. While the former yields a poor generalization to the true population loss, the latter was observed to actually correspond to the minimiser of this true loss. This paper explores theoretically this phenomenon in the context of two-layer ReLU networks. We demonstrate that, despite overparametrization, networks often converge toward simpler solutions rather than interpolating the training data, which can lead to a drastic improvement on the test loss with respect to interpolating solutions. Our analysis relies on the so called early alignment phase, during which neurons align towards specific directions. This directional alignment, which occurs in the early stage of training, leads to a simplicity bias, wherein the network approximates the ground truth model without converging to the global minimum of the training loss. Our results suggest that this bias, resulting in an optimization threshold from which interpolation is not reached anymore, is beneficial and enhances the generalization of trained models.

The generation and manipulation of non-classical light states is central to emerging quantum technologies. Color centers in insulating crystals have been extensively studied for single-photon generation, but organic molecules immobilized on substrates have gained attention due to their superior scalability, large oscillator strengths, and tunable emission frequency. Here, we use first principles calculations to investigate the photoemission spectrum of organic molecules adsorbed on various 2D materials. Machine learning interatomic potentials are combined with density functional theory to accelerate the search for stable adsorption configurations. The calculated zero phonon line (ZPL) energies and emission lineshapes show excellent agreement with experiments. Our results indicate that the 2D substrate couples weakly to the molecular transitions and that emission characteristics are almost universal across different substrates. The unique effect of the 2D substrate is to introduce a sharp sideband(s) near the ZPL as a fingerprint of hindered rotational and translational modes of the molecule.