Search

Basket trials can efficiently evaluate a single treatment across multiple diseases with a common shared target. Prior methods for randomized basket trials required baskets to have the same sample and effect sizes. To that end, we developed a general randomized basket trial with an interim analysis (RaBIt) that allows for unequal sample sizes and effect sizes per basket. RaBIt is characterized by pruning at an interim stage and then analyzing a pooling of the remaining baskets. We derived the analytical power and type 1 error for the design. We first show that our results are consistent with the prior methods when the sample and effect sizes were the same across baskets. As we adjust the sample allocation between baskets, our threshold for the final test statistic becomes more stringent in order to maintain the same overall type 1 error. Finally, we notice that if we fix a sample size for the baskets proportional to their accrual rate, then at the cost of an almost negligible amount of power, the trial overall is expected to take substantially less time than the non-generalized version., Comment: 23 pages, 3 figures

Building pluralistic AI requires designing models that are able to be shaped to represent a wide range of value systems and cultures. Achieving this requires first being able to evaluate the degree to which a given model is capable of reflecting various personas. To this end, we propose a benchmark for evaluating the steerability of model personas as a function of prompting. Our design is based on a formal definition of prompt steerability, which analyzes the degree to which a model's joint behavioral distribution can be shifted from its baseline behavior. By defining steerability indices and inspecting how these indices change as a function of steering effort, we can estimate the steerability of a model across various persona dimensions and directions. Our benchmark reveals that the steerability of many current models is limited -- due to both a skew in their baseline behavior and an asymmetry in their steerability across many persona dimensions. We release an implementation of our benchmark at https://github.com/IBM/prompt-steering.

Using the electrostatic analogy, we derive an exact formula for the limiting Yang-Lee zero distribution in the random allocation model of general weights. This exhibits a real-space condensation phase transition, which is induced by a pressure change. The exact solution allows one to read off the scaling of the density of zeros at the critical point and the angle at which locus of zeros hits the critical point. Since the order of the phase transition and critical exponents can be tuned with a single parameter for several families of weights, the model provides a useful testing ground for verifying various relations between the distribution of zeros and the critical behavior, as well as for exploring the behavior of physical quantities in the mesoscopic regime, i.e., systems of large but finite size. The main result is that asymptotically the Yang-Lee zeros are images of a conformal mapping, given by the generating function for the weights, of uniformly distributed complex phases.

We perform a Bayesian analysis of anisotropy in binary galaxy spin directions in the Hyper-Suprime Cam Data Release 3 catalogue, in response to a recent claim that it exhibits a dipole (Shamir 2024). We find no significant evidence for anisotropy, or for a direction-independent spin probability that differs from 0.5. These results are unchanged allowing for a quadrupole or simply searching for a fixed anisotropy between any two hemispheres, and the Bayes factor indicates decisive evidence for the isotropic model. Our principled method contrasts with the statistic employed by Shamir 2024, which lacks a strong theoretical foundation., Comment: 3 pages, 1 figure. Accepted in RNAAS

Current and future large scale structure surveys aim to constrain the neutrino mass and the equation of state of dark energy. We aim to construct accurate and interpretable symbolic approximations to the linear and nonlinear matter power spectra as a function of cosmological parameters in extended $\Lambda$CDM models which contain massive neutrinos and non-constant equations of state for dark energy. This constitutes an extension of the syren-halofit emulators to incorporate these two effects, which we call syren-new (SYmbolic-Regression-ENhanced power spectrum emulator with NEutrinos and $W_0-w_a$). We also obtain a simple approximation to the derived parameter $\sigma_8$ as a function of the cosmological parameters for these models. Our results for the linear power spectrum are designed to emulate CLASS, whereas for the nonlinear case we aim to match the results of EuclidEmulator2. We compare our results to existing emulators and $N$-body simulations. Our analytic emulators for $\sigma_8$, the linear and nonlinear power spectra achieve root mean squared errors of 0.1%, 0.3% and 1.3%, respectively, across a wide range of cosmological parameters, redshifts and wavenumbers. We verify that emulator-related discrepancies are subdominant compared to observational errors and other modelling uncertainties when computing shear power spectra for LSST-like surveys. Our expressions have similar accuracy to existing (numerical) emulators, but are at least an order of magnitude faster, both on a CPU and GPU. Our work greatly improves the accuracy, speed and range of applicability of current symbolic approximations to the linear and nonlinear matter power spectra. We provide publicly available code for all symbolic approximations found., Comment: 18 pages, 15 figures

Large Language Models (LLMs) store and retrieve vast amounts of factual knowledge acquired during pre-training. Prior research has localized and identified mechanisms behind knowledge recall; however, it has primarily focused on English monolingual models. The question of how these processes generalize to other languages and multilingual LLMs remains unexplored. In this paper, we address this gap by conducting a comprehensive analysis of two highly multilingual LLMs. We assess the extent to which previously identified components and mechanisms of factual recall in English apply to a multilingual context. Then, we examine when language plays a role in the recall process, uncovering evidence of language-independent and language-dependent mechanisms.

We study how features emerge, disappear, and persist across models fine-tuned on different domains of text. More specifically, we start from a base one-layer Transformer language model that is trained on a combination of the BabyLM corpus, and a collection of Python code from The Stack. This base model is adapted to two new domains of text: TinyStories, and the Lua programming language, respectively; and then these two models are merged using these two models using spherical linear interpolation. Our exploration aims to provide deeper insights into the stability and transformation of features across typical transfer-learning scenarios using small-scale models and sparse auto-encoders.

LMCe055-1 was recently discovered in a survey for WRs in the Large Magellanic Cloud, and classified as a WN4/O4, a lower excitation version of the WN3/O3 class discovered as part of the same survey. Its absolute magnitude precluded it from being a WN4+O4 binary. OGLE photometry show shallow primary and secondary eclipses with a 2.2 day period. The spectral characteristics and short period pointed to a possible origin due to binary stripping. Such stripped WR binaries should be common but have proven elusive to identify conclusively. In order to establish its nature, we obtained HST UV and Magellan optical spectra, along with imaging. Our work shows that the WR emission and He II absorption arise in one star, and the He I absorption in another. The He I contributor is the primary of the 2.2-day system and exhibits ~300 km/sed radial velocity variations on that time scale. However, the WR star shows 30-40 km/sed radial velocity variations, with a likely 35-day period and a highly eccentric orbit. Possibly LMCe055-1 is a physical triple, but that would require the 2.2-day pair to have been captured by the WR star. A more likely explanation is that the WR star has an unseen companion in a 35-day orbit and that the 2.2-day pair is in a longer period orbit about the two. Such examples of multiple systems are well known among massive stars, such as HD 5980. Regardless, we argue that it is highly unlikely that the WR component of the LMCe055-1 system resulted from stripping., Comment: Accepted for publication in the Astrophysical Journal

LLM-as-a-Judge is a widely used method for evaluating the performance of Large Language Models (LLMs) across various tasks. We address the challenge of quantifying the uncertainty of LLM-as-a-Judge evaluations. While uncertainty quantification has been well-studied in other domains, applying it effectively to LLMs poses unique challenges due to their complex decision-making capabilities and computational demands. In this paper, we introduce a novel method for quantifying uncertainty designed to enhance the trustworthiness of LLM-as-a-Judge evaluations. The method quantifies uncertainty by analyzing the relationships between generated assessments and possible ratings. By cross-evaluating these relationships and constructing a confusion matrix based on token probabilities, the method derives labels of high or low uncertainty. We evaluate our method across multiple benchmarks, demonstrating a strong correlation between the accuracy of LLM evaluations and the derived uncertainty scores. Our findings suggest that this method can significantly improve the reliability and consistency of LLM-as-a-Judge evaluations.

As premodern texts are passed down over centuries, errors inevitably accrue. These errors can be challenging to identify, as some have survived undetected for so long precisely because they are so elusive. While prior work has evaluated error detection methods on artificially-generated errors, we introduce the first dataset of real errors in premodern Greek, enabling the evaluation of error detection methods on errors that genuinely accumulated at some stage in the centuries-long copying process. To create this dataset, we use metrics derived from BERT conditionals to sample 1,000 words more likely to contain errors, which are then annotated and labeled by a domain expert as errors or not. We then propose and evaluate new error detection methods and find that our discriminator-based detector outperforms all other methods, improving the true positive rate for classifying real errors by 5%. We additionally observe that scribal errors are more difficult to detect than print or digitization errors. Our dataset enables the evaluation of error detection methods on real errors in premodern texts for the first time, providing a benchmark for developing more effective error detection algorithms to assist scholars in restoring premodern works.

Contextual Self-Modulation (CSM) is a potent regularization mechanism for the Neural Context Flow (NCF) framework which demonstrates powerful meta-learning of physical systems. However, CSM has limitations in its applicability across different modalities and in high-data regimes. In this work, we introduce two extensions: $i$CSM, which expands CSM to infinite-dimensional tasks, and StochasticNCF, which improves scalability. These extensions are demonstrated through comprehensive experimentation on a range of tasks, including dynamical systems with parameter variations, computer vision challenges, and curve fitting problems. $i$CSM embeds the contexts into an infinite-dimensional function space, as opposed to CSM which uses finite-dimensional context vectors. StochasticNCF enables the application of both CSM and $i$CSM to high-data scenarios by providing an unbiased approximation of meta-gradient updates through a sampled set of nearest environments. Additionally, we incorporate higher-order Taylor expansions via Taylor-Mode automatic differentiation, revealing that higher-order approximations do not necessarily enhance generalization. Finally, we demonstrate how CSM can be integrated into other meta-learning frameworks with FlashCAVIA, a computationally efficient extension of the CAVIA meta-learning framework (Zintgraf et al. 2019). FlashCAVIA outperforms its predecessor across various benchmarks and reinforces the utility of bi-level optimization techniques. Together, these contributions establish a robust framework for tackling an expanded spectrum of meta-learning tasks, offering practical insights for out-of-distribution generalization. Our open-sourced library, designed for flexible integration of self-modulation into contextual meta-learning workflows, is available at \url{github.com/ddrous/self-mod}., Comment: 23 pages, 11 figures, 5 tables

Evaluation of large language model (LLM) outputs requires users to make critical judgments about the best outputs across various configurations. This process is costly and takes time given the large amounts of data. LLMs are increasingly used as evaluators to filter training data, evaluate model performance or assist human evaluators with detailed assessments. To support this process, effective front-end tools are critical for evaluation. Two common approaches for using LLMs as evaluators are direct assessment and pairwise comparison. In our study with machine learning practitioners (n=15), each completing 6 tasks yielding 131 evaluations, we explore how task-related factors and assessment strategies influence criteria refinement and user perceptions. Findings show that users performed more evaluations with direct assessment by making criteria task-specific, modifying judgments, and changing the evaluator model. We conclude with recommendations for how systems can better support interactions in LLM-assisted evaluations.

Natural language processing (NLP) in the medical domain can underperform in real-world applications involving small datasets in a non-English language with few labeled samples and imbalanced classes. There is yet no consensus on how to approach this problem. We evaluated a set of NLP models including BERT-like transformers, few-shot learning with sentence transformers (SetFit), and prompted large language models (LLM), using three datasets of radiology reports on magnetic resonance images of epilepsy patients in Danish, a low-resource language. Our results indicate that BERT-like models pretrained in the target domain of radiology reports currently offer the optimal performances for this scenario. Notably, the SetFit and LLM models underperformed compared to BERT-like models, with LLM performing the worst. Importantly, none of the models investigated was sufficiently accurate to allow for text classification without any supervision. However, they show potential for data filtering, which could reduce the amount of manual labeling required.

We report the first measurement of the azimuthal anisotropy of J$/\psi$ at forward rapidity ($1.2<|\eta|<2.2$) in Au$+$Au collisions at $\sqrt{s_{_{NN}}}=200$ GeV at the Relativistic Heavy Ion Collider. The data were collected by the PHENIX experiment in 2014 and 2016 with integrated luminosity of 14.5~nb$^{-1}$. The second Fourier coefficient ($v_2$) of the azimuthal distribution of $J/\psi$ is determined as a function of the transverse momentum ($p_T$) using the event-plane method. The measurements were performed for several selections of collision centrality: 0\%--50\%, 10\%--60\%, and 10\%-40\%. We find that in all cases the values of $v_2(p_T)$, which quantify the elliptic flow of J$/\psi$, are consistent with zero. The results are consistent with measurements at midrapidity, indicating no significant elliptic flow of the J$/\psi$ within the quark-gluon-plasma medium at collision energies of $\sqrt{s_{_{NN}}}=200$ GeV., Comment: 369 authors from 72 institutions, 12 pages, 7 figures, 5 tables. v1 is version submitted to Physical Review C. HEPdata tables for the points plotted in figures for this and previous PHENIX publications are (or will be) publicly available at http://www.phenix.bnl.gov/papers.html

We present the first forward-rapidity measurements of elliptic anisotropy of open-heavy-flavor muons at the BNL Relativistic Heavy Ion Collider. The measurements are based on data samples of Au$+$Au collisions at $\sqrt{s_{_{NN}}}=200$ GeV collected by the PHENIX experiment in 2014 and 2016 with integrated luminosity of 14.5~nb$^{-1}$. The measurements are performed in the pseudorapidity range $1.2<|\eta|<2$ and cover transverse momenta $1

Understanding emotions is fundamental to human interaction and experience. Humans easily infer emotions from situations or facial expressions, situations from emotions, and do a variety of other affective cognition. How adept is modern AI at these inferences? We introduce an evaluation framework for testing affective cognition in foundation models. Starting from psychological theory, we generate 1,280 diverse scenarios exploring relationships between appraisals, emotions, expressions, and outcomes. We evaluate the abilities of foundation models (GPT-4, Claude-3, Gemini-1.5-Pro) and humans (N = 567) across carefully selected conditions. Our results show foundation models tend to agree with human intuitions, matching or exceeding interparticipant agreement. In some conditions, models are ``superhuman'' -- they better predict modal human judgements than the average human. All models benefit from chain-of-thought reasoning. This suggests foundation models have acquired a human-like understanding of emotions and their influence on beliefs and behavior.

In this paper, we investigate the impact of hallucinations and cognitive forcing functions in human-AI collaborative text generation tasks, focusing on the use of Large Language Models (LLMs) to assist in generating high-quality conversational data. LLMs require data for fine-tuning, a crucial step in enhancing their performance. In the context of conversational customer support, the data takes the form of a conversation between a human customer and an agent and can be generated with an AI assistant. In our inquiry, involving 11 users who each completed 8 tasks, resulting in a total of 88 tasks, we found that the presence of hallucinations negatively impacts the quality of data. We also find that, although the cognitive forcing function does not always mitigate the detrimental effects of hallucinations on data quality, the presence of cognitive forcing functions and hallucinations together impacts data quality and influences how users leverage the AI responses presented to them. Our analysis of user behavior reveals distinct patterns of reliance on AI-generated responses, highlighting the importance of managing hallucinations in AI-generated content within conversational AI contexts.

The adaptive immune response, largely mediated by B-cell receptors (BCRs), plays a crucial role for effective pathogen neutralization due to its diversity and antigen specificity. Designing BCRs de novo, or from scratch, has been challenging because of their complex structure and diverse binding requirements. Protein language models (PLMs) have shown remarkable performance in contextualizing and performing various downstream tasks without relying on structural information. However, these models often lack a comprehensive understanding of the entire protein space, which limits their application in antibody design. In this study, we introduce Antibody Generative Pretrained Transformer (AbGPT), a model fine-tuned from a foundational PLM to enable a more informed design of BCR sequences. Using a custom generation and filtering pipeline, AbGPT successfully generated a high-quality library of 15,000 BCR sequences, demonstrating a strong understanding of the intrinsic variability and conserved regions within the antibody repertoire.

The $J/\psi$ and $\psi(2S)$ charmonium states, composed of $c\bar{c}$ quark pairs and known since the 1970s, are widely believed to serve as ideal probes to test quantum chromodynamics in high-energy hadronic interactions. However, there is not yet a complete understanding of the charmonium-production mechanism. Recent measurements of $J/\psi$ production as a function of event charged-particle multiplicity at the collision energies of both the Large Hadron Collider (LHC) and the Relativistic Heavy Ion Collider (RHIC) show enhanced $J/\psi$ production yields with increasing multiplicity. One potential explanation for this type of dependence is multiparton interactions (MPI). We carry out the first measurements of self-normalized $J/\psi$ yields and the $\psi(2S)$ to $J/\psi$ ratio at both forward and backward rapidities as a function of self-normalized charged-particle multiplicity in $p$$+$$p$ collisions at $\sqrt{s}=200$ GeV. In addition, detailed {\sc pythia} studies tuned to RHIC energies were performed to investigate the MPI impacts. We find that the PHENIX data at RHIC are consistent with recent LHC measurements and can only be described by {\sc pythia} calculations that include MPI effects. The forward and backward $\psi(2S)$ to $J/\psi$ ratio, which serves as a unique and powerful approach to study final-state effects on charmonium production, is found to be less dependent on the charged-particle multiplicity., Comment: 301 authors from 69 institutions, 8 pages, 3 figures. v1 is version submitted to Physical Review D Letters. HEPdata tables for the points plotted in figures for this and previous PHENIX publications are (or will be) publicly available at http://www.phenix.bnl.gov/papers.html

Fix a set-theoretic universe $V$. We look at small extensions of $V$ as generalised degrees of computability over $V$. We also formalise and investigate the complexity of certain methods one can use to define, in $V$, subclasses of degrees over $V$. Finally, we give a nice characterisation of the complexity of forcing within this framework., Comment: 36 pages, 1 figure. arXiv admin note: text overlap with arXiv:2402.01213

Building high-quality datasets for specialized tasks is a time-consuming and resource-intensive process that often requires specialized domain knowledge. We propose Corpus Retrieval and Augmentation for Fine-Tuning (CRAFT), a method for generating synthetic datasets, given a small number of user-written few-shots that demonstrate the task to be performed. Given the few-shot examples, we use large-scale public web-crawled corpora and similarity-based document retrieval to find other relevant human-written documents. Lastly, instruction-tuned large language models (LLMs) augment the retrieved documents into custom-formatted task samples, which then can be used for fine-tuning. We demonstrate that CRAFT can efficiently generate large-scale task-specific training datasets for four diverse tasks: biology question-answering (QA), medicine QA and commonsense QA as well as summarization. Our experiments show that CRAFT-based models outperform or achieve comparable performance to general LLMs for QA tasks, while CRAFT-based summarization models outperform models trained on human-curated data by 46 preference points.

Intracorporeal needle-based therapeutic ultrasound (NBTU) is a minimally invasive option for intervening in malignant brain tumors, commonly used in thermal ablation procedures. This technique is suitable for both primary and metastatic cancers, utilizing a high-frequency alternating electric field (up to 10 MHz) to excite a piezoelectric transducer. The resulting rapid deformation of the transducer produces an acoustic wave that propagates through tissue, leading to localized high-temperature heating at the target tumor site and inducing rapid cell death. To optimize the design of NBTU transducers for thermal dose delivery during treatment, numerical modeling of the acoustic pressure field generated by the deforming piezoelectric transducer is frequently employed. The bioheat transfer process generated by the input pressure field is used to track the thermal propagation of the applicator over time. Magnetic resonance thermal imaging (MRTI) can be used to experimentally validate these models. Validation results using MRTI demonstrated the feasibility of this model, showing a consistent thermal propagation pattern. However, a thermal damage isodose map is more advantageous for evaluating therapeutic efficacy. To achieve a more accurate simulation based on the actual brain tissue environment, a new finite element method (FEM) simulation with enhanced damage evaluation capabilities was conducted. The results showed that the highest temperature and ablated volume differed between experimental and simulation results by 2.1884{\deg}C (3.71%) and 0.0631 cm$^3$ (5.74%), respectively. The lowest Pearson correlation coefficient (PCC) for peak temperature was 0.7117, and the lowest Dice coefficient for the ablated area was 0.7021, indicating a good agreement in accuracy between simulation and experiment., Comment: 9 pages, 9 figures, 7 tables

Resolved rotation curves (RCs) are our best probe of the dark matter distribution around individual galaxies. However their acquisition is resource-intensive, rendering them impractical for large-scale surveys and studies at higher redshift. Spatially integrated HI flux profiles on the other hand are observationally abundant and also probe dynamics across the whole HI disc. Despite this, they are typically only studied using the highly compressed linewidth summary statistic, discarding much of the available information. Here we construct a Bayesian model to infer halo properties from the full shape of the spatially integrated 21-cm line profile of a galaxy, utilising all the available information. We validate our model by assessing the consistency of halo parameters obtained from the flux profile with those obtained from RC fits for galaxies where both are available, finding good agreement provided the profile is not strongly asymmetric. We study the relative constraining power (quantified using the Kullback--Leibler divergence of the posterior from the prior), finding the flux profile inference recovers posteriors on generalised Navarro--Frenk--White halo parameters on average three times tighter than those from the linewidth, and in some cases as tight as those from resolved RCs. Finally we introduce and validate a probabilistic empirical model for the spatial distribution of HI, enabling our model to be applied to datasets for which no spatially resolved HI information is available. As the next-generation of HI observatories comes online, our framework will enable mass modelling in new regimes, with particular utility for constraining the dark matter content of galaxies across cosmic time., Comment: 13 pages, 6 figures, to be submitted to MNRAS

Quantum error correction provides a path to reach practical quantum computing by combining multiple physical qubits into a logical qubit, where the logical error rate is suppressed exponentially as more qubits are added. However, this exponential suppression only occurs if the physical error rate is below a critical threshold. In this work, we present two surface code memories operating below this threshold: a distance-7 code and a distance-5 code integrated with a real-time decoder. The logical error rate of our larger quantum memory is suppressed by a factor of $\Lambda$ = 2.14 $\pm$ 0.02 when increasing the code distance by two, culminating in a 101-qubit distance-7 code with 0.143% $\pm$ 0.003% error per cycle of error correction. This logical memory is also beyond break-even, exceeding its best physical qubit's lifetime by a factor of 2.4 $\pm$ 0.3. We maintain below-threshold performance when decoding in real time, achieving an average decoder latency of 63 $\mu$s at distance-5 up to a million cycles, with a cycle time of 1.1 $\mu$s. To probe the limits of our error-correction performance, we run repetition codes up to distance-29 and find that logical performance is limited by rare correlated error events occurring approximately once every hour, or 3 $\times$ 10$^9$ cycles. Our results present device performance that, if scaled, could realize the operational requirements of large scale fault-tolerant quantum algorithms., Comment: 10 pages, 4 figures, Supplementary Information

The jet cross-section and jet-substructure observables in $p$$+$$p$ collisions at $\sqrt{s}=200$ GeV were measured by the PHENIX Collaboration at the Relativistic Heavy Ion Collider (RHIC). Jets are reconstructed from charged-particle tracks and electromagnetic-calorimeter clusters using the anti-$k_{t}$ algorithm with a jet radius $R=0.3$ for jets with transverse momentum within $8.0

Physics, as a fundamental science, aims to understand the laws of Nature and describe them in mathematical equations. While the physical reality manifests itself in a wide range of phenomena with varying levels of complexity, the equations that describe them display certain statistical regularities and patterns, which we begin to explore here. By drawing inspiration from linguistics, where Zipf's law states that the frequency of any word in a large corpus of text is roughly inversely proportional to its rank in the frequency table, we investigate whether similar patterns for the distribution of operators emerge in the equations of physics. We analyse three corpora of formulae and find, using sophisticated implicit-likelihood methods, that the frequency of operators as a function of their rank in the frequency table is best described by an exponential law with a stable exponent, in contrast with Zipf's inverse power-law. Understanding the underlying reasons behind this statistical pattern may shed light on Nature's modus operandi or reveal recurrent patterns in physicists' attempts to formalise the laws of Nature. It may also provide crucial input for symbolic regression, potentially augmenting language models to generate symbolic models for physical phenomena. By pioneering the study of statistical regularities in the equations of physics, our results open the door for a meta-law of Nature, a (probabilistic) law that all physical laws obey., Comment: 9 pages, 5 figures

Resolved rotation curves (RCs) are the gold-standard measurements for inferring dark matter distributions in $\Lambda$CDM and testing alternative theories of dynamics in galaxies. However they are expensive to obtain, making them prohibitive for large galaxy samples and at higher redshift. Spatially integrated HI flux profiles are more accessible and present the information in a different form, but -- except in a highly compressed form, as linewidths -- have not so far been compared in detail with RCs or employed for dynamical inferences. Here we study the consistency of RCs and HI surface density profiles from SPARC with spatially integrated HI flux profiles from ALFALFA, by combining the resolved properties in a forward model for the flux profile. We define a new metric for asymmetry in the flux profiles, enabling us to cleanly identify those unsuitable for our axisymmetric method. Among all SPARC galaxies the agreement between RCs and flux profiles is satisfactory within the limitations of the data -- with most galaxies having an uncertainty normalised mean squared error (MSE) below 10 -- while no galaxy deemed symmetric has a MSE above 1.2. Most cases of good agreement prefer a HI gas dispersion $\sigma_{\mathrm{HI}}$ of ~13 km/s, consistent with resolved studies of gas dispersion from the literature. These results open the door for spatially integrated HI flux profiles to be used as proxies for spatially resolved dynamics, including a robust appraisal of the degree of asymmetry., Comment: Submitted to MNRAS

Fifth forces are ubiquitous in modified gravity theories, and must be screened to evade stringent local tests. This can introduce unusual behaviour in galaxy phenomenology by affecting galaxies' components differently. Here we use the SDSS-IV-MaNGA dataset to search for a systematic excess of gas circular velocity over stellar circular velocity, expected in thin-shell-screened theories in the partially screened regime. Accounting for asymmetric drift and calibrating our model on screened subsamples, we find no significant evidence for a screened fifth force. We bound the fifth-force strength to $\Delta G/G_\text{N} < 0.1$ for all astrophysical ranges, strengthening to $\sim$0.01 at Compton wavelength of 3 Mpc for the Hu-Sawicki model, for instance. This implies a stringent constraint on scalar-tensor theories, for example $f_{\mathcal{R}0} \lesssim 10^{-8}$ in Hu-Sawicki $f(\mathcal{R})$ gravity., Comment: Version accepted for publication in MNRAS. 13 pages, 18 figures, 2 tables, references added

The PHENIX experiment measured the centrality dependence of two-pion Bose-Einstein correlation functions in $\sqrt{s_{_{NN}}}=200$~GeV Au$+$Au collisions at the Relativistic Heavy Ion Collider at Brookhaven National Laboratory. The data are well represented by L\'evy-stable source distributions. The extracted source parameters are the correlation-strength parameter $\lambda$, the L\'evy index of stability $\alpha$, and the L\'evy-scale parameter $R$ as a function of transverse mass $m_T$ and centrality. The $\lambda(m_T)$ parameter is constant at larger values of $m_T$, but decreases as $m_T$ decreases. The L\'evy scale parameter $R(m_T)$ decreases with $m_T$ and exhibits proportionality to the length scale of the nuclear overlap region. The L\'evy exponent $\alpha(m_T)$ is independent of $m_T$ within uncertainties in each investigated centrality bin, but shows a clear centrality dependence. At all centralities, the L\'evy exponent $\alpha$ is significantly different from that of Gaussian ($\alpha=2$) or Cauchy ($\alpha=1$) source distributions. Comparisons to the predictions of Monte-Carlo simulations of resonance-decay chains show that in all but the most peripheral centrality class (50%-60%), the obtained results are inconsistent with the measurements, unless a significant reduction of the in-medium mass of the $\eta'$ meson is included. In each centrality class, the best value of the in-medium $\eta'$ mass is compared to the mass of the $\eta$ meson, as well as to several theoretical predictions that consider restoration of $U_A(1)$ symmetry in hot hadronic matter., Comment: 401 authors from 75 institutions, 20 pages, 15 figures, 2 tables. v1 is version submitted to Physical Review C. HEPdata tables for the points plotted in figures for this and previous PHENIX publications are (or will be) publicly available at http://www.phenix.bnl.gov/papers.html

Traditional reference-based metrics, such as BLEU and ROUGE, are less effective for assessing outputs from Large Language Models (LLMs) that produce highly creative or superior-quality text, or in situations where reference outputs are unavailable. While human evaluation remains an option, it is costly and difficult to scale. Recent work using LLMs as evaluators (LLM-as-a-judge) is promising, but trust and reliability remain a significant concern. Integrating human input is crucial to ensure criteria used to evaluate are aligned with the human's intent, and evaluations are robust and consistent. This paper presents a user study of a design exploration called EvaluLLM, that enables users to leverage LLMs as customizable judges, promoting human involvement to balance trust and cost-saving potential with caution. Through interviews with eight domain experts, we identified the need for assistance in developing effective evaluation criteria aligning the LLM-as-a-judge with practitioners' preferences and expectations. We offer findings and design recommendations to optimize human-assisted LLM-as-judge systems., Comment: 14 pages, 6 figures, Accepted for publication in ACL 2024 Workshop HuCLLM

We introduce the concept of deceptive diffusion -- training a generative AI model to produce adversarial images. Whereas a traditional adversarial attack algorithm aims to perturb an existing image to induce a misclassificaton, the deceptive diffusion model can create an arbitrary number of new, misclassified images that are not directly associated with training or test images. Deceptive diffusion offers the possibility of strengthening defence algorithms by providing adversarial training data at scale, including types of misclassification that are otherwise difficult to find. In our experiments, we also investigate the effect of training on a partially attacked data set. This highlights a new type of vulnerability for generative diffusion models: if an attacker is able to stealthily poison a portion of the training data, then the resulting diffusion model will generate a similar proportion of misleading outputs.

There is an increasing trend towards evaluating NLP models with LLM-generated judgments instead of human judgments. In the absence of a comparison against human data, this raises concerns about the validity of these evaluations; in case they are conducted with proprietary models, this also raises concerns over reproducibility. We provide JUDGE-BENCH, a collection of 20 NLP datasets with human annotations, and comprehensively evaluate 11 current LLMs, covering both open-weight and proprietary models, for their ability to replicate the annotations. Our evaluations show that each LLM exhibits a large variance across datasets in its correlation to human judgments. We conclude that LLMs are not yet ready to systematically replace human judges in NLP.

We reveal the theoretical foundations of techniques for editing large language models, and present new methods which can do so without requiring retraining. Our theoretical insights show that a single metric (a measure of the intrinsic dimension of the model's features) can be used to assess a model's editability and reveals its previously unrecognised susceptibility to malicious stealth attacks. This metric is fundamental to predicting the success of a variety of editing approaches, and reveals new bridges between disparate families of editing methods. We collectively refer to these as stealth editing methods, because they directly update a model's weights to specify its response to specific known hallucinating prompts without affecting other model behaviour. By carefully applying our theoretical insights, we are able to introduce a new jet-pack network block which is optimised for highly selective model editing, uses only standard network operations, and can be inserted into existing networks. We also reveal the vulnerability of language models to stealth attacks: a small change to a model's weights which fixes its response to a single attacker-chosen prompt. Stealth attacks are computationally simple, do not require access to or knowledge of the model's training data, and therefore represent a potent yet previously unrecognised threat to redistributed foundation models. Extensive experimental results illustrate and support our methods and their theoretical underpinnings. Demos and source code are available at https://github.com/qinghua-zhou/stealth-edits., Comment: 28 pages, 14 figures. Open source implementation: https://github.com/qinghua-zhou/stealth-edits

Food is a rich and varied dimension of cultural heritage, crucial to both individuals and social groups. To bridge the gap in the literature on the often-overlooked regional diversity in this domain, we introduce FoodieQA, a manually curated, fine-grained image-text dataset capturing the intricate features of food cultures across various regions in China. We evaluate vision-language Models (VLMs) and large language models (LLMs) on newly collected, unseen food images and corresponding questions. FoodieQA comprises three multiple-choice question-answering tasks where models need to answer questions based on multiple images, a single image, and text-only descriptions, respectively. While LLMs excel at text-based question answering, surpassing human accuracy, the open-sourced VLMs still fall short by 41% on multi-image and 21% on single-image VQA tasks, although closed-weights models perform closer to human levels (within 10%). Our findings highlight that understanding food and its cultural implications remains a challenging and under-explored direction.

Large Language Models have taken the cognitive science world by storm. It is perhaps timely now to take stock of the various research paradigms that have been used to make scientific inferences about ``cognition" in these models or about human cognition. We review several emerging research paradigms -- GPT-ology, LLMs-as-computational-models, and ``silicon sampling" -- and review recent papers that have used LLMs under these paradigms. In doing so, we discuss their claims as well as challenges to scientific inference under these various paradigms. We highlight several outstanding issues about LLMs that have to be addressed to push our science forward: closed-source vs open-sourced models; (the lack of visibility of) training data; and reproducibility in LLM research, including forming conventions on new task ``hyperparameters" like instructions and prompts., Comment: CogSci 2024; 6 pages + 2 page of references

High-momentum two-particle correlations are a useful tool for studying jet-quenching effects in the quark-gluon plasma. Angular correlations between neutral-pion triggers and charged hadrons with transverse momenta in the range 4--12~GeV/$c$ and 0.5--7~GeV/$c$, respectively, have been measured by the PHENIX experiment in 2014 for Au$+$Au collisions at $\sqrt{s_{_{NN}}}=200$~GeV. Suppression is observed in the yield of high-momentum jet fragments opposite the trigger particle, which indicates jet suppression stemming from in-medium partonic energy loss, while enhancement is observed for low-momentum particles. The ratio and differences between the yield in Au$+$Au collisions and $p$$+$$p$ collisions, $I_{AA}$ and $\Delta_{AA}$, as a function of the trigger-hadron azimuthal separation, $\Delta\phi$, are measured for the first time at the Relativistic Heavy Ion Collider. These results better quantify how the yield of low-$p_T$ associated hadrons is enhanced at wide angle, which is crucial for studying energy loss as well as medium-response effects., Comment: 535 authors from 84 institutions, 12 pages, 8 figures. v2 is version accepted for publication in Physical Review C. HEPdata tables for the points plotted in figures for this and previous PHENIX publications are (or will be) publicly available at http://www.phenix.bnl.gov/papers.html

Recent advances in retrieval-augmented models for image captioning highlight the benefit of retrieving related captions for efficient, lightweight models with strong domain-transfer capabilities. While these models demonstrate the success of retrieval augmentation, retrieval models are still far from perfect in practice: the retrieved information can sometimes mislead the model, resulting in incorrect generation and worse performance. In this paper, we analyze the robustness of a retrieval-augmented captioning model SmallCap. Our analysis shows that the model is sensitive to tokens that appear in the majority of the retrieved captions, and the input attribution shows that those tokens are likely copied into the generated output. Given these findings, we propose to train the model by sampling retrieved captions from more diverse sets. This decreases the chance that the model learns to copy majority tokens, and improves both in-domain and cross-domain performance., Comment: 9 pages, long paper at ACL 2024