Search

Multimodal Large Language Models (MLLMs) can enhance trustworthiness by aligning with human preferences. As human preference labeling is laborious, recent works employ evaluation models for assessing MLLMs' responses, using the model-based assessments to automate preference dataset construction. This approach, however, faces challenges with MLLMs' lengthy and compositional responses, which often require diverse reasoning skills that a single evaluation model may not fully possess. Additionally, most existing methods rely on closed-source models as evaluators. To address limitations, we propose DecompGen, a decomposable framework that uses an ensemble of open-sourced expert models. DecompGen breaks down each response into atomic verification tasks, assigning each task to an appropriate expert model to generate fine-grained assessments. The DecompGen feedback is used to automatically construct our preference dataset, DGPref. MLLMs aligned with DGPref via preference learning show improvements in trustworthiness, demonstrating the effectiveness of DecompGen.

In this work, we elaborate further on a 4D cosmological Running-Vacuum-type Model (RVM) of inflation that characterises string-inspired Chern-Simons (CS) gravity. It has been shown that inflation in such models is caused by a condensation of the gravitational CS (gCS) terms induced by primordial gravitational waves (GW), which leads to a linear-axion potential, thus breaking the shift symmetry, and lifting its periodicity (monodromy). We demonstrate here that this inflationary phase may be metastable, due to the existence of non-trivial imaginary parts of the gCS condensate. These imaginary parts are quantum effects, proportional to appropriate commutators of GW perturbations. As we stress, their existence is quantum-ordering-scheme dependent. We argue here in favor of a physical importance of such imaginary parts, which we compute to second order in the GW (tensor) perturbations in the framework of a specific gauge-fixed effective Lagrangian, within a (mean field) weak-quantum-gravity path integral approach. We pay specific attention to the various space-time boundary terms. We thus provide an estimate of the life time of inflation. On matching our results with the relevant inflationary phenomenology, we fix the quantum-ordering ambiguities, and obtain an order-of-magnitude constraint on the ratio of the string energy scale $M_s$ in this model to the (four-spacetime-dimensional) reduced Planck mass $M_{\rm Pl}$, specifically, $M_s/M_{\rm Pl} = \mathcal{O}(10^{-1})$. This is consistent with the corresponding estimate obtained in previous analyses by the authors in this framework, based on a dynamical-system approach to linear-axion RVM inflation. Finally, we examine the role of periodic modulations in the axion potential induced by non-perturbative stringy effects on the slow-roll inflationary parameters, and find compatibility with the cosmological data., Comment: 38 pages, revtex4, no figures

Reranking a list of candidates from a machine translation system with an external scoring model and returning the highest-scoring candidate remains a simple and effective method for improving the overall output quality. Translation scoring models continue to grow in size, with the best models being comparable to generation models. Thus, reranking can add substantial computational cost to the translation pipeline. In this work, we pose reranking as a Bayesian optimization (BayesOpt) problem. By strategically selecting candidates to score based on a balance of exploration and exploitation, we show that it is possible to find top-scoring candidates when scoring only a fraction of the candidate list. For instance, our method achieves the same CometKiwi score using only 70 scoring evaluations compared a baseline system using 180. We present a multi-fidelity setting for BayesOpt, where the candidates are first scored with a cheaper but noisier proxy scoring model, which further improves the cost-performance tradeoff when using smaller but well-trained distilled proxy scorers., Comment: v1: Preprint version

Current automated fact-checking (AFC) approaches commonly evaluate evidence either implicitly via the predicted verdicts or by comparing retrieved evidence with a predefined closed knowledge source, such as Wikipedia. However, these methods suffer from limitations, resulting from their reliance on evaluation metrics developed for different purposes and constraints imposed by closed knowledge sources. Recent advances in natural language generation (NLG) evaluation offer new possibilities for evidence assessment. In this work, we introduce Ev2R, an evaluation framework for AFC that comprises three types of approaches for evidence evaluation: reference-based, proxy-reference, and reference-less. We evaluate their effectiveness through agreement with human ratings and adversarial tests, and demonstrate that prompt-based scorers, particularly those leveraging LLMs and reference evidence, outperform traditional evaluation approaches., Comment: 10 pages

Fact verification on tabular evidence incentivises the use of symbolic reasoning models where a logical form is constructed (e.g. a LISP-style program), providing greater verifiability than fully neural approaches. However, these systems typically rely on well-formed tables, restricting their use in many scenarios. An emerging symbolic reasoning paradigm for textual evidence focuses on natural logic inference, which constructs proofs by modelling set-theoretic relations between a claim and its evidence in natural language. This approach provides flexibility and transparency but is less compatible with tabular evidence since the relations do not extend to arithmetic functions. We propose a set-theoretic interpretation of numerals and arithmetic functions in the context of natural logic, enabling the integration of arithmetic expressions in deterministic proofs. We leverage large language models to generate arithmetic expressions by generating questions about salient parts of a claim which are answered by executing appropriate functions on tables. In a few-shot setting on FEVEROUS, we achieve an accuracy of 71.4, outperforming both fully neural and symbolic reasoning models by 3.4 points. When evaluated on TabFact without any further training, our method remains competitive with an accuracy lead of 0.5 points., Comment: Accepted to TACL. This is a slightly extended version

The Automated Verification of Textual Claims (AVeriTeC) shared task asks participants to retrieve evidence and predict veracity for real-world claims checked by fact-checkers. Evidence can be found either via a search engine, or via a knowledge store provided by the organisers. Submissions are evaluated using AVeriTeC score, which considers a claim to be accurately verified if and only if both the verdict is correct and retrieved evidence is considered to meet a certain quality threshold. The shared task received 21 submissions, 18 of which surpassed our baseline. The winning team was TUDA_MAI with an AVeriTeC score of 63%. In this paper we describe the shared task, present the full results, and highlight key takeaways from the shared task.

The conformity effect describes the tendency of individuals to align their responses with the majority. Studying this bias in large language models (LLMs) is crucial, as LLMs are increasingly used in various information-seeking and decision-making tasks as conversation partners to improve productivity. Thus, conformity to incorrect responses can compromise their effectiveness. In this paper, we adapt psychological experiments to examine the extent of conformity in state-of-the-art LLMs. Our findings reveal that all models tested exhibit varying levels of conformity toward the majority, regardless of their initial choice or correctness, across different knowledge domains. Notably, we are the first to show that LLMs are more likely to conform when they are more uncertain in their own prediction. We further explore factors that influence conformity, such as training paradigms and input characteristics, finding that instruction-tuned models are less susceptible to conformity, while increasing the naturalness of majority tones amplifies conformity. Finally, we propose two interventions--Devil's Advocate and Question Distillation--to mitigate conformity, providing insights into building more robust language models., Comment: 16 pages (8 pages main body), 14 figures

Active Learning aims to minimize annotation effort by selecting the most useful instances from a pool of unlabeled data. However, typical active learning methods overlook the presence of distinct example groups within a class, whose prevalence may vary, e.g., in occupation classification datasets certain demographics are disproportionately represented in specific classes. This oversight causes models to rely on shortcuts for predictions, i.e., spurious correlations between input attributes and labels occurring in well-represented groups. To address this issue, we propose Active Learning Via INterpolation (ALVIN), which conducts intra-class interpolations between examples from under-represented and well-represented groups to create anchors, i.e., artificial points situated between the example groups in the representation space. By selecting instances close to the anchors for annotation, ALVIN identifies informative examples exposing the model to regions of the representation space that counteract the influence of shortcuts. Crucially, since the model considers these examples to be of high certainty, they are likely to be ignored by typical active learning methods. Experimental results on six datasets encompassing sentiment analysis, natural language inference, and paraphrase detection demonstrate that ALVIN outperforms state-of-the-art active learning methods in both in-distribution and out-of-distribution generalization., Comment: Accepted to EMNLP 2024 (Main)

The recent development of fact verification systems with natural logic has enhanced their explainability by aligning claims with evidence through set-theoretic operators, providing faithful justifications. Despite these advancements, such systems often rely on a large amount of training data annotated with natural logic. To address this issue, we propose a zero-shot method that utilizes the generalization capabilities of instruction-tuned large language models. To comprehensively assess the zero-shot capabilities of our method and other fact verification systems, we evaluate all models on both artificial and real-world claims, including multilingual datasets. We also compare our method against other fact verification systems in two setups. First, in the zero-shot generalization setup, we demonstrate that our approach outperforms other systems that were not specifically trained on natural logic data, achieving an average accuracy improvement of 8.96 points over the best-performing baseline. Second, in the zero-shot transfer setup, we show that current systems trained on natural logic data do not generalize well to other domains, and our method outperforms these systems across all datasets with real-world claims., Comment: Accepted to EMNLP 2024

In this paper, we consider the problem of predicting unknown targets from data. We propose Online Residual Learning (ORL), a method that combines online adaptation with offline-trained predictions. At a lower level, we employ multiple offline predictions generated before or at the beginning of the prediction horizon. We augment every offline prediction by learning their respective residual error concerning the true target state online, using the recursive least squares algorithm. At a higher level, we treat the augmented lower-level predictors as experts, adopting the Prediction with Expert Advice framework. We utilize an adaptive softmax weighting scheme to form an aggregate prediction and provide guarantees for ORL in terms of regret. We employ ORL to boost performance in the setting of online pedestrian trajectory prediction. Based on data from the Stanford Drone Dataset, we show that ORL can demonstrate best-of-both-worlds performance., Comment: Accepted to CDC 2024, v2: fixed certain typos

Quantum sensors promise revolutionary advances in medical imaging, energy production, mass detection, geodesy, foundational physics research, and a host of other fields. In many sensors, the signal takes the form of a changing qubit frequency, which is detected with an interference measurement. Unfortunately, environmental noise decoheres the qubit state, reducing signal-to-noise ratio (SNR). Here we introduce a protocol for enhancing the sensitivity of a measurement of a qubit's frequency in the presence of decoherence. We use a continuous drive to stabilize one component of the qubit's Bloch vector, enhancing the effect of a small static frequency shift. We demonstrate our protocol on a superconducting qubit, enhancing SNR per measurement shot by 1.65$\times$ and SNR per qubit evolution time by 1.09$\times$ compared to standard Ramsey interferometry. We explore the protocol theoretically and numerically, finding maximum enhancements of 1.96$\times$ and 1.18$\times$, respectively. We also show that the protocol is robust to parameter miscalibrations. Our protocol provides an unconditional enhancement in signal-to-noise ratio compared to standard Ramsey interferometry. It requires no feedback and no extra control or measurement resources, and can be immediately applied in a wide variety of quantum computing and quantum sensor technologies to enhance their sensitivities.

We leverage generative large language models for language learning applications, focusing on estimating the difficulty of foreign language texts and simplifying them to lower difficulty levels. We frame both tasks as prediction problems and develop a difficulty classification model using labeled examples, transfer learning, and large language models, demonstrating superior accuracy compared to previous approaches. For simplification, we evaluate the trade-off between simplification quality and meaning preservation, comparing zero-shot and fine-tuned performances of large language models. We show that meaningful text simplifications can be obtained with limited fine-tuning. Our experiments are conducted on French texts, but our methods are language-agnostic and directly applicable to other foreign languages., Comment: 10 pages, 4 figures

Non-invasive flow measurement techniques, such as particle tracking velocimetry, resolve 3D velocity fields by pairing tracer particle positions in successive time steps. These trajectories are crucial for evaluating physical quantities like vorticity, shear stress, pressure, and coherent structures. Traditional approaches deterministically reconstruct particle positions and extract particle tracks using tracking algorithms. However, reliable track estimation is challenging due to measurement noise caused by high particle density, particle image overlap, and falsely reconstructed 3D particle positions. To overcome this challenge, probabilistic approaches quantify the epistemic uncertainty in particle positions, typically using a Gaussian probability distribution. However, the standard deterministic tracking algorithms relying on nearest-neighbor search do not directly extend to the probabilistic setting. Moreover, such algorithms do not necessarily find globally consistent solutions robust to reconstruction errors. This paper aims to develop a globally consistent nearest-neighborhood algorithm that robustly extracts stochastic particle tracks from the reconstructed Gaussian particle distributions in all frames. Our tracking algorithm relies on the unbalanced optimal transport theory in the metric space of Gaussian measures. Specifically, we optimize a binary transport plan for efficiently moving the Gaussian distributions of reconstructed particle positions between time frames. We achieve this by computing the partial Wasserstein distance in the metric space of Gaussian measures. Our tracking algorithm is robust to position reconstruction errors since it automatically detects the number of particles that should be matched through hyperparameter optimization. Finally, we validate our method using an in vitro flow experiment using a 3D-printed cerebral aneurysm.

Research on dialogue constructiveness assessment focuses on (i) analysing conversational factors that influence individuals to take specific actions, win debates, change their perspectives or broaden their open-mindedness and (ii) predicting constructiveness outcomes following dialogues for such use cases. These objectives can be achieved by training either interpretable feature-based models (which often involve costly human annotations) or neural models such as pre-trained language models (which have empirically shown higher task accuracy but lack interpretability). In this paper we propose an LLM feature-based framework for dialogue constructiveness assessment that combines the strengths of feature-based and neural approaches, while mitigating their downsides. The framework first defines a set of dataset-independent and interpretable linguistic features, which can be extracted by both prompting an LLM and simple heuristics. Such features are then used to train LLM feature-based models. We apply this framework to three datasets of dialogue constructiveness and find that our LLM feature-based models outperform or performs at least as well as standard feature-based models and neural models. We also find that the LLM feature-based model learns more robust prediction rules instead of relying on superficial shortcuts, which often trouble neural models., Comment: Paper accepted by EMNLP 2024

Our work revisits the design of mechanisms via the learning-augmented framework. In this model, the algorithm is enhanced with imperfect (machine-learned) information concerning the input, usually referred to as prediction. The goal is to design algorithms whose performance degrades gently as a function of the prediction error and, in particular, perform well if the prediction is accurate, but also provide a worst-case guarantee under any possible error. This framework has been successfully applied recently to various mechanism design settings, where in most cases the mechanism is provided with a prediction about the types of the players. We adopt a perspective in which the mechanism is provided with an output recommendation. We make no assumptions about the quality of the suggested outcome, and the goal is to use the recommendation to design mechanisms with low approximation guarantees whenever the recommended outcome is reasonable, but at the same time to provide worst-case guarantees whenever the recommendation significantly deviates from the optimal one. We propose a generic, universal measure, which we call quality of recommendation, to evaluate mechanisms across various information settings. We demonstrate how this new metric can provide refined analysis in existing results. This model introduces new challenges, as the mechanism receives limited information comparing to settings that use predictions about the types of the agents. We study, through this lens, several well-studied mechanism design paradigms, devising new mechanisms, but also providing refined analysis for existing ones, using as a metric the quality of recommendation. We complement our positive results, by exploring the limitations of known classes of strategyproof mechanisms that can be devised using output recommendation., Comment: This paper has been accepted by the 38th Conference on Neural Information Processing Systems (NeurIPS 2024)

We investigate specific intrinsic curvatures $\rho_k$ (where $1\leq k\leq n$) that interpolate between the minimum Ricci curvature $\rho_1$ and the normalized scalar curvature $\rho_n=\rho$ of $n$-dimensional Riemannian manifolds. For $n$-dimensional submanifolds in space forms, these curvatures satisfy an inequality involving the mean curvature $H$ and the normal scalar curvature $\rho^\perp$, which reduces to the well-known DDVV inequality when $k=n$. We derive topological obstructions for compact $n$-dimensional submanifolds based on universal lower bounds of the $L^{n/2}$-norms of certain functions involving $\rho_k,H$ and $\rho^\perp$. These obstructions are expressed in terms of the Betti numbers. Our main result applies for any $1\leq k \leq n-1$, but it generally fails for $k=n$, where the involved norm vanishes precisely for Wintgen ideal submanifolds. We demonstrate this by providing a method of constructing new compact 3-dimensional minimal Wintgen ideal submanifolds in even-dimensional spheres. Specifically, we prove that such submanifolds exist in $\mathbb{S}^6$ with arbitrarily large first Betti number.

We investigate isometric immersions $f\colon M^n\to\R^{n+2}$, $n\geq 3$, of Riemannian manifolds into Euclidean space with codimension two that admit isometric deformations that preserve the metric of the Gauss map. In precise terms, the preservation of the third fundamental form of the submanifold must be ensured throughout the deformation. For minimal isometric deformations of minimal submanifolds this is always the case. Our main result is of a local nature and states that if $f$ is neither minimal nor reducible, then it is a hypersurface of an isometrically deformable hypersurface $F\colon\tilde{M}^{n+1}\to\R^{n+2}$ such that the deformations of $F$ induce those of $f$. Moreover, for a particular class of such submanifolds, a complete local parametric description is provided.

Creating effective and reliable task-oriented dialog systems (ToDSs) is challenging, not only because of the complex structure of these systems, but also due to the scarcity of training data, especially when several modules need to be trained separately, each one with its own input/output training examples. Data augmentation (DA), whereby synthetic training examples are added to the training data, has been successful in other NLP systems, but has not been explored as extensively in ToDSs. We empirically evaluate the effectiveness of DA methods in an end-to-end ToDS setting, where a single system is trained to handle all processing stages, from user inputs to system outputs. We experiment with two ToDSs (UBAR, GALAXY) on two datasets (MultiWOZ, KVRET). We consider three types of DA methods (word-level, sentence-level, dialog-level), comparing eight DA methods that have shown promising results in ToDSs and other NLP systems. We show that all DA methods considered are beneficial, and we highlight the best ones, also providing advice to practitioners. We also introduce a more challenging few-shot cross-domain ToDS setting, reaching similar conclusions., Comment: There are 25 pages in total, 23 tables, 18 figures. Accepted in ACL 2024

Understanding memorisation in language models has practical and societal implications, e.g., studying models' training dynamics or preventing copyright infringements. Prior work defines memorisation as the causal effect of training with an instance on the model's ability to predict that instance. This definition relies on a counterfactual: the ability to observe what would have happened had the model not seen that instance. Existing methods struggle to provide computationally efficient and accurate estimates of this counterfactual. Further, they often estimate memorisation for a model architecture rather than for a specific model instance. This paper fills an important gap in the literature, proposing a new, principled, and efficient method to estimate memorisation based on the difference-in-differences design from econometrics. Using this method, we characterise a model's memorisation profile--its memorisation trends across training--by only observing its behaviour on a small set of instances throughout training. In experiments with the Pythia model suite, we find that memorisation (i) is stronger and more persistent in larger models, (ii) is determined by data order and learning rate, and (iii) has stable trends across model sizes, thus making memorisation in larger models predictable from smaller ones., Comment: Published at the ACL 2024 Conference (main)

Selecting which claims to check is a time-consuming task for human fact-checkers, especially from documents consisting of multiple sentences and containing multiple claims. However, existing claim extraction approaches focus more on identifying and extracting claims from individual sentences, e.g., identifying whether a sentence contains a claim or the exact boundaries of the claim within a sentence. In this paper, we propose a method for document-level claim extraction for fact-checking, which aims to extract check-worthy claims from documents and decontextualise them so that they can be understood out of context. Specifically, we first recast claim extraction as extractive summarization in order to identify central sentences from documents, then rewrite them to include necessary context from the originating document through sentence decontextualisation. Evaluation with both automatic metrics and a fact-checking professional shows that our method is able to extract check-worthy claims from documents more accurately than previous work, while also improving evidence retrieval., Comment: Accepted to ACL 2024

Neural networks are lately more and more often being used in the context of data-driven control, as an approximate model of the true system dynamics. Model Predictive Control (MPC) adopts this practise leading to neural MPC strategies. This raises a question of whether the trained neural network has converged and generalized in a way that the learned model encapsulates an accurate approximation of the true dynamic model of the system, thus making it a reliable choice for model-based control, especially for disturbed and uncertain systems. To tackle that, we propose Dropout MPC, a novel sampling-based ensemble neural MPC algorithm that employs the Monte-Carlo dropout technique on the learned system model. The closed loop is based on an ensemble of predictive controllers, that are used simultaneously at each time-step for trajectory optimization. Each member of the ensemble influences the control input, based on a weighted voting scheme, thus by employing different realizations of the learned system dynamics, neural control becomes more reliable by design. An additional strength of the method is that it offers by design a way to estimate future uncertainty, leading to cautious control. While the method aims in general at uncertain systems with complex dynamics, where models derived from first principles are hard to infer, to showcase the application we utilize data gathered in the laboratory from a real mobile manipulator and employ the proposed algorithm for the navigation of the robot in simulation.

Scientific writing is a challenging task, particularly for novice researchers who often rely on feedback from experienced peers. Recent work has primarily focused on improving surface form and style rather than manuscript content. In this paper, we propose a novel task: automated focused feedback generation for scientific writing assistance. We present SWIF$^{2}$T: a Scientific WrIting Focused Feedback Tool. It is designed to generate specific, actionable and coherent comments, which identify weaknesses in a scientific paper and/or propose revisions to it. Our approach consists of four components - planner, investigator, reviewer and controller - leveraging multiple Large Language Models (LLMs) to implement them. We compile a dataset of 300 peer reviews citing weaknesses in scientific papers and conduct human evaluation. The results demonstrate the superiority in specificity, reading comprehension, and overall helpfulness of SWIF$^{2}$T's feedback compared to other approaches. In our analysis, we also identified cases where automatically generated reviews were judged better than human ones, suggesting opportunities for integration of AI-generated feedback in scientific writing., Comment: Accepted to ACL 2024 (Findings)

We discuss the role of torsion in string theory on inducing pseudoscalar degrees of freedom (axions), which in turn couple to (gravitational) Chern-Simons (CS) anomalous terms. Such interactions can induce inflation, of running vacuum type, not requiring external inflaton fields, through condensation of the anomalous terms as a consequence of primordial chiral gravitational-wave (GW) tensor perturbations in a weak-quantum gravity setting. The presence of an UV cutoff for the GW quantum graviton modes opens up the system, leading to a dissipative behaviour realised via the presence of non trivial imaginary parts of the gravitational CS terms. The naive estimate of the life time of inflation based on such imaginary parts, which afflict the pertinent GW Hamiltonian, is quite consistent with the estimates of the duration of inflation based on an analysis of the condensate-induced linear-axion-potential by means of dynamical systems. Such quantum-gravity effects can also contribute positively to the alleviation of cosmological tensions if they survive today. In the talk we discuss the conditions under which such a result may be achieved. We also discuss the potential role of other axions in string theory, coming from compactification, in inducing enhanced densities of primordial black holes during RVM inflation, thereby contributing to significantly increased percentages of these black holes that can play the role of dark matter components. Moreover, under certain circumstances, that we shall discuss in some detail, it is also possible that the initially massless torsion-induced axions can acquire a non-trivial mass during the radiation era, thereby providing additional dark matter components in the Universe. With regards to this aspect, we also emphasise the role of massive right-handed neutrinos, provided that such excitations exist in the relevant spectra., Comment: 23 pages latex, several pdf figures incorporated, uses special macros (PoS). Invited plenary talk by NEM in Corfu Summer Institute 2023 Workshop on the Standard Model and beyond, August 27-September 7 2023

We identify as topological spheres those complete submanifolds lying with any codimension in hyperbolic space whose Ricci curvature satisfies a lower bound contingent solely upon the length of the mean curvature vector of the immersion.

The thermodynamics and transport properties of polymeric materials are essential for the design of reactors and for the development of polymer deconstruction processes. Existing property prediction tools such as correlations based on entropy scaling, kinetic gas theory, and free-volume model are inadequate for polymers. In this paper, we introduce a data-driven model for polyolefins based on data from molecular dynamics simulations that can accurately predict the transport properties of polyethylenes and their binary mixtures with higher n-alkanes across a range of temperatures, pressures, concentrations, and oligomer molecular weights.

Active learning for imbalanced classification tasks is challenging as the minority classes naturally occur rarely. Gathering a large pool of unlabelled data is thus essential to capture minority instances. Standard pool-based active learning is computationally expensive on large pools and often reaches low accuracy by overfitting the initial decision boundary, thus failing to explore the input space and find minority instances. To address these issues we propose AnchorAL. At each iteration, AnchorAL chooses class-specific instances from the labelled set, or anchors, and retrieves the most similar unlabelled instances from the pool. This resulting subpool is then used for active learning. Using a small, fixed-sized subpool AnchorAL allows scaling any active learning strategy to large pools. By dynamically selecting different anchors at each iteration it promotes class balance and prevents overfitting the initial decision boundary, thus promoting the discovery of new clusters of minority instances. In experiments across different classification tasks, active learning strategies, and model architectures AnchorAL is (i) faster, often reducing runtime from hours to minutes, (ii) trains more performant models, (iii) and returns more balanced datasets than competing methods., Comment: Published at the NAACL 2024 Conference (main)

This paper introduces PRobELM (Plausibility Ranking Evaluation for Language Models), a benchmark designed to assess language models' ability to discern more plausible from less plausible scenarios through their parametric knowledge. While benchmarks such as TruthfulQA emphasise factual accuracy or truthfulness, and others such as COPA explore plausible scenarios without explicitly incorporating world knowledge, PRobELM seeks to bridge this gap by evaluating models' capabilities to prioritise plausible scenarios that leverage world knowledge over less plausible alternatives. This design allows us to assess the potential of language models for downstream use cases such as literature-based discovery where the focus is on identifying information that is likely but not yet known. Our benchmark is constructed from a dataset curated from Wikidata edit histories, tailored to align the temporal bounds of the training data for the evaluated models. PRobELM facilitates the evaluation of language models across multiple prompting types, including statement, text completion, and question-answering. Experiments with 10 models of various sizes and architectures on the relationship between model scales, training recency, and plausibility performance, reveal that factual accuracy does not directly correlate with plausibility performance and that up-to-date training data enhances plausibility assessment across different model architectures.

Understanding the physics of electromagnetic pulse emission and nozzle damage is critical for the long-term operation of laser experiments with gas targets, particularly at facilities looking to produce stable sources of radiation at high repetition rate. We present a theoretical model of plasma formation and electrostatic charging when high-power lasers are focused inside gases. The model can be used to estimate the amplitude of gigahertz electromagnetic pulses (EMPs) produced by the laser and the extent of damage to the gas jet nozzle. Looking at a range of laser and target properties relevant to existing high-power laser systems, we find that EMP fields of tens to hundreds of kV/m can be generated several metres from the gas jet. Model predictions are compared with measurements of EMP, plasma formation and nozzle damage from two experiments on the VEGA-3 laser and one experiment on the Vulcan Petawatt laser., Comment: 18 pages (total), 12 figures, 3 tables

We present a novel scheme for rapid quantitative analysis of debris generated during experiments with solid targets following relativistic laser-plasma interaction at high-power laser facilities. Experimental data indicates that predictions by available modelling for non-mass-limited targets are reasonable, with debris on the order of hundreds ug-per-shot. We detect for the first time that several % of the debris is ejected directional following the target normal (rear- and interaction side); and confirm previous work that found the debris ejection in direction of the interaction side to be larger than on the side of the target rear.

In this work, we elaborate further on a cosmological model of inflation that characterises Chern-Simons (CS) gravity models inspired from string theory. Such models are known to belong to the class of the so-called String-Inspired Running Vacuum Cosmologies. In particular, by applying methods of dynamical systems, commonly used in scalar-field cosmology, we examine in detail, for the first time, the passage from a pre-inflationary era dominated by a stiff-axion-matter equation of state, characteristic of the model, to inflation of Running Vacuum Model (RVM) type. By a careful discussion of the formation of the condensate of the CS gravitational anomaly term, induced by populations of primordial gravitational waves at the end of the stiff-axion-matter era, we show that an effectively linear axion-monodromy potential arises. This eventually causes the transition from the matter to the RVM inflation. By taking into account terms that have previously been ignored in the relevant literature of weak-graviton quantisation, we show that the effect of such terms is to diminish the value of the condensate by half, remaining however in the same order of magnitude. This, in turn, implies that the qualitative conclusions of previous works on the subject remain valid. Moreover, on assuming the approximate cosmic-time independence of the gravitational-CS condensate, we also provide an estimate of the number of sources of the primordial gravitational waves, upon the requirement of respecting the transplanckian conjecture., Comment: 29 pages revtex, 4 pdf figures incorporated (references added ([70],[71],[72] and also brief discussion at the end of subsection IVA on UV completion of local effective field theories vs our stringy running vacuum model), no effects on conclusions

We explore different aspects of cognitive diversity and its effect on the success of group deliberation. To evaluate this, we use 500 dialogues from small, online groups discussing the Wason Card Selection task - the DeliData corpus. Leveraging the corpus, we perform quantitative analysis evaluating three different measures of cognitive diversity. First, we analyse the effect of group size as a proxy measure for diversity. Second, we evaluate the effect of the size of the initial idea pool. Finally, we look into the content of the discussion by analysing discussed solutions, discussion patterns, and how conversational probing can improve those characteristics. Despite the reputation of groups for compounding bias, we show that small groups can, through dialogue, overcome intuitive biases and improve individual decision-making. Across a large sample and different operationalisations, we consistently find that greater cognitive diversity is associated with more successful group deliberation. Code and data used for the analysis are available in the repository: https://github.com/gkaradzhov/cognitive-diversity-groups-cogsci24.

This paper investigates the potential benefits of language-specific fact-checking models, focusing on the case of Chinese. We first demonstrate the limitations of translation-based methods and multilingual large language models (e.g., GPT-4), highlighting the need for language-specific systems. We further propose a Chinese fact-checking system that can better retrieve evidence from a document by incorporating context information. To better analyze token-level biases in different systems, we construct an adversarial dataset based on the CHEF dataset, where each instance has large word overlap with the original one but holds the opposite veracity label. Experimental results on the CHEF dataset and our adversarial dataset show that our proposed method outperforms translation-based methods and multilingual LLMs and is more robust toward biases, while there is still large room for improvement, emphasizing the importance of language-specific fact-checking systems., Comment: EMNLP 2024 Main

We present an open-source database of superconducting quantum device designs that may be used as the starting point for customized devices. Each design can be generated programmatically using the open-source Qiskit Metal package, and simulated using finite-element electromagnetic solvers. We present a robust workflow for achieving high accuracy on design simulations. Many designs in the database are experimentally validated, showing excellent agreement between simulated and measured parameters. Our database includes a front-end interface that allows users to generate ``best-guess'' designs based on desired circuit parameters. This project lowers the barrier to entry for research groups seeking to make a new class of devices by providing them a well-characterized starting point from which to refine their designs.

Despite progress in automated fact-checking, most systems require a significant amount of labeled training data, which is expensive. In this paper, we propose a novel zero-shot method, which instead of operating directly on the claim and evidence sentences, decomposes them into semantic triples augmented using external knowledge graphs, and uses large language models trained for natural language inference. This allows it to generalize to adversarial datasets and domains that supervised models require specific training data for. Our empirical results show that our approach outperforms previous zero-shot approaches on FEVER, FEVER-Symmetric, FEVER 2.0, and Climate-FEVER, while being comparable or better than supervised models on the adversarial and the out-of-domain datasets.

We investigate the geometry and topology of compact submanifolds of arbitrary codimension in space forms satisfying a certain pinching condition involving the length of the second fundamental form and the mean curvature. We prove that this pinching condition either forces homology to vanish in a range of intermediate dimensions, or completely determines the submanifold up to congruence. The results are sharp and extend previous results due to several authors without imposing any further assumption on the mean curvature.

Prior research has shown that typical fact-checking models for stand-alone claims struggle with claims made in dialogues. As a solution, fine-tuning these models on labelled dialogue data has been proposed. However, creating separate models for each use case is impractical, and we show that fine-tuning models for dialogue results in poor performance on typical fact-checking. To overcome this challenge, we present techniques that allow us to use the same models for both dialogue and typical fact-checking. These mainly focus on retrieval adaptation and transforming conversational inputs so that they can be accurately predicted by models trained on stand-alone claims. We demonstrate that a typical fact-checking model incorporating these techniques is competitive with state-of-the-art models fine-tuned for dialogue, while maintaining its accuracy on stand-alone claims., Comment: Accepted to EMNLP 2023

Ion acceleration from gaseous targets driven by relativistic-intensity lasers was demonstrated as early as the late 90s, yet most of the experiments conducted to date have involved picosecond-duration, Nd:glass lasers operating at low repetition rate. Here, we present measurements on the interaction of ultraintense ($\sim 10^{20}\,\rm W\,cm^{-2}$, 1 PW), ultrashort ($\sim 70\,\rm fs$) Ti:Sa laser pulses with near-critical ($\sim 10^{20}\,\rm cm^{-3}$) helium gas jets, a debris-free targetry compatible with high ($\sim 1\,\rm Hz$) repetition rate operation. We provide evidence of $\alpha$ particles being forward accelerated up to $\sim 2.7\,\rm MeV$ energy with a total flux of $\sim 10^{11}\,\rm sr^{-1}$ as integrated over $>0.1 \,\rm MeV$ energies and detected within a $0.5\,\rm mrad$ solid angle. We also report on on-axis emission of relativistic electrons with an exponentially decaying spectrum characterized by a $\sim 10\,\rm MeV$ slope, i.e., five times larger than the standard ponderomotive scaling. The total charge of these electrons with energy above 2 MeV is estimated to be of $\sim 1 \,\rm nC$, corresponding to $\sim 0.1\,\%$ of the laser drive energy. In addition, we observe the formation of a plasma channel, extending longitudinally across the gas density maximum and expanding radially with time. These results are well captured by large-scale particle-in-cell simulations, which reveal that the detected fast ions most likely originate from reflection off the rapidly expanding channel walls. The latter process is predicted to yield ion energies in the MeV range, which compare well with the measurements. Finally, direct laser acceleration is shown to be the dominant mechanism behind the observed electron energization.

We investigate the compact submanifolds in Riemannian space forms of nonnegative sectional curvature that satisfy a lower bound on the Ricci curvature, that bound depending solely on the length of the mean curvature vector of the immersion. While generalizing the results, we give a positive answer to a conjecture by H. Xu and J. Gu in (2013, Geom. Funct. Anal. 23). Our main accomplishment is the elimination of the need for the mean curvature vector field to be parallel.

We present a superconducting circuit device that provides active, on-demand, tunable dissipation on a target mode of the electromagnetic field. Our device is based on a tunable "dissipator" that can be made lossy when tuned into resonance with a broadband filter mode. When driven parametrically, this dissipator induces loss on any mode coupled to it with energy detuning equal to the drive frequency. We demonstrate the use of this device to reset a superconducting qubit's readout cavity after a measurement, removing photons with a characteristic rate above $50\ \mu\mathrm{s}^{-1}$. We also demonstrate that the dissipation can be driven constantly to simultaneously damp and cool the cavity, effectively eliminating thermal photon fluctuations as a relevant decoherence channel. Our results demonstrate the utility of our device as a modular tool for environmental engineering and entropy removal in circuit QED., Comment: 12 pages, 6 figures

In this paper, we study the problem of extremely large (XL) multiple-input multiple-output (MIMO) channel estimation in the terahertz (THz) frequency band, considering the presence of propagation delays across the entire array apertures at both communication ends, which naturally leads to frequency selectivity. This problem is known as beam squint and may be pronounced when communications are subject to multipath fading conditions. Multi-carrier (MC) transmission schemes, which are usually deployed in THz communication systems to address these issues, suffer from high peak-to-average power ratio, which is specifically dominant in this frequency band where low transmit power is mostly feasible. Furthermore, the frequency selectivity caused by severe molecular absorption in the THz band necessitates delicate consideration in MC system design. Motivated by the benefits of single-carrier (SC) waveforms for practical THz communication systems, diverging from the current dominant research trend on MC systems, we devise a novel channel estimation problem formulation in the time domain for SC XL MIMO systems subject to multipath signal propagation, spatial wideband effects, and molecular absorption. An efficient alternating minimization approach is presented to solve the proposed mixed-integer sparse problem formulation. The conducted extensive performance evaluation results validate that the proposed XL MIMO estimation scheme exhibits superior performance than conventional SC- and MC-based techniques, approaching the idealized lower bound.

Fact verification systems assess a claim's veracity based on evidence. An important consideration in designing them is faithfulness, i.e. generating explanations that accurately reflect the reasoning of the model. Recent works have focused on natural logic, which operates directly on natural language by capturing the semantic relation of spans between an aligned claim with its evidence via set-theoretic operators. However, these approaches rely on substantial resources for training, which are only available for high-resource languages. To this end, we propose to use question answering to predict natural logic operators, taking advantage of the generalization capabilities of instruction-tuned language models. Thus, we obviate the need for annotated training data while still relying on a deterministic inference system. In a few-shot setting on FEVER, our approach outperforms the best baseline by $4.3$ accuracy points, including a state-of-the-art pre-trained seq2seq natural logic system, as well as a state-of-the-art prompt-based classifier. Our system demonstrates its robustness and portability, achieving competitive performance on a counterfactual dataset and surpassing all approaches without further annotation on a Danish verification dataset. A human evaluation indicates that our approach produces more plausible proofs with fewer erroneous natural logic operators than previous natural logic-based systems., Comment: EMNLP 2023

We investigate the topology of the compact hypersurfaces in round spheres whose Ricci curvature satisfies an appropriate bound that only depends on the mean curvature of the submanifold. In this paper, the use of the Bochner technique allows same stronger results than the ones obtained by us in the case of submanifolds lying in any codimension.