Search

Linear regressions with endogeneity are widely used to estimate causal effects. This paper studies a statistical framework that has two common issues, endogeneity of the regressors, and heteroskedasticity that is allowed to depend on endogenous regressors, i.e., endogenous heteroskedasticity. We show that the presence of such conditional heteroskedasticity in the structural regression renders the two-stages least squares estimator inconsistent. To solve this issue, we propose sufficient conditions together with a control function approach to identify and estimate the causal parameters of interest. We establish statistical properties of the estimator, say consistency and asymptotic normality, and propose valid inference procedures. Monte Carlo simulations provide evidence of the finite sample performance of the proposed methods, and evaluate different implementation procedures. We revisit an empirical application about job training to illustrate the methods.

In addition to describing our universe, gravitational theories profoundly inspire the study of emergent properties of exotic phases of matter. While the Anti-de Sitter/conformal field theory (AdS/CFT) correspondence is one of the most celebrated examples, the field of fractonic matter -- driven in part by gapless phases resembling linearized gravity -- has also seen rapid developments. Despite the deep implications of both areas, connections between them remain sparse, primarily due to the difficulty in constructing explicit models that encapsulate both fields' essential features. Here we demonstrate the efficacy of the recently proposed Hyperbolic Fracton Model as a concrete model for AdS/CFT duality. Using explicit numerical and analytical calculations on the discrete hyperbolic lattice, we show that the boundary state exhibits conformal field theory properties. Our main result is that bulk defects induce an emergent temperature for the boundary state, proportional to the defect perimeter, in quantitative agreement with the expected behaviour of a black hole in AdS spacetime. The Hyperbolic Fracton Model thus emerges as a unique lattice model of holographic principle equipped with a well-defined bulk Hamiltonian, and offers a promising gateway for studying a wide range of holographic phenomena., Comment: 6 pages, 7 figures

This work describes the elaboration of rutile titanium dioxide films with high refractive indices and low scattering by sol-gel process and controlled crystallization. The evolutions of the optical properties and crystalline structure of sol-gel processed titania coatings on fused silica were investigated for different thermal budgets of the annealing post-treatment using ellipsometry, spectrophotometry, X-ray diffraction and electronic microscopy. It reveals that anatase and rutile coatings with refractive indices of 2.5 and 2.7 can be prepared with associated optical loss of 0.5% and 1%, respectively, which are excellent compromise for applications in integrated photonics. These evolutions are associated to the thermally induced mass transfer and phase transitions occurring during thermal annealing that involves first the nucleation growth and sintering of anatase polyoriented nanocrystals, followed by the transformation into rutile polyoriented nanocrystals. Concomitantly, rutile crystals with (110) faces parallel to the surface consume surrounding anatase and rutile nanocrystals by diffusive sintering to yield micron-size rutile monocrystalline and monooriented platelets patchwork, exhibiting refractive index of 2.73 and 1.2% optical loss. The formation of these platelets is governed by surface energies and is responsible for the increase in optical loss.

We consider the subcritical nonlinear Schr\"odinger (NLS) in dimension one posed on the unbounded real line. Several previous works have considered the deep neural network approximation of NLS solutions from the numerical and theoretical point of view in the case of bounded domains. In this paper, we introduce a new PINNs method to treat the case of unbounded domains and show rigorous bounds on the associated approximation error in terms of the energy and Strichartz norms, provided a reasonable integration scheme is available. Applications to traveling waves, breathers and solitons, as well as numerical experiments confirming the validity of the approximation are also presented as well., Comment: 24 pages

CP-violation (CPV) is a rare phenomenon in the Standard Model whilst there is compelling indirect evidence for additional CPV sources in the Universe. The search for CPV effects at the LHC is thus one of the best-motivated precision tests of the Standard Model (SM) and an excellent probe of New Physics. NLO QCD corrections can affect the predictions for those measurements substantially. We study the impact of NLO QCD corrections in $WH$ and $ZH$ production in the Standard Model Effective Field Theory with bosonic CP-odd dimension-6 operators. We analyze the angular distributions at LO of those processes that can be used to probe CPV effects. We then show how NLO QCD effects modify those distributions. We encounter that the corrections have a clear angular dependence and differ between the SM, the dimension-6 squared and their interference, emphasising the need for an exact inclusion of NLO QCD in precision computations. We then perform a phenomenological analysis of $WH$ production at the LHC to study the impact of NLO QCD effects on the projected bounds on the CP-odd Wilson Coefficient $c_{\varphi\widetilde{W}}$. NLO QCD effects in the signal improve the bounds by $\sim10\%$ but reduce the significance of the interference., Comment: 32 pages, 19 figures, 13 tables, 4 appendices

We consider the Principal Chiral Field model posed in 1+1 dimensions into the Lie group $\text{SL}(2,\mathbb R)$. In this work we show the nonlinear stability of small enough nonsingular solitons. The method of proof involves the use of vector field methods as in a previous work by the second and third authors dealing with the Einstein's field equations under the Belinski-Zakharov formalism, extending for all times the size of suitable null weighted norms of the perturbations at time zero., Comment: 15 pages

In this work, a rigorous proof of the nonexistence of breather solutions for NLS equations is presented. By using suitable virial functionals, we are able to characterize the nonexistence of breather solutions, different from standing waves, by only using their inner energy and the power of the corresponding nonlinearity of the equation. We extend this result for several NLS models with different power nonlinearities and even the derivative and logarithmic NLS equations., Comment: 22 pages; typos corrected

We report on a direct search for elastic photon-photon scattering using x-ray and $\gamma$ photons from a laser-plasma based experiment. A gamma photon beam produced by a laser wakefield accelerator provided a broadband gamma spectrum extending to above $E_\gamma = 200$ MeV. These were collided with a dense x-ray field produced by the emission from a laser heated germanium foil at $E_x \approx 1.4$ keV, corresponding to an invariant mass of $\sqrt{s} = 1.22 \pm 0.22$ MeV. In these asymmetric collisions elastic scattering removes one x-ray and one high-energy $\gamma$ photon and outputs two lower energy $\gamma$ photons. No changes in the $\gamma$ photon spectrum were observed as a result of the collisions allowing us to place a 95% upper bound on the cross section of $1.5 \times 10^{15}\,\mu$b. Although far from the QED prediction, this represents the lowest upper limit obtained so far for $\sqrt{s} \lesssim 1$ MeV., Comment: 6 pages, 10 figures

We define a congruence that copes with null next-symbol probabilities that arise when the output of a language model is constrained by some means during text generation. We develop an algorithm for efficiently learning the quotient with respect to this congruence and evaluate it on case studies for analyzing statistical properties of LLM., Comment: Workshop Paper

Recently, using large pretrained Transformer models for transfer learning tasks has evolved to the point where they have become one of the flagship trends in the Natural Language Processing (NLP) community, giving rise to various outlooks such as prompt-based, adapters or combinations with unsupervised approaches, among many others. This work proposes a 3 Phase technique to adjust a base model for a classification task. First, we adapt the model's signal to the data distribution by performing further training with a Denoising Autoencoder (DAE). Second, we adjust the representation space of the output to the corresponding classes by clustering through a Contrastive Learning (CL) method. In addition, we introduce a new data augmentation approach for Supervised Contrastive Learning to correct the unbalanced datasets. Third, we apply fine-tuning to delimit the predefined categories. These different phases provide relevant and complementary knowledge to the model to learn the final task. We supply extensive experimental results on several datasets to demonstrate these claims. Moreover, we include an ablation study and compare the proposed method against other ways of combining these techniques., Comment: 1 figure, 7 tables, 12 pages

The expansion of streaming media and e-commerce has led to a boom in recommendation systems, including Sequential recommendation systems, which consider the user's previous interactions with items. In recent years, research has focused on architectural improvements such as transformer blocks and feature extraction that can augment model information. Among these features are context and attributes. Of particular importance is the temporal footprint, which is often considered part of the context and seen in previous publications as interchangeable with positional information. Other publications use positional encodings with little attention to them. In this paper, we analyse positional encodings, showing that they provide relative information between items that are not inferable from the temporal footprint. Furthermore, we evaluate different encodings and how they affect metrics and stability using Amazon datasets. We added some new encodings to help with these problems along the way. We found that we can reach new state-of-the-art results by finding the correct positional encoding, but more importantly, certain encodings stabilise the training., Comment: 19 pages, 3 figures, 12 tables

Composing poetry or lyrics involves several creative factors, but a challenging aspect of generation is the adherence to a more or less strict metric and rhyming pattern. To address this challenge specifically, previous work on the task has mainly focused on reverse language modeling, which brings the critical selection of each rhyming word to the forefront of each verse. On the other hand, reversing the word order requires that models be trained from scratch with this task-specific goal and cannot take advantage of transfer learning from a Pretrained Language Model (PLM). We propose a novel fine-tuning approach that prepends the rhyming word at the start of each lyric, which allows the critical rhyming decision to be made before the model commits to the content of the lyric (as during reverse language modeling), but maintains compatibility with the word order of regular PLMs as the lyric itself is still generated in left-to-right order. We conducted extensive experiments to compare this fine-tuning against the current state-of-the-art strategies for rhyming, finding that our approach generates more readable text and better rhyming capabilities. Furthermore, we furnish a high-quality dataset in English and 12 other languages, analyse the approach's feasibility in a multilingual context, provide extensive experimental results shedding light on good and bad practices for lyrics generation, and propose metrics to compare methods in the future., Comment: 18 pages, 1 figure

We present SMEFiT3.0, an updated global SMEFT analysis of Higgs, top quark, and diboson production data from the LHC complemented by electroweak precision observables (EWPOs) from LEP and SLD. We consider recent inclusive and differential measurements from the LHC Run II, alongside with a novel implementation of the EWPOs based on independent calculations of the relevant EFT contributions. We estimate the impact of HL-LHC measurements on the SMEFT parameter space when added on top of SMEFiT3.0, through dedicated projections extrapolating from Run II data. We quantify the significant constraints that measurements from two proposed high-energy circular $e^+e^-$ colliders, the FCC-ee and the CEPC, would impose on both the SMEFT parameter space and on representative UV-complete models. Our analysis considers projections for the FCC-ee and the CEPC based on the latest running scenarios and includes $Z$-pole EWPOs, fermion-pair, Higgs, diboson, and top quark production, using optimal observables for both the $W^+W^-$ and the $t\bar{t}$ channels. The framework presented in this work may be extended to other future colliders and running scenarios, providing timely input to ongoing studies towards future high-energy particle physics facilities., Comment: 72 pages, 22 figures

The ARS 548 RDI Radar is a premium model of the fifth generation of 77 GHz long range radar sensors with new RF antenna arrays, which offer digital beam forming. This radar measures independently the distance, speed and angle of objects without any reflectors in one measurement cycle based on Pulse Compression with New Frequency Modulation. Unfortunately, to the best of our knowledge, there are no open source drivers available for Linux systems to enable users to analyze the data acquired by the sensor. In this paper, we present a driver that can interpret the data from the ARS 548 RDI sensor and make it available over the Robot Operating System versions 1 and 2 (ROS and ROS2). Thus, these data can be stored, represented, and analyzed using the powerful tools offered by ROS. Besides, our driver offers advanced object features provided by the sensor, such as relative estimated velocity and acceleration of each object, its orientation and angular velocity. We focus on the configuration of the sensor and the use of our driver including its filtering and representation tools. Besides, we offer a video tutorial to help in its configuration process. Finally, a dataset acquired with this sensor and an Ouster OS1-32 LiDAR sensor, to have baseline measurements, is available, so that the user can check the correctness of our driver., Comment: 19 pages, 6 figures and 22 references

We study homotopic-to-the-identity torus homeomorphisms, whose rotation set has nonempty interior. We prove that any such map is monotonically semiconjugate to a homeomorphism that preserves the Lebesgue measure, and that has the same rotation set. Furthermore, the dynamics of the quotient map has several interesting chaotic traits: for instance, it is topologically mixing, has a dense set of periodic points and is continuum-wise expansive. In particular, this shows that a convex compact set of $\mathbb{R}^2$ with nonempty interior is the rotation set of the lift of a homeomorphism of $\mathbb{T}^2$ if and only if it is the rotation set of the lift of a conservative homeomorphism., Comment: 95 pages, 16 figures

Transforming education starts with ensuring all children attain foundational learning--literacy, numeracy and socio-emotional skills which provide the building blocks for all other learning. To this end, a global coalition has been established, inviting governments and stakeholders to endorse the Commitment to Action on Foundational Learning as a first step to accelerate the agenda on foundational learning. Organized around the RAPID Framework, this report presents insights on the progress countries have made towards securing foundational learning, using data based on survey responses from 94 low- and middle-income countries and UNICEF's yearly monitoring exercise with its programme country offices. Complementing the Foundational Learning Action Tracker, an initiative by UNICEF and the Hempel Foundation, this report presents findings on the extent to which RAPID policy actions are being implemented and enabling systems are in place to support them, towards the goal of achieving foundational learning for all. [The Hempel Foundation provided support for this initiative.]

Pre-trained Large Language Models (LLMs) often struggle on out-of-domain datasets like healthcare focused text. We explore specialized pre-training to adapt smaller LLMs to different healthcare datasets. Three methods are assessed: traditional masked language modeling, Deep Contrastive Learning for Unsupervised Textual Representations (DeCLUTR), and a novel pre-training objective utilizing metadata categories from the healthcare settings. These schemes are evaluated on downstream document classification tasks for each dataset, with additional analysis of the resultant embedding spaces. Contrastively trained models outperform other approaches on the classification tasks, delivering strong performance from limited labeled data and with fewer model parameter updates required. While metadata-based pre-training does not further improve classifications across the datasets, it yields interesting embedding cluster separability. All domain adapted LLMs outperform their publicly available general base LLM, validating the importance of domain-specialization. This research illustrates efficient approaches to instill healthcare competency in compact LLMs even under tight computational budgets, an essential capability for responsible and sustainable deployment in local healthcare settings. We provide pre-training guidelines for specialized healthcare LLMs, motivate continued inquiry into contrastive objectives, and demonstrates adaptation techniques to align small LLMs with privacy-sensitive medical tasks.

Contemporary large language models (LLMs) may have utility for processing unstructured, narrative free-text clinical data contained in electronic health records (EHRs) -- a particularly important use-case for mental health where a majority of routinely-collected patient data lacks structured, machine-readable content. A significant problem for the the United Kingdom's National Health Service (NHS) are the long waiting lists for specialist mental healthcare. According to NHS data, in each month of 2023, there were between 370,000 and 470,000 individual new referrals into secondary mental healthcare services. Referrals must be triaged by clinicians, using clinical information contained in the patient's EHR to arrive at a decision about the most appropriate mental healthcare team to assess and potentially treat these patients. The ability to efficiently recommend a relevant team by ingesting potentially voluminous clinical notes could help services both reduce referral waiting times and with the right technology, improve the evidence available to justify triage decisions. We present and evaluate three different approaches for LLM-based, end-to-end ingestion of variable-length clinical EHR data to assist clinicians when triaging referrals. Our model is able to deliver triage recommendations consistent with existing clinical practices and it's architecture was implemented on a single GPU, making it practical for implementation in resource-limited NHS environments where private implementations of LLM technology will be necessary to ensure confidential clinical data is appropriately controlled and governed.

We study a non-universal flavor scenario at the level of the Standard Model Effective Field Theory, according to which the matrix of Wilson coefficients $c_{uW}$ of an up-type electroweak quark dipole operator is aligned with the up-type Yukawa coupling. Such an alignment usually follows from the assumption of Minimal Flavor Violation (MFV), away from which we step by allowing the entries of $c_{uW}$ to be sizable along the first quark generations. A particular example, which we refer to as ``inverse hierarchy MFV", features Wilson coefficients inversely proportional to quark masses, and arises from BSM models respecting MFV and containing heavy fields that replicate the mass hierarchy of SM quarks. We then analyze the phenomenology driven by $c_{uW}$ at colliders and at lower-energy flavor experiments. We show that precision measurements of the process $pp\rightarrow W h\rightarrow \gamma\gamma\ell\nu$ at FCC-$hh$ could set an upper bound on $|c_{uW}|\lesssim\mathcal{O}(10^{-2})(\Lambda/{\rm TeV})^{2}$, with $\Lambda$ the cutoff of the effective field theory. This bound is an order of magnitude stronger than the existing LHC bounds. Moreover, we estimate that $W h\rightarrow b\bar b \ell\nu$ at HL-LHC could also give competitive bounds. In the low-energy regime, we consider bounds arising from rare kaon decays, which turn out to be loose, $|c_{uW}^{11}|<\mathcal{O}(1)(\Lambda/{\rm TeV})^{2}$. We finally demonstrate that our flavor and operator assumptions can be derived from a weakly-coupled UV model, which we choose to simultaneously illustrate the UV origin of inverse hierarchy MFV., Comment: 22 pages (main) + 15 pages (appendices & references), 6 figures, 7 tables. JHEP published version. Typos are corrected

The entry of large language models (LLMs) into research and commercial spaces has led to a trend of ever-larger models, with initial promises of generalisability, followed by a widespread desire to downsize and create specialised models without the need for complete fine-tuning, using Parameter Efficient Fine-tuning (PEFT) methods. We present an investigation into the suitability of different PEFT methods to clinical decision-making tasks, across a range of model sizes, including extremely small models with as few as $25$ million parameters. Our analysis shows that the performance of most PEFT approaches varies significantly from one task to another, with the exception of LoRA, which maintains relatively high performance across all model sizes and tasks, typically approaching or matching full fine-tuned performance. The effectiveness of PEFT methods in the clinical domain is evident, particularly for specialised models which can operate on low-cost, in-house computing infrastructure. The advantages of these models, in terms of speed and reduced training costs, dramatically outweighs any performance gain from large foundation LLMs. Furthermore, we highlight how domain-specific pre-training interacts with PEFT methods and model size, and discuss how these factors interplay to provide the best efficiency-performance trade-off. Full code available at: tbd.

Difficult-to-treat depression (DTD) has been proposed as a broader and more clinically comprehensive perspective on a person's depressive disorder where despite treatment, they continue to experience significant burden. We sought to develop a Large Language Model (LLM)-based tool capable of interrogating routinely-collected, narrative (free-text) electronic health record (EHR) data to locate published prognostic factors that capture the clinical syndrome of DTD. In this work, we use LLM-generated synthetic data (GPT3.5) and a Non-Maximum Suppression (NMS) algorithm to train a BERT-based span extraction model. The resulting model is then able to extract and label spans related to a variety of relevant positive and negative factors in real clinical data (i.e. spans of text that increase or decrease the likelihood of a patient matching the DTD syndrome). We show it is possible to obtain good overall performance (0.70 F1 across polarity) on real clinical data on a set of as many as 20 different factors, and high performance (0.85 F1 with 0.95 precision) on a subset of important DTD factors such as history of abuse, family history of affective disorder, illness severity and suicidality by training the model exclusively on synthetic data. Our results show promise for future healthcare applications especially in applications where traditionally, highly confidential medical data and human-expert annotation would normally be required.

Graph Neural Networks (GNNs) have achieved great success in Knowledge Graph Completion (KGC) by modelling how entities and relations interact in recent years. However, the explanation of the predicted facts has not caught the necessary attention. Proper explanations for the results of GNN-based KGC models increase model transparency and help researchers develop more reliable models. Existing practices for explaining KGC tasks rely on instance/subgraph-based approaches, while in some scenarios, paths can provide more user-friendly and interpretable explanations. Nonetheless, the methods for generating path-based explanations for KGs have not been well-explored. To address this gap, we propose Power-Link, the first path-based KGC explainer that explores GNN-based models. We design a novel simplified graph-powering technique, which enables the generation of path-based explanations with a fully parallelisable and memory-efficient training scheme. We further introduce three new metrics for quantitative evaluation of the explanations, together with a qualitative human evaluation. Extensive experiments demonstrate that Power-Link outperforms the SOTA baselines in interpretability, efficiency, and scalability., Comment: Accepted by KDD 2024