Search

Large Language Models (LLMs) have shown impressive results in multiple domains of natural language processing (NLP) but are mainly focused on the English language. Recently, more LLMs have incorporated a larger proportion of multilingual text to represent low-resource languages. In Arabic NLP, several Arabic-centric LLMs have shown remarkable results on multiple benchmarks in the past two years. However, most Arabic LLMs have more than 7 billion parameters, which increases their hardware requirements and inference latency, when compared to smaller LLMs. This paper introduces Arabic Stable LM 1.6B in a base and chat version as a small but powerful Arabic-centric LLM. Our Arabic Stable LM 1.6B chat model achieves impressive results on several benchmarks beating multiple models with up to 8x the parameters. In addition, we show the benefit of mixing in synthetic instruction tuning data by augmenting our fine-tuning data with a large synthetic dialogue dataset.

Let $K_n=(V,E)$ be the complete graph with $n\geq 3$ vertices (here $V$ and $E$ denote the set of vertices and edges of $K_n$ respectively). We find the optimal value ${\bf{C}}_{n,p}$ such that the inequality $$\|f-m_f\|_p\le {\bf C}_{n,p}{\rm Var}_{p}f$$ holds for every $f:V\to \mathbb{R},$ where ${\rm Var}_p$ stands for the $p$-variation, and $m_f$ stands for the average value of $f$, for all $p\in[1,3+\delta^1_n)\cup (3+\delta^2_n,+\infty)$, for $\delta^1_n=\frac{1}{2n^2\log(n)}+O(1/n^3)$ and $\delta^2_n=\frac{2}{n}+O(1/n^2).$ Moreover, we characterize all the maximizer functions in that case. The behavior of the maximizers is different in each of the intervals $(1,2)$, $(2,3+\delta^{1}_n)$ and $(3+\delta^{2}_n,\infty).$, Comment: 11 pages

Recently published work on rephrasing natural text data for pre-training LLMs has shown promising results when combining the original dataset with the synthetically rephrased data. We build upon previous work by replicating existing results on C4 and extending them with our optimized rephrasing pipeline to the English, German, Italian, and Spanish Oscar subsets of CulturaX. Our pipeline leads to increased performance on standard evaluation benchmarks in both the mono- and multilingual setup. In addition, we provide a detailed study of our pipeline, investigating the choice of the base dataset and LLM for the rephrasing, as well as the relationship between the model size and the performance after pre-training. By exploring data with different perceived quality levels, we show that gains decrease with higher quality. Furthermore, we find the difference in performance between model families to be bigger than between different model sizes. This highlights the necessity for detailed tests before choosing an LLM to rephrase large amounts of data. Moreover, we investigate the effect of pre-training with synthetic data on supervised fine-tuning. Here, we find increasing but inconclusive results that highly depend on the used benchmark. These results (again) highlight the need for better benchmarking setups. In summary, we show that rephrasing multilingual and low-quality data is a very promising direction to extend LLM pre-training data., Comment: 21 pages, 4 figures, 12 tables

We investigate the dynamical and physical structures of bright-rimmed clouds (BRCs) in a nearby HII region. We focused on carbon- and oxygen-bearing species that trace photon-dominated regions (PDRs) and warm molecular cloud surfaces in order to understand the effect of UV radiation from the exciting stars on the cloud structure. We mapped four regions around the most prominent BRCs at scales of 4--10 arcmin in the HII region IC 1396 in [CII] 158 micron with (up)GREAT on board SOFIA. IC 1396 is predominantly excited by an O6.5V star. Toward IC 1396A, we also observed [OI] 63 micron and 145 micron. We combined these observations with JCMT archive data, which provide the low-J transitions of CO, $^{13}$CO, and C$^{18}$O. All spectra are velocity-resolved. The line profiles show a variety of velocity structures, which we investigated in detail for all observed emission lines. We find no clear sign of photoevaporating flows in the [CII] spectra, although the uncertainty in the location of the BRCs along the line of sight makes this interpretation inconclusive. Our analysis of the [$^{13}$CII] emission in IC 1396A suggests that the [CII] is likely mostly optically thin. The heating efficiency, measured by the ([CII]+[OI] 63 micron)/far-infrared intensity ratio, is higher in the northern part of IC 1396A than in the southern part, which may indicate a difference in the dust properties of the two areas. The complex velocity structures identified in the BRCs of IC 1396, which is apparently a relatively simple HII region, highlight the importance of velocity-resolved data for disentangling different components along the line of sight and thus facilitating a detailed study of the dynamics of the cloud. We also demonstrate that the optically thin [$^{13}$CII] and [OI] 145 micron emission lines are essential for a conclusive interpretation of the [CII] 158 micron and [OI] 63 micron line profiles., Comment: Accepted for publication in A&A, 26 pages including Appendices

The incompatibility of explicit diffeomorphism violation with Riemannian geometry within the gravitational Standard-Model Extension (SME) is revisited. We review two methods of how to deal with this problem. The first is based on an approach proposed originally by St\"{u}ckelberg and the latter is to restrict spacetime geometry via the dynamical field equations and the second Bianchi identity. Moreover, a third technique is introduced, which is novel and relies on isometries of a gravitational system. Our conclusion is that an SME background field configuration compatible with Riemannian geometry is more likely to be determined the more diffeomorphisms are isometries of the particular system. This finding has the potential to provide an alternative treatment of explicit spacetime symmetry violation in gravity., Comment: 6 pages, 1 figure

Scaling up interferometry to 8m collectors should smooth-out the optical piston perturbations and allow a slow fringe tracker to obtain high precision correction on faint targets. In practice, the GRAVITY fringe tracker still observes high frequency OPD components that limit the exposure time, its precision and limiting magnitude. Perturbations seem to come from mechanical vibrations in the train of mirrors. As part of the GRAVITY+ efforts, accelerometers were added to all the mirrors of the coud\'e train to compensate in real-time the optical path using the main delay lines. We show their effectiveness on vibrations peaks between 40 and 200Hz and outline prospects for the upgrade of the deformable mirrors and the beam-compressor differential delay lines., Comment: 11 pages, 12 figures, Proc. SPIE Astronomical Telescopes + Instrumentation 2024 (Yokohama Japan), Optical and Infrared Interferometry and Imaging VI

The expanding molecular ring (EMR) manifests itself as a parallelogram in the position-velocity diagram of spectral line emission from the Central Molecular Zone (CMZ) surrounding the Galacic centre (GC). Using multiwavelength data, we investigate the gas kinematics, star formation activity, and the presence of shocked gas in a 200 pc long high velocity gas stream (V~ +150 km/s) with a double helix morphology named the helix stream, that is located 15-55 pc above the CMZ and is kinematically associated with the EMR/parallelogram. We carried out molecular line observations using the IRAM 30m, Yebes 40m, and APEX 12m telescopes. The detection of four rotational transitions of the SiO molecule indicate the presence of shocks. We derived the SiO column densities and abundances in different regions of the helix stream. The presence of protostellar clumps and a candidate HII region signify the ongoing star formation activity within the helix stream. The cloud is massive (2.5x10^6 M_sun) and highly turbulent. We find evidence of cloud-cloud collisions towards the eastern edge (l~1.3{\deg}), suggesting a dynamic interaction with the CMZ. An expanding shell is detected within the cloud with radius of 6.7 pc and an expansion velocity of 35 km/s. The shell might be powered by several supernovae or a single hypernova. The SiO abundance within the helix stream implies extensive shock processes occurring on large scales. The helical or cork-screw velocity structure of the helix stream indicates twisting and turning motions within the cloud. We propose that the helix stream is the continuation of the near side bar lane, that is overshooting after brushing the CMZ. Our findings carry profound implications for understanding star formation in extreme conditions and elucidate the intricate properties of gas and dust associated with nuclear inflows in barred spiral galaxies., Comment: 20 pages, 20 figures, 4 tables, accepted for publication in A&A

We study a general class of effective backgrounds that break diffeomorphism invariance and their potential roles in cosmology. Specifically, we examine both explicit and spontaneous background fields which display distinct transformation properties and are characterized with different dynamics. For explicit breaking, we focus on the $t$-sector of the minimal gravitational Standard-Model Extension (SME) and for spontaneous breaking on a vector field called the bumblebee model. In both cases, we derive the modified Friedmann equations and find a configuration of the background fields that preserve isotropy and homogeneity. We show that the explicit $t$-sector admits phases of accelerated expansion of the universe with standard matter., Comment: 45 pages and 2 figures

Sharp restriction theory and the finite field extension problem have both received a great deal of attention in the last two decades, but so far they have not intersected. In this paper, we initiate the study of sharp restriction theory on finite fields. We prove that constant functions maximize the Fourier extension inequality from the parabola $\mathbb{P}^1\subset \mathbb{F}^{2\ast}_q$ and the paraboloid $\mathbb{P}^2\subset \mathbb{F}_q^{3\ast}$ at the euclidean Stein-Tomas endpoint; here, $\mathbb{F}_q^{d\ast}$ denotes the (dual) $d$-dimensional vector space over the finite field $\mathbb F_q$ with $q=p^n$ elements, where $p$ is a prime number greater than $3$ or $2$, respectively. We fully characterize the maximizers for the $L^2\to L^4$ extension inequality from $\mathbb{P}^2$ whenever $q\equiv 1(\text{mod}\, 4)$. Our methods lead to analogous results on the hyperbolic paraboloid, whose corresponding euclidean problem remains open. We further establish that constants maximize the $L^2\to L^4$ extension inequality from the cone $\Gamma^3:=\{(\boldsymbol{\xi},\tau, \sigma)\in \mathbb{F}^{4\ast}_q: \tau\sigma=\boldsymbol{\xi}^2\}\setminus \{{\bf 0}\}$ whenever $q\equiv 3(\text{mod}\, 4)$. By contrast, we prove that constant functions fail to be critical points for the corresponding inequality on $\Gamma^3\cup \{{\bf 0}\}$ over $\mathbb{F}_p^4$. While some inspiration is drawn from the euclidean setting, entirely new phenomena emerge which are related to the underlying arithmetic and discrete structures., Comment: 34 pages; v2: typos corrected, references added

Context. The diffuse TeV gamma-ray emission detected in the inner $\sim$ 100 pc of the Galactic Center suggests the existence of a central cosmic-ray accelerator reaching $\sim$ PeV energies. It is interesting to associate this so-called PeVatron with the point source HESS J1745$-$290, whose position is consistent with that of the central supermassive black hole, Sgr A*. However, the point source shows a spectral break at a few TeV, which is not shown by the diffuse emission, challenging this association. Aims. We seek to build an emission model for the point source consistent with both emissions being produced by the same population of relativistic protons, continuously injected with a power-law spectrum up to $\sim$ PeV energies, near Sgr A*. Methods. In our model, we assume that the point source is produced by hadronic collisions between the cosmic rays and the gas in the accretion flow of Sgr A*. The cosmic-ray density is calculated taking into consideration cosmic-ray transport due to diffusion and advection, while the properties of the gas are obtained from previous numerical simulations of the accretion flow. Results. Our model succeeds in explaining both the point source and the diffuse emission with the same cosmic rays injected in the vicinity of Sgr A*, as long as the coherence length of the magnetic turbulence in the accretion flow is $l_c\sim(1-3)\times 10^{14}\,\mathrm{cm}$. The spectral break of the point source appears naturally due to an energy-dependent transition in the way the cosmic rays diffuse within the inner $\sim 0.1$ pc of the accretion flow (where most of the emission is produced) Conclusions. Our model supports the idea that Sgr A* can be a PeVatron, whose accelerated cosmic rays give rise to both the point source and the diffuse emission. Future TeV telescopes, like CTAO, will be able to test this model.

We introduce Stable Code, the first in our new-generation of code language models series, which serves as a general-purpose base code language model targeting code completion, reasoning, math, and other software engineering-based tasks. Additionally, we introduce an instruction variant named Stable Code Instruct that allows conversing with the model in a natural chat interface for performing question-answering and instruction-based tasks. In this technical report, we detail the data and training procedure leading to both models. Their weights are available via Hugging Face for anyone to download and use at https://huggingface.co/stabilityai/stable-code-3b and https://huggingface.co/stabilityai/stable-code-instruct-3b. This report contains thorough evaluations of the models, including multilingual programming benchmarks, and the MT benchmark focusing on multi-turn dialogues. At the time of its release, Stable Code is the state-of-the-art open model under 3B parameters and even performs comparably to larger models of sizes 7 billion and 15 billion parameters on the popular Multi-PL benchmark. Stable Code Instruct also exhibits state-of-the-art performance on the MT-Bench coding tasks and on Multi-PL completion compared to other instruction tuned models. Given its appealing small size, we also provide throughput measurements on a number of edge devices. In addition, we open source several quantized checkpoints and provide their performance metrics compared to the original model.

We introduce StableLM 2 1.6B, the first in a new generation of our language model series. In this technical report, we present in detail the data and training procedure leading to the base and instruction-tuned versions of StableLM 2 1.6B. The weights for both models are available via Hugging Face for anyone to download and use. The report contains thorough evaluations of these models, including zero- and few-shot benchmarks, multilingual benchmarks, and the MT benchmark focusing on multi-turn dialogues. At the time of publishing this report, StableLM 2 1.6B was the state-of-the-art open model under 2B parameters by a significant margin. Given its appealing small size, we also provide throughput measurements on a number of edge devices. In addition, we open source several quantized checkpoints and provide their performance metrics compared to the original model., Comment: 23 pages, 6 figures

Mixture-of-Experts (MoE) models are a promising way to scale up model capacity without significantly increasing computational cost. A key component of MoEs is the router, which decides which subset of parameters (experts) process which feature embeddings (tokens). In this paper, we present a comprehensive study of routers in MoEs for computer vision tasks. We introduce a unified MoE formulation that subsumes different MoEs with two parametric routing tensors. This formulation covers both sparse MoE, which uses a binary or hard assignment between experts and tokens, and soft MoE, which uses a soft assignment between experts and weighted combinations of tokens. Routers for sparse MoEs can be further grouped into two variants: Token Choice, which matches experts to each token, and Expert Choice, which matches tokens to each expert. We conduct head-to-head experiments with 6 different routers, including existing routers from prior work and new ones we introduce. We show that (i) many routers originally developed for language modeling can be adapted to perform strongly in vision tasks, (ii) in sparse MoE, Expert Choice routers generally outperform Token Choice routers, and (iii) soft MoEs generally outperform sparse MoEs with a fixed compute budget. These results provide new insights regarding the crucial role of routers in vision MoE models.

Tight relationships exist in the local universe between the central stellar properties of galaxies and the mass of their supermassive black hole. These suggest galaxies and black holes co-evolve, with the main regulation mechanism being energetic feedback from accretion onto the black hole during its quasar phase. A crucial question is how the relationship between black holes and galaxies evolves with time; a key epoch to probe this relationship is at the peaks of star formation and black hole growth 8-12 billion years ago (redshifts 1-3). Here we report a dynamical measurement of the mass of the black hole in a luminous quasar at a redshift of 2, with a look back time of 11 billion years, by spatially resolving the broad line region. We detect a 40 micro-arcsecond (0.31 pc) spatial offset between the red and blue photocenters of the H$\alpha$ line that traces the velocity gradient of a rotating broad line region. The flux and differential phase spectra are well reproduced by a thick, moderately inclined disk of gas clouds within the sphere of influence of a central black hole with a mass of 3.2x10$^{8}$ solar masses. Molecular gas data reveal a dynamical mass for the host galaxy of 6x10$^{11}$ solar masses, which indicates an under-massive black hole accreting at a super-Eddington rate. This suggests a host galaxy that grew faster than the supermassive black hole, indicating a delay between galaxy and black hole formation for some systems., Comment: 5 pages Main text, 8 figures, 2 tables, to be published in Nature, under embargo until 29 January 2024 16:00 (London)

Background: Pain assessment in individuals with neurological conditions, especially those with limited self-report ability and altered facial expressions, presents challenges. Existing measures, relying on direct observation by caregivers, lack sensitivity and specificity. In cerebral palsy, pain is a common comorbidity and a reliable evaluation protocol is crucial. Thus, having an automatic system that recognizes facial expressions could be of enormous help when diagnosing pain in this type of patient. Objectives: 1) to build a dataset of facial pain expressions in individuals with cerebral palsy, and 2) to develop an automated facial recognition system based on deep learning for pain assessment addressed to this population. Methods: Ten neural networks were trained on three pain image databases, including the UNBC-McMaster Shoulder Pain Expression Archive Database, the Multimodal Intensity Pain Dataset, and the Delaware Pain Database. Additionally, a curated dataset (CPPAIN) was created, consisting of 109 preprocessed facial pain expression images from individuals with cerebral palsy, categorized by two physiotherapists using the Facial Action Coding System observational scale. Results: InceptionV3 exhibited promising performance on the CP-PAIN dataset, achieving an accuracy of 62.67% and an F1 score of 61.12%. Explainable artificial intelligence techniques revealed consistent essential features for pain identification across models. Conclusion: This study demonstrates the potential of deep learning models for robust pain detection in populations with neurological conditions and communication disabilities. The creation of a larger dataset specific to cerebral palsy would further enhance model accuracy, offering a valuable tool for discerning subtle and idiosyncratic pain expressions. The insights gained could extend to other complex neurological conditions.

Dye Sensitized Solar Cells (DSSCs) have recently regained attention for indoor light harvesting and powering wireless devices. Advantages such as cost-effectiveness, flexibility, wide angular response, and lightweight design have driven the fostering of the implementation of advanced photonic architectures, dedicated photosensitizers and compatibility with wearable carriers. However, to fully exploit their potential, crucial aspects require further attention, in particular the improvement of spectral compatibility and low-light harvesting mechanisms, as well as the development of efficient photoanodes through high-yield scalable methods. In this article, we propose the use of nanocomposite photoanodes integrating mesoporous TiO2 (m-TiO2) nanoparticles, ITO nanotubes (NTs) and TiO2 anatase shells (ITO@TiO2 NTs) prepared by step-by-step method relying on mild temperature conditions and avoiding toxic precursors. These photoanodes outperform previous attempts to implement low-dimensional ITO and ITO@TiO2 nanowires and nanotubes for outdoor light conversion, demonstrating an outstanding PCE under low artificial light intensity of 24 % for at 0.014 mWcm-2, a 166 % increase compared to the conventional architectures. Advanced microstructural, optical, and electrochemical characterizations have revealed that the strong scattering effect of the light in the visible range coupled with enhanced charge collection at low-intensity illumination are the essential mechanisms responsible for such enhanced energy conversion. Remarkably, our devices retain up to 90% of the normal incidence efficiency even under glancing illumination, while conventional reference devices show a drop down to 50%.

Multivariate spatial modeling is key to understanding the behavior of materials downstream in a mining operation. The ore recovery depends on the mineralogical composition, which needs to be properly captured by the model to allow for good predictions. Multivariate modeling must also capture the behavior of tailings and waste materials to understand the environmental risks involved in their disposal. However, multivariate spatial modeling is challenging when the variables show complex relationships, such as non-linear correlation, heteroscedastic behavior, or spatial trends. This contribution proposes a novel methodology for general multivariate contexts, with the idea of disaggregating the global non-linear behavior among variables into the spatial domain in a piece-wise linear fashion. We demonstrate that the complex multivariate behavior can be reproduced by looking at local correlations between variables at sample locations, inferred from a local neighborhood, and interpolating these local linear dependencies by using a non-stationary version of the Linear Model of Coregionalization. This mixture of locally varying linear correlations is combined to reproduce the global complex behavior seen in the multivariate distribution. The main challenge is to solve appropriately the interpolation of the known correlation matrices over the domain, as these local correlations defined at sample locations can be endowed with a manifold structure, on which the Euclidean distance is not a suitable metric for interpolation of such correlations. This is addressed by using tools from Riemannian geometry: correlation matrices are interpolated using a weighted Fr\'echet mean of the correlations inferred at sample locations. An application of the procedure is shown in a real case study with good results in terms of accuracy and reproduction of the reference multivariate distributions and semi-variograms.