Search

Deep neural networks (DNNs) struggle at systematic generalization (SG). Several studies have evaluated the possibility to promote SG through the proposal of novel architectures, loss functions or training methodologies. Few studies, however, have focused on the role of training data properties in promoting SG. In this work, we investigate the impact of certain data distributional properties, as inductive biases for the SG ability of a multi-modal language model. To this end, we study three different properties. First, data diversity, instantiated as an increase in the possible values a latent property in the training distribution may take. Second, burstiness, where we probabilistically restrict the number of possible values of latent factors on particular inputs during training. Third, latent intervention, where a particular latent factor is altered randomly during training. We find that all three factors significantly enhance SG, with diversity contributing an 89\% absolute increase in accuracy in the most affected property. Through a series of experiments, we test various hypotheses to understand why these properties promote SG. Finally, we find that Normalized Mutual Information (NMI) between latent attributes in the training distribution is strongly predictive of out-of-distribution generalization. We find that a mechanism by which lower NMI induces SG is in the geometry of representations. In particular, we find that NMI induces more parallelism in neural representations (i.e., input features coded in parallel neural vectors) of the model, a property related to the capacity of reasoning by analogy.

The Mixture of Experts (MoE) architecture reduces the training and inference cost significantly compared to a dense model of equivalent capacity. Upcycling is an approach that initializes and trains an MoE model using a pre-trained dense model. While upcycling leads to initial performance gains, the training progresses slower than when trained from scratch, leading to suboptimal performance in the long term. We propose Drop-Upcycling - a method that effectively addresses this problem. Drop-Upcycling combines two seemingly contradictory approaches: utilizing the knowledge of pre-trained dense models while statistically re-initializing some parts of the weights. This approach strategically promotes expert specialization, significantly enhancing the MoE model's efficiency in knowledge acquisition. Extensive large-scale experiments demonstrate that Drop-Upcycling significantly outperforms previous MoE construction methods in the long term, specifically when training on hundreds of billions of tokens or more. As a result, our MoE model with 5.9B active parameters achieves comparable performance to a 13B dense model in the same model family, while requiring approximately 1/4 of the training FLOPs. All experimental resources, including source code, training data, model checkpoints and logs, are publicly available to promote reproducibility and future research on MoE., Comment: To appear at the 13th International Conference on Learning Representations (ICLR 2025)

The study of collective excitations is a crucial tool for understanding many-body quantum systems. For instance, they play a central role in the exploration of superfluidity and other quantum macroscopic phenomena in Bose and Fermi systems. In this work we present a variational and a numerical study of a parametrically driven Bose-Einstein condensate confined in a cylindrical harmonic trap in which the aspect ratio can be varied from a prolate (cigar-shaped) to an oblate (pancake-shaped) system. The excitation can be applied by periodically modulating the harmonic frequencies of the trap or, alternatively, the interatomic interaction strength at a frequency that matches that of the system breathing mode. As a result, we observe the formation of dynamical density patterns that depend on the geometry of the trap: a fringe pattern in a prolate system and a ring pattern in an oblate one. By decomposing the total energy into its kinetic, potential, and interaction terms, we show that the onset of these patterns coincides with the redistribution of kinetic energy along the weakly trapped directions of the sample, indicating the three-dimensional nature of the studied phenomena. Finally, our analysis shows that the difference between the two excitation mechanisms lies on the system stability. Modulating the trap destabilizes the system quicker than modulating the interactions, leading to earlier formation of the patterns.

This paper presents a hierarchical framework for demand response optimization in air separation units (ASUs) that combines reinforcement learning (RL) with linear model predictive control (LMPC). We investigate two control architectures: a direct RL approach and a control-informed methodology where an RL agent provides setpoints to a lower-level LMPC. The proposed RL-LMPC framework demonstrates improved sample efficiency during training and better constraint satisfaction compared to direct RL control. Using an industrial ASU case study, we show that our approach successfully manages operational constraints while optimizing electricity costs under time-varying pricing. Results indicate that the RL-LMPC architecture achieves comparable economic performance to direct RL while providing better robustness and requiring fewer training samples to converge. The framework offers a practical solution for implementing flexible operation strategies in process industries, bridging the gap between data-driven methods and traditional control approaches.

Assuming that the two integrals in the Change of Variable Formula for the unidimensional Riemann integral are finite, one can ask if they have equal value. We give a positive answer to this question. The proof is very easy to follow and to keep in mind. An example is given., Comment: 15 pages, 3 figures

In this paper, we propose a novel Mixed-Integer Non-Linear Optimization formulation to construct a risk score, where we optimize the logistic loss with sparsity constraints. Previous approaches are typically designed to handle binary datasets, where continuous predictor variables are discretized in a preprocessing step by using arbitrary thresholds, such as quantiles. In contrast, we allow the model to decide for each continuous predictor variable the particular threshold that is critical for prediction. The usefulness of the resulting optimization problem is tested in synthetic datasets.

We present the one-loop matching coefficients necessary to match all of the leading-twist generalised transverse-momentum-dependent distributions (GTMDs) onto generalised parton distributions (GPDs). Matching functions are extracted by computing the first radiative corrections to partonic bilocal correlators with staple-like Wilson lines, as appropriate for high-energy collisions. These correlators are characterised by a transverse displacement and skewed kinematics of external states. Using the proton helicity basis, they are parametrised in terms of GTMDs, which are subsequently related to leading-twist GPDs. Our results provide new insights into the complex dynamics of GTMDs generated by radiative corrections. In particular, we show that time-reversal even and odd contributions to GTMDs in the so-called ERBL region mix both under matching and evolution. Finally, we present a selection of numerical results and comment on the quantitative behaviour of GTMDs., Comment: 27 pages, 3 figures

The goal of the TANGERINE project is to develop the next generation of monolithic silicon pixel detectors using a 65 nm CMOS imaging process, which offers a higher logic density and overall lower power consumption compared to previously used processes. A combination of Technology Computer-Aided Design (TCAD) and Monte Carlo (MC) simulations is used to understand the physical processes within the sensing element and thus the overall performance of the pixel detector. The response of the sensors can then be tested in laboratory and test beam facilities and compared to simulation results. Transient simulations allow for studying the response of the sensor as a function of time, such as the signal produced after a charged particle passes through the sensor. The study of these signals is important to understand the magnitude and timing of the response from the sensors and improve upon them. While TCAD simulations are accurate, the time required to produce a single pulse is large compared to a combination of MC and TCAD simulations. In this work, a validation of the transient simulation approach and studies on charge collection are presented., Comment: TIPP 2023 Proceedings

The high energy physics community recently gained access to the TPSCo 65 nm ISC (Image Sensor CMOS), which enables a higher in-pixel logic density in monolithic active pixel sensors (MAPS) compared to processes with larger feature sizes. To explore this novel technology, the Hybrid-to-Monolithic (H2M) test chip has been designed and manufactured. The design followed a digital-on-top design workflow and ports a hybrid pixel-detector architecture, with digital pulse processing in each pixel, into a monolithic chip. The chip matrix consists of 64$\times$16 square pixels with a size of 35$\times$35 um2, and a total active area of approximately 1.25 um2. The chip has been successfully integrated into the Caribou DAQ system. It is fully functional, and the measured threshold dispersion and noise agree with the expectation from front-end simulations. However, a non-uniform in-pixel response related to the size and location of the n-wells in the analog circuitry has been observed in test beam measurements and will be discussed in this contribution. This asymmetry in the pixel response, enhanced by the 35 um pixel pitch - larger than in other prototypes - and certain features of the readout circuit, has not been observed in prototypes with smaller pixel pitches in this technology.

Hierarchical low-rank approximation of dense matrices can reduce the complexity of their factorization from O(N^3) to O(N). However, the complex structure of such hierarchical matrices makes them difficult to parallelize. The block size and ranks can vary between the sub-blocks, which creates load imbalance. The dependency between the sub-blocks during factorization results in serialization. Since many sub-blocks are low-rank, their small computational load exposes the overhead of runtime systems. The combination of these factors makes it challenging to implement these methods on GPUs. In this work, we show that dense matrices can be factorized with linear complexity, while extracting the potential parallelism of GPUs. This is made possible through the H2-ULV factorization, which removes the dependency on trailing sub-matrices.

Motivated by the increasing deployment of reinforcement learning in the real world, involving a large consumption of personal data, we introduce a differentially private (DP) policy gradient algorithm. We show that, in this setting, the introduction of Differential Privacy can be reduced to the computation of appropriate trust regions, thus avoiding the sacrifice of theoretical properties of the DP-less methods. Therefore, we show that it is possible to find the right trade-off between privacy noise and trust-region size to obtain a performant differentially private policy gradient algorithm. We then outline its performance empirically on various benchmarks. Our results and the complexity of the tasks addressed represent a significant improvement over existing DP algorithms in online RL.

We investigate the collinear matching of transverse momentum dependent (TMD) distributions at large values of $x$, computing and resumming the leading large-$x$ asymptotics for matching coefficients. The large-$x$ resummation is done directly within TMD distributions, ensuring the process-independence of the result. The derived resummation formulas are valid for all TMD distributions (except the pretzelosity). Their application improves perturbative convergence, provides practical estimation for unknown higher-order contributions, and sets restrictions for the nonperturbative part of models. Using the known anomalous dimensions, resummation can reach N$^3$LL, often exceeding the accuracy of known coefficient functions., Comment: 29 pages, 3 figures

The main challenges limiting the adoption of deep learning-based solutions in medical workflows are the availability of annotated data and the lack of interpretability of such systems. Concept Bottleneck Models (CBMs) tackle the latter by constraining the final disease prediction on a set of predefined and human-interpretable concepts. However, the increased interpretability achieved through these concept-based explanations implies a higher annotation burden. Moreover, if a new concept needs to be added, the whole system needs to be retrained. Inspired by the remarkable performance shown by Large Vision-Language Models (LVLMs) in few-shot settings, we propose a simple, yet effective, methodology, CBVLM, which tackles both of the aforementioned challenges. First, for each concept, we prompt the LVLM to answer if the concept is present in the input image. Then, we ask the LVLM to classify the image based on the previous concept predictions. Moreover, in both stages, we incorporate a retrieval module responsible for selecting the best examples for in-context learning. By grounding the final diagnosis on the predicted concepts, we ensure explainability, and by leveraging the few-shot capabilities of LVLMs, we drastically lower the annotation cost. We validate our approach with extensive experiments across four medical datasets and twelve LVLMs (both generic and medical) and show that CBVLM consistently outperforms CBMs and task-specific supervised methods without requiring any training and using just a few annotated examples. More information on our project page: https://cristianopatricio.github.io/CBVLM/., Comment: This work has been submitted to the IEEE for possible publication

Precessing binary black-hole mergers can produce a net flux of circularly-polarized gravitational waves. This imbalance between left- and right-handed circularly polarized waves, quantified via the Stokes pseudo-scalar $V_{\rm GW}$, originated from mirror asymmetries in the binary. We scan the parameter space of black-hole mergers to investigate correlations between $V_{\rm GW}$ and chiral magnitudes constructed out of the intrinsic parameters of the binary. To this end, we use both numerical-relativity simulations for (quasi-circular) and eccentric precessing mergers from both the SXS and RIT catalogues, as well as the state-of-the-art surrogate model for quasi-circular precessing mergers NRSur7dq4. We find that, despite being computed by manifestly different formulas, $V_{\rm GW}$ is linearly correlated to the helicity of the final black hole, defined as the projection of its recoil velocity onto its spin. Next, we test our ability to perform accurate measurements of $V_{\rm GW}$ in gravitational-wave observations through the injection and recovery of numerically simulated signals. We show that $V_{\rm GW}$ can be estimated unbiasedly using the surrogate waveform model NRSur7dq4 even for signal-to-noise ratios of nearly 50, way beyond current gravitational-wave observations.

Foundation models require fine-tuning to ensure their generative outputs align with intended results for specific tasks. Automating this fine-tuning process is challenging, as it typically needs human feedback that can be expensive to acquire. We present AutoRefine, a method that leverages reinforcement learning for targeted fine-tuning, utilizing direct feedback from measurable performance improvements in specific downstream tasks. We demonstrate the method for a problem arising in algorithmic hiring platforms where linguistic biases influence a recommendation system. In this setting, a generative model seeks to rewrite given job specifications to receive more diverse candidate matches from a recommendation engine which matches jobs to candidates. Our model detects and regulates biases in job descriptions to meet diversity and fairness criteria. The experiments on a public hiring dataset and a real-world hiring platform showcase how large language models can assist in identifying and mitigation biases in the real world., Comment: Accepted to AAAI 2025, AI Governance Workshop

The conefield variant of a Converse KAM method for 3D vector fields, identifying regions through which no invariant 2-tori pass transverse to a specified direction field, is tested on some helical perturbations of an axisymmetric magnetic field in toroidal geometry. This implementation computes bounds on the slopes of invariant tori of a given class and allows to apply a subsidiary criterion for the extension of the non-existence region, saving significant computation time. The method finds regions corresponding to magnetic islands and chaos for the fieldline flow., Comment: 18 pages, 11 figures

K-means is a popular clustering algorithm with significant applications in numerous scientific and engineering areas. One drawback of K-means is its inability to identify non-linearly separable clusters, which may lead to inaccurate solutions in certain cases. Kernel K-means is a variant of classical K-means that can find non-linearly separable clusters. However, it scales quadratically with respect to the size of the dataset, taking several minutes to cluster even medium-sized datasets on traditional CPU-based machines. In this paper, we present a formulation of Kernel K-means using sparse-dense matrix multiplication (SpMM) and sparse matrix-vector multiplication (SpMV), and we show that our formulation enables the rapid implementation of a fast GPU-based version of Kernel K-means with little programming effort. Our implementation, named Popcorn, is the first open-source GPU-based implementation of Kernel K-means. Popcorn achieves a speedup of up to 123.8x over a CPU implementation of Kernel K-means and a speedup of up to 2.6x over a GPU implementation of Kernel K-means that does not use sparse matrix computations. Our results support the effectiveness of sparse matrices as tools for efficient parallel programming.

We present a study of atmospheric disturbances at Jezero Crater, Mars, using ground-based measurements of surface pressure by the Perseverance rover in combination with orbital images from the Mars Express and Mars Reconnaissance Orbiter missions. The study starts at Ls $\sim$ 13.3{\deg} in MY36 (March 6th, 2021) and extends up to Ls $\sim$ 30.3{\deg} in MY37 (February 28th, 2023). We focus on the characterization of the major atmospheric phenomena at synoptic and planetary-scales. These are the thermal tides (measured up to the sixth component), long-period pressure oscillations (periods > 1 sol), the Aphelion Cloud Belt, and the occasional development of regional dust storms over Jezero. We present the seasonal evolution of the amplitudes and phases of the thermal tides and their relation with the atmospheric dust content (optical depth). Three regional dust storms and one polar storm extending over Jezero produced an increase in the diurnal and semidiurnal amplitudes but resulted in inverse responses in their phases. We show that the primary regular wave activity is due to baroclinic disturbances with periods of 2-4 sols and amplitudes $\sim$ 1-15 Pa increasing with dust content, in good agreement with theoretical predictions by model calculations. The spacecraft images show a number of arc-shaped, spiral and irregular cyclonic vortices, traced by dust and clouds at the edge of the North Polar Cap, that could be behind some of the pressure oscillations measured at Jezero.

We revise the role that ultraviolet divergences of quantum fields play in slow-roll inflation, and discuss the renormalization of cosmological observables from a space-time perspective, namely the angular power spectrum. We first derive an explicit expression for the multipole coefficients $C_{\ell}$ in the Sachs-Wolfe regime in terms of the two-point function of primordial perturbations. We then analyze the ultraviolet behavior, and point out that the standard result in the literature is equivalent to a renormalization of $C_{\ell}$ at zero ``adiabatic'' order. We further argue that renormalization at second ``adiabatic'' order may be more appropriate from the viewpoint of standard quantum field theory. This may change significantly the predictions for $C_{\ell}$, while maintaining scale invariance., Comment: Proceedings of the "Spanish Relativity Meeting (ERE 2014): almost 100 years after Einstein's revolution"

In this work, we applied the Chaos Game Representation (CGR) to the complete human genomic sequence T2T-CHM13v2.0, analyzing the entire chromosome assembly and each chromosome separately, including mitochondrial DNA. Multifractal spectra were determined using two types of box-counting coverage, revealing slight variations across most chromosomes. While the geometric support remained consistent, distinct distributions were observed for each chromosome. Chromosomes 9 and Y exhibited the greatest differences in singularity (H\"older exponent), with minor variations in their fractal support. The CGR distributions generally demonstrated an approximate separation between coding and non-coding sections, as well as CpG or GpC islands. A base-by-base analysis of the fractal support of the CGR uncovered characteristic structural bands in chromosome sequences, which align with patterns identified in cytogenetic studies. Using the complete assembly as a reference, we compared two alternative representations: the Binary Genomic Representation (RGB) and the Markov Chain (MC) representation. Both methods tended toward the same fractal support but displayed differing distributions based on the assigned length parameter. Multifractal analysis highlighted quantitative differences between these representations: RGB aligned more closely with high-frequency components, while MC showed better correspondence with low frequencies. The optimal fit was achieved using MC for twelve-base chains, yielding an average percentage error of 2% relative to the full genomic assembly.

Economic agent-based models (ABMs) are becoming more and more data-driven, establishing themselves as increasingly valuable tools for economic research and policymaking. We propose to classify the extent to which an ABM is data-driven based on whether agent-level quantities are initialized from real-world micro-data and whether the ABM's dynamics track empirical time series. This paper discusses how making ABMs data-driven helps overcome limitations of traditional ABMs and makes ABMs a stronger alternative to equilibrium models. We review state-of-the-art methods in parameter calibration, initialization, and data assimilation, and then present successful applications that have generated new scientific knowledge and informed policy decisions. This paper serves as a manifesto for data-driven ABMs, introducing a definition and classification and outlining the state of the field, and as a guide for those new to the field.

Why do we build local large language models (LLMs)? What should a local LLM learn from the target language? Which abilities can be transferred from other languages? Do language-specific scaling laws exist? To explore these research questions, we evaluated 35 Japanese, English, and multilingual LLMs on 19 evaluation benchmarks for Japanese and English, taking Japanese as a local language. Adopting an observational approach, we analyzed correlations of benchmark scores, and conducted principal component analysis (PCA) on the scores to derive \textit{ability factors} of local LLMs. We found that training on English text can improve the scores of academic subjects in Japanese (JMMLU). In addition, it is unnecessary to specifically train on Japanese text to enhance abilities for solving Japanese code generation, arithmetic reasoning, commonsense, and reading comprehension tasks. In contrast, training on Japanese text could improve question-answering tasks about Japanese knowledge and English-Japanese translation, which indicates that abilities for solving these two tasks can be regarded as \textit{Japanese abilities} for LLMs. Furthermore, we confirmed that the Japanese abilities scale with the computational budget for Japanese text., Comment: Preprint. Under review

Optimization plays an important role in chemical engineering, impacting cost-effectiveness, resource utilization, product quality, and process sustainability metrics. This chapter broadly focuses on data-driven optimization, particularly, on model-based derivative-free techniques, also known as surrogate-based optimization. The chapter introduces readers to the theory and practical considerations of various algorithms, complemented by a performance assessment across multiple dimensions, test functions, and two chemical engineering case studies: a stochastic high-dimensional reactor control study and a low-dimensional constrained stochastic reactor optimization study. This assessment sheds light on each algorithm's performance and suitability for diverse applications. Additionally, each algorithm is accompanied by background information, mathematical foundations, and algorithm descriptions. Among the discussed algorithms are Bayesian Optimization (BO), including state-of-the-art TuRBO, Constrained Optimization by Linear Approximation (COBYLA), the Ensemble Tree Model Optimization Tool (ENTMOOT) which uses decision trees as surrogates, Stable Noisy Optimization by Branch and Fit (SNOBFIT), methods that use radial basis functions such as DYCORS and SRBFStrategy, Constrained Optimization by Quadratic Approximations (COBYQA), as well as a few others recognized for their effectiveness in surrogate-based optimization. By combining theory with practice, this chapter equips readers with the knowledge to integrate surrogate-based optimization techniques into chemical engineering. The overarching aim is to highlight the advantages of surrogate-based optimization, introduce state-of-the-art algorithms, and provide guidance for successful implementation within process systems engineering.

Acquiring a comprehensive understanding of the interplay between foam friction and surface roughness is essential for achieving precise control over their flow dynamics. In particular, a major challenge is to reduce friction, which can be achieved with rough surfaces in the situation where a liquid infuses the asperities. In this study, we propose to explore self-infused surfaces. We first present simple observations to demonstrate the effectiveness of our surface design by recording the motion of a foam puddle on a smooth surface and a self-SLIPS. To quantify friction reduction, we conduct stress measurements on surfaces moved at a constant velocity. Finally, we interpret the variation of the friction force with the velocity by a model considering an effective slip length of the surface. This research paves the way for a novel approach to mitigate dissipation in liquid foam flows, holding significant implications for reducing energy consumption in conveying foams for industrial processes and various end-use applications.

Word error rate (WER) as a metric has a variety of limitations that have plagued the field of speech recognition. Evaluation datasets suffer from varying style, formality, and inherent ambiguity of the transcription task. In this work, we attempt to mitigate some of these differences by performing style-agnostic evaluation of ASR systems using multiple references transcribed under opposing style parameters. As a result, we find that existing WER reports are likely significantly over-estimating the number of contentful errors made by state-of-the-art ASR systems. In addition, we have found our multireference method to be a useful mechanism for comparing the quality of ASR models that differ in the stylistic makeup of their training data and target task.

In this article we briefly review the adiabatic renormalization program for spin 1/2 fields in expanding universes. We introduce the method and provide explicit expressions for the renormalized vacuum expectation value of the stress-energy tensor. Then, we discuss its application to some cosmological scenario of physical interest. We end up sketching out the proof that adiabatic and DeWitt-Schwinger point-splitting schemes provide the same renormalized expectation values of the stress-energy tensor for Dirac fields., Comment: Proceedings of the 14th Marcel Grossmann Meeting, 2816-2821 (2017)

We introduce and study some families of groups whose irreducible characters take values on quadratic extensions of the rationals. We focus mostly on a generalization of inverse semi-rational groups, which we call uniformly semi-rational groups. Moreover, we associate to every finite group two invariants, called rationality and semi-rationality of the group. They measure respectively how far a group is from being rational and how much uniformly rational it is. We determine the possible values that these invariants may take for finite nilpotent groups. We also classify the fields that can occur as the field generated by the character values of a finite nilpotent group., Comment: 17 pages, 1 figure, 1 table

Communication overhead is a key challenge in distributed deep learning, especially on slower Ethernet interconnects, and given current hardware trends, communication is likely to become a major bottleneck. While gradient compression techniques have been explored for SGD and Adam, the Lion optimizer has the distinct advantage that its update vectors are the output of a sign operation, enabling straightforward quantization. However, simply compressing updates for communication and using techniques like majority voting fails to lead to end-to-end speedups due to inefficient communication algorithms and reduced convergence. We analyze three factors critical to distributed learning with Lion: optimizing communication methods, identifying effective quantization methods, and assessing the necessity of momentum synchronization. Our findings show that quantization techniques adapted to Lion and selective momentum synchronization can significantly reduce communication costs while maintaining convergence. We combine these into Lion Cub, which enables up to 5x speedups in end-to-end training compared to Lion. This highlights Lion's potential as a communication-efficient solution for distributed training.

Graph Convolutional Networks (GCNs) are widely used in various domains. However, training distributed full-batch GCNs on large-scale graphs poses challenges due to inefficient memory access patterns and high communication overhead. This paper presents general and efficient aggregation operators designed for irregular memory access patterns. Additionally, we propose a pre-post-aggregation approach and a quantization with label propagation method to reduce communication costs. Combining these techniques, we develop an efficient and scalable distributed GCN training framework, \emph{SuperGCN}, for CPU-powered supercomputers. Experimental results on multiple large graph datasets show that our method achieves a speedup of up to 6$\times$ compared with the SoTA implementations, and scales to 1000s of HPC-grade CPUs, without sacrificing model convergence and accuracy. Our framework achieves performance on CPU-powered supercomputers comparable to that of GPU-powered supercomputers, with a fraction of the cost and power budget.

It is known that an electric-magnetic duality transformation is a symmetry of the classical source-free Maxwell theory in generic spacetimes. This provides a conserved Noether charge, physically related to the polarization state of the electromagnetic field. We argue that this conservation law fails to hold at the quantum level in presence of a background classical gravitational field with non-trivial dynamics, as determined by the Chern-Pontryagin scalar. This is the spin 1 analog of the chiral anomaly for massless Dirac fermions., Comment: Proceedings of the 15th Marcel Grossmann Meeting, 1461-1466 (2022)

We analyze the semiclassical Schwarzschild geometry in the Boulware quantum state in the framework of two-dimensional (2D) dilaton gravity. The classical model is defined by the spherical reduction of Einstein's gravity sourced with conformal scalar fields. The expectation value of the stress-energy tensor in the Boulware state is singular at the classical horizon of the Schwarzschild spacetime, but when backreaction effects are considered, previous results have shown that the 2D geometry is horizonless and described by a non-symmetric wormhole with a curvature singularity on the other side of the throat. In this work we show that reversing the sign of the central charge of the conformal matter removes the curvature singularity of the 2D backreacted geometry, which happens to be horizonless and asymptotically flat. This result is consistent with a similar analysis recently performed for the CGHS model. We also argue the physical significance of negative central charges in conformal anomalies from a four-dimensional perspective.

Efficient multiple setpoint tracking can enable advanced biotechnological applications, such as maintaining desired population levels in co-cultures for optimal metabolic division of labor. In this study, we employ reinforcement learning as a control method for population setpoint tracking in co-cultures, focusing on policy-gradient techniques where the control policy is parameterized by neural networks. However, achieving accurate tracking across multiple setpoints is a significant challenge in reinforcement learning, as the agent must effectively balance the contributions of various setpoints to maximize the expected system performance. Traditional return functions, such as those based on a quadratic cost, often yield suboptimal performance due to their inability to efficiently guide the agent toward the simultaneous satisfaction of all setpoints. To overcome this, we propose a novel return function that rewards the simultaneous satisfaction of multiple setpoints and diminishes overall reward gains otherwise, accounting for both stage and terminal system performance. This return function includes parameters to fine-tune the desired smoothness and steepness of the learning process. We demonstrate our approach considering an $\textit{Escherichia coli}$ co-culture in a chemostat with optogenetic control over amino acid synthesis pathways, leveraging auxotrophies to modulate growth.

Large Language Models (LLMs) have attracted significant attention due to their human-like language understanding and generation capabilities, as well as their applicability across various domains. These models, characterized by their massive scale and extensive training data, continue to push the boundaries of what is possible in natural language processing. The Llama 3 series, for instance, exemplifies this trend with its flagship model boasting 405 billion parameters trained on 15.6 trillion tokens. The immense computational demands associated with training such models have spurred ongoing research into optimizing the efficiency of the training process, particularly through the use of lower-precision formats. NVIDIA's H100 GPU, which introduces support for FP8 in addition to the more conventional FP16 and BF16 formats, has emerged as a focal point in this optimization effort. Preliminary studies suggest that FP8 could offer substantial reductions in training time without sacrificing model performance when compared to BF16, making it a promising candidate for large-scale model training. However, the broader implications of adopting FP8, particularly in terms of training stability and downstream task performance, have yet to be fully understood. In this study, we delve into the practical trade-offs involved in adopting FP8 over BF16 for training LLMs., Comment: 2 pages,extended abstract

In large language model (LLM) training, several parallelization strategies, including Tensor Parallelism (TP), Pipeline Parallelism (PP), Data Parallelism (DP), as well as Sequence Parallelism (SP) and Context Parallelism (CP), are employed to distribute model parameters, activations, and optimizer states across devices. Identifying the optimal parallelization configuration for each environment while avoiding GPU memory overflow remains a challenging task. In this study, we provide precise formulas to estimate the memory consumed by parameters, gradients, optimizer states, and activations for 4D parallel training (DP, TP, PP, CP) in the Llama architecture. We conducted 454 experiments on A100 and H100 GPUs, incorporating often neglected factors such as temporary buffers and memory fragmentation into our analysis. Results indicate that when the estimated memory usage is below 80\% of the available GPU memory, the training never encounters out-of-memory errors. This simple yet effective formula allows us to identify parallelization configurations that could lead to memory overflow in advance, significantly reducing the configuration search space. Additionally, through a comprehensive exploration of optimal configurations in 4D parallelism, our analysis of the 454 experimental results provides empirical insights into optimal 4D parallelism configurations.

We show that variational learning can significantly improve the accuracy and calibration of Low-Rank Adaptation (LoRA) without a substantial increase in the cost. We replace AdamW by the Improved Variational Online Newton (IVON) algorithm to finetune large language models. For Llama-2 with 7 billion parameters, IVON improves the accuracy over AdamW by 2.8% and expected calibration error by 4.6%. The accuracy is also better than the other Bayesian alternatives, yet the cost is lower and the implementation is easier. Our work provides additional evidence for the effectiveness of IVON for large language models. The code is available at https://github.com/team-approx-bayes/ivon-lora., Comment: Published at 38th Workshop on Fine-Tuning in Machine Learning (NeurIPS 2024). Code available at https://github.com/team-approx-bayes/ivon-lora. In version 2 we fixed a typo in the equation of prior in section 2

The Privacy Preserving Federated Learning Document VQA (PFL-DocVQA) competition challenged the community to develop provably private and communication-efficient solutions in a federated setting for a real-life use case: invoice processing. The competition introduced a dataset of real invoice documents, along with associated questions and answers requiring information extraction and reasoning over the document images. Thereby, it brings together researchers and expertise from the document analysis, privacy, and federated learning communities. Participants fine-tuned a pre-trained, state-of-the-art Document Visual Question Answering model provided by the organizers for this new domain, mimicking a typical federated invoice processing setup. The base model is a multi-modal generative language model, and sensitive information could be exposed through either the visual or textual input modality. Participants proposed elegant solutions to reduce communication costs while maintaining a minimum utility threshold in track 1 and to protect all information from each document provider using differential privacy in track 2. The competition served as a new testbed for developing and testing private federated learning methods, simultaneously raising awareness about privacy within the document image analysis and recognition community. Ultimately, the competition analysis provides best practices and recommendations for successfully running privacy-focused federated learning challenges in the future., Comment: 27 pages, 6 figures

Local learning, which trains a network through layer-wise local targets and losses, has been studied as an alternative to backpropagation (BP) in neural computation. However, its algorithms often become more complex or require additional hyperparameters because of the locality, making it challenging to identify desirable settings in which the algorithm progresses in a stable manner. To provide theoretical and quantitative insights, we introduce the maximal update parameterization ($\mu$P) in the infinite-width limit for two representative designs of local targets: predictive coding (PC) and target propagation (TP). We verified that $\mu$P enables hyperparameter transfer across models of different widths. Furthermore, our analysis revealed unique and intriguing properties of $\mu$P that are not present in conventional BP. By analyzing deep linear networks, we found that PC's gradients interpolate between first-order and Gauss-Newton-like gradients, depending on the parameterization. We demonstrate that, in specific standard settings, PC in the infinite-width limit behaves more similarly to the first-order gradient. For TP, even with the standard scaling of the last layer, which differs from classical $\mu$P, its local loss optimization favors the feature learning regime over the kernel regime.

PC-Gym is an open-source tool for developing and evaluating reinforcement learning (RL) algorithms in chemical process control. It features environments that simulate various chemical processes, incorporating nonlinear dynamics, disturbances, and constraints. The tool includes customizable constraint handling, disturbance generation, reward function design, and enables comparison of RL algorithms against Nonlinear Model Predictive Control (NMPC) across different scenarios. Case studies demonstrate the framework's effectiveness in evaluating RL approaches for systems like continuously stirred tank reactors, multistage extraction processes, and crystallization reactors. The results reveal performance gaps between RL algorithms and NMPC oracles, highlighting areas for improvement and enabling benchmarking. By providing a standardized platform, PC-Gym aims to accelerate research at the intersection of machine learning, control, and process systems engineering. By connecting theoretical RL advances with practical industrial process control applications, offering researchers a tool for exploring data-driven control solutions.

Microkinetic models are key for evaluating industrial processes' efficiency and chemicals' environmental impact. Manual construction of these models is difficult and time-consuming, prompting a shift to automated methods. This study introduces SiMBA (Simplest Mechanism Builder Algorithm), a novel approach for generating microkinetic models from kinetic data. SiMBA operates through four phases: mechanism generation, mechanism translation, parameter estimation, and model comparison. Our approach systematically proposes reaction mechanisms, using matrix representations and a parallelized backtracking algorithm to manage complexity. These mechanisms are then translated into microkinetic models represented by ordinary differential equations, and optimized to fit available data. Models are compared using information criteria to balance accuracy and complexity, iterating until convergence to an optimal model is reached. Case studies on an aldol condensation reaction, and the dehydration of fructose demonstrate SiMBA's effectiveness in distilling complex kinetic behaviors into simple yet accurate models. While SiMBA predicts intermediates correctly for all case studies, it does not chemically identify intermediates, requiring expert input for complex systems. Despite this, SiMBA significantly enhances mechanistic exploration, offering a robust initial mechanism that accelerates the development and modeling of chemical processes.

Inventory control in modern supply chains has attracted significant attention due to the increasing number of disruptive shocks and the challenges posed by complex dynamics, uncertainties, and limited collaboration. Traditional methods, which often rely on static parameters, struggle to adapt to changing environments. This paper proposes a Multi-Agent Reinforcement Learning (MARL) framework with Graph Neural Networks (GNNs) for state representation to address these limitations. Our approach redefines the action space by parameterizing heuristic inventory control policies, making it adaptive as the parameters dynamically adjust based on system conditions. By leveraging the inherent graph structure of supply chains, our framework enables agents to learn the system's topology, and we employ a centralized learning, decentralized execution scheme that allows agents to learn collaboratively while overcoming information-sharing constraints. Additionally, we incorporate global mean pooling and regularization techniques to enhance performance. We test the capabilities of our proposed approach on four different supply chain configurations and conduct a sensitivity analysis. This work paves the way for utilizing MARL-GNN frameworks to improve inventory management in complex, decentralized supply chain environments.

The gravitational physics landscape is evolving rapidly, driven by our ability to study strong-field regions, in particular black holes. Black Holes Inside and Out gathered world experts to discuss the status of the field and prospects ahead. We hope that the ideas and perspectives are a source of inspiration. Structure: Black Hole Evaporation - 50 Years by William Unruh The Stability Problem for Extremal Black Holes by Mihalis Dafermos The Entropy of Black Holes by Robert M. Wald The Non-linear Regime of Gravity by Luis Lehner Black Holes Galore in D > 4 by Roberto Emparan Same as Ever: Looking for (In)variants in the Black Holes Landscape by Carlos A. R. Herdeiro Black Holes, Cauchy Horizons, and Mass Inflation by Matt Visser The Backreaction Problem for Black Holes in Semiclassical Gravity by Adrian del Rio Black Holes Beyond General Relativity by Enrico Barausse and Jutta Kunz Black Holes as Laboratories: Searching for Ultralight Fields by Richard Brito Primordial Black Holes from Inflation by Misao Sasaki Tests of General Relativity with Future Detectors by Emanuele Berti Black Holes as Laboratories: Tests of General Relativity by Ruth Gregory and Samaya Nissanke Simulating Black Hole Imposters by Frans Pretorius Black Hole Spectroscopy: Status Report by Gregorio Carullo VLBI as a Precision Strong Gravity Instrument by Paul Tiede Testing the nature of compact objects and the black hole paradigm by Mariafelicia De Laurentis and Paolo Pani Some Thoughts about Black Holes in Asymptotic Safety by Alessia Platania Black Hole Evaporation in Loop Quantum Gravity by Abhay Ashtekar How the Black Hole Puzzles are Resolved in String Theory by Samir D. Mathur Quantum Black Holes: From Regularization to Information Paradoxes by Niayesh Afshordi and Stefano Liberati, Comment: 221 pages, 21 contributions

We introduce a model that can accurately simulate radiation from undulator sources for ray tracing applications. It incorporates several key effects relevant to 4$^\text{th}$ generation synchrotron sources, such as electron emittance, energy spread, and diffraction-limited beam size. This code has been developed as part of SHADOW4, the latest version of the widely used SHADOW X-ray optics ray tracing program. The approach relies on calculating the field distribution in the far field, which determines the ray divergences. The integration of existing models for electron energy spread is also addressed. Rays sampled at the source follow a size distribution derived by backpropagating the far field radiation. These models are detailed, and several examples are provided., Comment: 12 page, 12 figures

Vision Language Models (VLMs) often struggle with culture-specific knowledge, particularly in languages other than English and in underrepresented cultural contexts. To evaluate their understanding of such knowledge, we introduce WorldCuisines, a massive-scale benchmark for multilingual and multicultural, visually grounded language understanding. This benchmark includes a visual question answering (VQA) dataset with text-image pairs across 30 languages and dialects, spanning 9 language families and featuring over 1 million data points, making it the largest multicultural VQA benchmark to date. It includes tasks for identifying dish names and their origins. We provide evaluation datasets in two sizes (12k and 60k instances) alongside a training dataset (1 million instances). Our findings show that while VLMs perform better with correct location context, they struggle with adversarial contexts and predicting specific regional cuisines and languages. To support future research, we release a knowledge base with annotated food entries and images along with the VQA data., Comment: Accepted by NAACL 2025

In the modern world, the development of Artificial Intelligence (AI) has contributed to improvements in various areas, including automation, computer vision, fraud detection, and more. AI can be leveraged to enhance the efficiency of Autonomous Smart Traffic Management (ASTM) systems and reduce traffic congestion rates. This paper presents an Autonomous Smart Traffic Management (STM) system that uses AI to improve traffic flow rates. The system employs the YOLO V5 Convolutional Neural Network to detect vehicles in traffic management images. Additionally, it predicts the number of vehicles for the next 12 hours using a Recurrent Neural Network with Long Short-Term Memory (RNN-LSTM). The Smart Traffic Management Cycle Length Analysis manages the traffic cycle length based on these vehicle predictions, aided by AI. From the results of the RNN-LSTM model for predicting vehicle numbers over the next 12 hours, we observe that the model predicts traffic with a Mean Squared Error (MSE) of 4.521 vehicles and a Root Mean Squared Error (RMSE) of 2.232 vehicles. After simulating the STM system in the CARLA simulation environment, we found that the Traffic Management Congestion Flow Rate with ASTM (21 vehicles per minute) is 50\% higher than the rate without STM (around 15 vehicles per minute). Additionally, the Traffic Management Vehicle Pass Delay with STM (5 seconds per vehicle) is 70\% lower than without STM (around 12 seconds per vehicle). These results demonstrate that the STM system using AI can increase traffic flow by 50\% and reduce vehicle pass delays by 70\%., Comment: Novel Autonomous Smart Traffic Management System using End-to-End Artificial Intelligence

Point cloud completion networks are conventionally trained to minimize the disparities between the completed point cloud and the ground-truth counterpart. However, an incomplete object-level point cloud can have multiple valid completion solutions when it is examined in isolation. This one-to-many mapping issue can cause contradictory supervision signals to the network because the loss function may produce different values for identical input-output pairs of the network. In many cases, this issue could adversely affect the network optimization process. In this work, we propose to enhance the conventional learning objective using a novel completion consistency loss to mitigate the one-to-many mapping problem. Specifically, the proposed consistency loss ensure that a point cloud completion network generates a coherent completion solution for incomplete objects originating from the same source point cloud. Experimental results across multiple well-established datasets and benchmarks demonstrated the proposed completion consistency loss have excellent capability to enhance the completion performance of various existing networks without any modification to the design of the networks. The proposed consistency loss enhances the performance of the point completion network without affecting the inference speed, thereby increasing the accuracy of point cloud completion. Notably, a state-of-the-art point completion network trained with the proposed consistency loss can achieve state-of-the-art accuracy on the challenging new MVP dataset. The code and result of experiment various point completion models using proposed consistency loss will be available at: https://github.com/kaist-avelab/ConsistencyLoss ., Comment: First version of Paper "Enhancing Performance of Point Cloud Completion Networks with Consistency Loss" by Kevin Tirta Wijaya and Christofel Rio Goenawan. In process submission to Neurocomputing Journal 2024

Today, we are open-sourcing our core speech recognition and diarization models for non-commercial use. We are releasing both a full production pipeline for developers as well as pared-down research models for experimentation. Rev hopes that these releases will spur research and innovation in the fast-moving domain of voice technology. The speech recognition models released today outperform all existing open source speech recognition models across a variety of long-form speech recognition domains.

In a previous work (Hern\'andez-Bernal et al. 2021) we performed an observational analysis of the Arsia Mons Elongated Cloud (AMEC), which stands out due to its impressive size and shape, quick dynamics, and the fact that it happens during the martian dusty season. Observations show that its morphology can be split in a head, on the western slope of the volcano of around 120 km in diameter; and a tail, that expands to the west reaching more than 1000 km in length, making the AMEC the longest orographic cloud observed so far in the solar system. In this work we run the LMD (Laboratoire de M\'et\'eorologie Dynamique) Mesoscale Model to gain insight into the physics of the AMEC. We note that it is coincident in terms of local time and seasonality with the fastest winds on the summit of Arsia Mons. A downslope windstorm on the western slope is followed by a hydraulic-like jump triggering a strong vertical updraft that propagates upwards in the atmosphere, causing a drop in temperatures of down to 30K at 40-50 km in altitude, spatially and temporarily coincident with the observed head of the AMEC. However the model does not reproduce the microphysics of this cloud: the optical depth is too low and the expansion of the tail does not happen in the model. The observed diurnal cycle is correctly captured by the model for the head of the cloud. This work raises new questions that will guide future observations of the AMEC.

The Visual Monitoring Camera (VMC) is a small imaging instrument onboard Mars Express with a field of view of ~40x30 degrees. The camera was initially intended to provide visual confirmation of the separation of the Beagle 2 lander and has similar technical specifications to a typical webcam of the 2000s. In 2007, a few years after the end of its original mission, VMC was turned on again to obtain full-disk images of Mars to be used for outreach purposes. As VMC obtained more images, the scientific potential of the camera became evident, and in 2018 the camera was given an upgraded status of a new scientific instrument, with science goals in the field of Martian atmosphere meteorology. The wide Field of View of the camera combined with the orbit of Mars Express enable the acquisition of full-disk images of the planet showing different local times, which for a long time has been rare among orbital missions around Mars. The small data volume of images also allows videos that show the atmospheric dynamics of dust and cloud systems to be obtained. This paper is intended to be the new reference paper for VMC as a scientific instrument, and thus provides an overview of the updated procedures to plan, command and execute science observations of the Martian atmosphere. These observations produce valuable science data that is calibrated and distributed to the community for scientific use.