Search

Self-play has powered breakthroughs in two-player and multi-player games. Here we show that self-play is a surprisingly effective strategy in another domain. We show that robust and naturalistic driving emerges entirely from self-play in simulation at unprecedented scale -- 1.6~billion~km of driving. This is enabled by Gigaflow, a batched simulator that can synthesize and train on 42 years of subjective driving experience per hour on a single 8-GPU node. The resulting policy achieves state-of-the-art performance on three independent autonomous driving benchmarks. The policy outperforms the prior state of the art when tested on recorded real-world scenarios, amidst human drivers, without ever seeing human data during training. The policy is realistic when assessed against human references and achieves unprecedented robustness, averaging 17.5 years of continuous driving between incidents in simulation.

Interactive digital agents (IDAs) leverage APIs of stateful digital environments to perform tasks in response to user requests. While IDAs powered by instruction-tuned large language models (LLMs) can react to feedback from interface invocations in multi-step exchanges, they have not been trained in their respective digital environments. Prior methods accomplish less than half of tasks in sophisticated benchmarks such as AppWorld. We present a reinforcement learning (RL) approach that trains IDAs directly in their target environments. We formalize this training as a partially observable Markov decision process and derive LOOP, a data- and memory-efficient variant of proximal policy optimization. LOOP uses no value network and maintains exactly one copy of the underlying LLM in memory, making its implementation straightforward and as memory-efficient as fine-tuning a single LLM. A 32-billion-parameter agent trained with LOOP in the AppWorld environment outperforms the much larger OpenAI o1 agent by 9 percentage points (15% relative). To our knowledge, this is the first reported application of RL to IDAs that interact with a stateful, multi-domain, multi-app environment via direct API calls. Our analysis sheds light on the effectiveness of RL in this area, showing that the agent learns to consult the API documentation, avoid unwarranted assumptions, minimize confabulation, and recover from setbacks.

As language models grow ever larger, so do their vocabularies. This has shifted the memory footprint of LLMs during training disproportionately to one single layer: the cross-entropy in the loss computation. Cross-entropy builds up a logit matrix with entries for each pair of input tokens and vocabulary items and, for small models, consumes an order of magnitude more memory than the rest of the LLM combined. We propose Cut Cross-Entropy (CCE), a method that computes the cross-entropy loss without materializing the logits for all tokens into global memory. Rather, CCE only computes the logit for the correct token and evaluates the log-sum-exp over all logits on the fly. We implement a custom kernel that performs the matrix multiplications and the log-sum-exp reduction over the vocabulary in flash memory, making global memory consumption for the cross-entropy computation negligible. This has a dramatic effect. Taking the Gemma 2 (2B) model as an example, CCE reduces the memory footprint of the loss computation from 24 GB to 1 MB, and the total training-time memory consumption of the classifier head from 28 GB to 1 GB. To improve the throughput of CCE, we leverage the inherent sparsity of softmax and propose to skip elements of the gradient computation that have a negligible (i.e., below numerical precision) contribution to the gradient. Experiments demonstrate that the dramatic reduction in memory consumption is accomplished without sacrificing training speed or convergence., Comment: Code is available at https://github.com/apple/ml-cross-entropy

This research represents a pioneering application of automated pose estimation from drone data to study elephant behavior in the wild, utilizing video footage captured from Samburu National Reserve, Kenya. The study evaluates two pose estimation workflows: DeepLabCut, known for its application in laboratory settings and emerging wildlife fieldwork, and YOLO-NAS-Pose, a newly released pose estimation model not previously applied to wildlife behavioral studies. These models are trained to analyze elephant herd behavior, focusing on low-resolution ($\sim$50 pixels) subjects to detect key points such as the head, spine, and ears of multiple elephants within a frame. Both workflows demonstrated acceptable quality of pose estimation on the test set, facilitating the automated detection of basic behaviors crucial for studying elephant herd dynamics. For the metrics selected for pose estimation evaluation on the test set -- root mean square error (RMSE), percentage of correct keypoints (PCK), and object keypoint similarity (OKS) -- the YOLO-NAS-Pose workflow outperformed DeepLabCut. Additionally, YOLO-NAS-Pose exceeded DeepLabCut in object detection evaluation. This approach introduces a novel method for wildlife behavioral research, including the burgeoning field of wildlife drone monitoring, with significant implications for wildlife conservation., Comment: Accepted to CV4Animals: Computer Vision for Animal Behavior Tracking and Modeling Workshop in conjunction with Computer Vision and Pattern Recognition 2024

Sequential recommendation systems often struggle to make predictions or take action when dealing with cold-start items that have limited amount of interactions. In this work, we propose SimRec - a new approach to mitigate the cold-start problem in sequential recommendation systems. SimRec addresses this challenge by leveraging the inherent similarity among items, incorporating item similarities into the training process through a customized loss function. Importantly, this enhancement is attained with identical model architecture and the same amount of trainable parameters, resulting in the same inference time and requiring minimal additional effort. This novel approach results in a robust contextual sequential recommendation model capable of effectively handling rare items, including those that were not explicitly seen during training, thereby enhancing overall recommendation performance. Rigorous evaluations against multiple baselines on diverse datasets showcase SimRec's superiority, particularly in scenarios involving items occurring less than 10 times in the training data. The experiments reveal an impressive improvement, with SimRec achieving up to 78% higher HR@10 compared to SASRec. Notably, SimRec outperforms strong baselines on sparse datasets while delivering on-par performance on dense datasets. Our code is available at https://github.com/amazon-science/sequential-recommendation-using-similarity., Comment: ACM RecSys 2024 Workshop on Context-Aware Recommender Systems

High-resolution flood probability maps are essential for addressing the limitations of existing flood risk assessment approaches but are often limited by the availability of historical event data. Also, producing simulated data needed for creating probabilistic flood maps using physics-based models involves significant computation and time effort inhibiting the feasibility. To address this gap, this study introduces Flood-Precip GAN (Flood-Precipitation Generative Adversarial Network), a novel methodology that leverages generative machine learning to simulate large-scale synthetic inundation data to produce probabilistic flood maps. With a focus on Harris County, Texas, Flood-Precip GAN begins with training a cell-wise depth estimator using a limited number of physics-based model-generated precipitation-flood events. This model, which emphasizes precipitation-based features, outperforms universal models. Subsequently, a Generative Adversarial Network (GAN) with constraints is employed to conditionally generate synthetic precipitation records. Strategic thresholds are established to filter these records, ensuring close alignment with true precipitation patterns. For each cell, synthetic events are smoothed using a K-nearest neighbors algorithm and processed through the depth estimator to derive synthetic depth distributions. By iterating this procedure and after generating 10,000 synthetic precipitation-flood events, we construct flood probability maps in various formats, considering different inundation depths. Validation through similarity and correlation metrics confirms the fidelity of the synthetic depth distributions relative to true data. Flood-Precip GAN provides a scalable solution for generating synthetic flood depth data needed to create high-resolution flood probability maps, significantly enhancing flood preparedness and mitigation efforts.

Response to a poll can be manipulated by means of a series of leading questions. We show that such phenomena cannot be explained by use of classical probability theory, whereas quantum probability theory admits a possibility of offering an explanation. Admissible transformation rules in quantum probability, however, do impose some constraints on the modelling of cognitive behaviour, which are highlighted here. Focusing on a recent poll conducted by Ipsos on a set of questions posed by Sir Humphrey Appleby in an episode of the British political satire \textit{Yes, Prime Minister}, we show that the resulting data cannot be explained quite so simply using quantum rules, although it seems not impossible., Comment: 12 pages, 1 figure

Modern applications often leverage multiple views of a subject of study. Within neuroscience, there is growing interest in large-scale simultaneous recordings across multiple brain regions. Understanding the relationship between views (e.g., the neural activity in each region recorded) can reveal fundamental principles about the characteristics of each representation and about the system. However, existing methods to characterize such relationships either lack the expressivity required to capture complex nonlinearities, describe only sources of variance that are shared between views, or discard geometric information that is crucial to interpreting the data. Here, we develop a nonlinear neural network-based method that, given paired samples of high-dimensional views, disentangles low-dimensional shared and private latent variables underlying these views while preserving intrinsic data geometry. Across multiple simulated and real datasets, we demonstrate that our method outperforms competing methods. Using simulated populations of lateral geniculate nucleus (LGN) and V1 neurons we demonstrate our model's ability to discover interpretable shared and private structure across different noise conditions. On a dataset of unrotated and corresponding but randomly rotated MNIST digits, we recover private latents for the rotated view that encode rotation angle regardless of digit class, and places the angle representation on a 1-d manifold, while shared latents encode digit class but not rotation angle. Applying our method to simultaneous Neuropixels recordings of hippocampus and prefrontal cortex while mice run on a linear track, we discover a low-dimensional shared latent space that encodes the animal's position. We propose our approach as a general-purpose method for finding succinct and interpretable descriptions of paired data sets in terms of disentangled shared and private latent variables.

High resolution calorimetry with state-of-the-art energy resolution performance for both electromagnetic (EM) and hadronic signals can be achieved using the dual-readout (DR) technique, both in a homogeneous scintillating-crystal calorimeter and in a traditional fiber and absorber-based DR hadronic section. We present results from the CalVision consortium studying the collection of Cerenkov and scintillation signals in PbWO$_4$ and BGO crystal samples exposed to 120\,GeV proton beams at the Fermilab Test Beam Facility, including proof-of-principle measurements aimed at demonstrating the identification of a sufficiently large Cerenkov signal in homogeneous scintillating crystals to support dual-readout capability., Comment: 4 pages, 7 figures, Submitted to Proceedings of the contributions to the CALOR2024, EPJ Web of Conferences

The emergence of classical behaviour in quantum theory is often ascribed to the interaction of a quantum system with its environment, which can be interpreted as environmental monitoring of the system. As a result, off-diagonal elements of the density matrix of the system are damped in the basis of a preferred observable, often taken to be the position, leading to the phenomenon of decoherence. This effect can be modelled dynamically in terms of a Lindblad equation driven by the position operator. Here the question of decoherence resulting from a monitoring of position and momentum, i.e. a phase-space measurement, by the environment is addressed. There is no standard quantum observable corresponding to the detection of phase-space points, which is forbidden by Heisenberg's uncertainty principle. This issue is addressed by use of a coherent-state-based positive operator-valued measure (POVM) for modelling phase-space monitoring by the environment. In this scheme, decoherence in phase space implies the diagonalisation of the density matrix in both position and momentum representations. This is shown to be linked to a Lindblad dynamics where position and momentum appear as two independent Lindblad operators., Comment: 5 pages, 2 figures

Near-real time estimation of damage to buildings and infrastructure, referred to as damage nowcasting in this study, is crucial for empowering emergency responders to make informed decisions regarding evacuation orders and infrastructure repair priorities during disaster response and recovery. Here, we introduce FloodDamageCast, a machine learning framework tailored for property flood damage nowcasting. The framework leverages heterogeneous data to predict residential flood damage at a resolution of 500 meters by 500 meters within Harris County, Texas, during the 2017 Hurricane Harvey. To deal with data imbalance, FloodDamageCast incorporates a generative adversarial networks-based data augmentation coupled with an efficient machine learning model. The results demonstrate the model's ability to identify high-damage spatial areas that would be overlooked by baseline models. Insights gleaned from flood damage nowcasting can assist emergency responders to more efficiently identify repair needs, allocate resources, and streamline on-the-ground inspections, thereby saving both time and effort., Comment: 20 pages, 6 figures

In this paper we study $d$-dependent random graphs -- introduced by Brody and Sanchez -- which are the family of random graph distributions where each edge is present with probability $p$, and each edge is independent of all but at most $d$ other edges. For this random graph model, we analyze degree sequences, jumbledness, connectivity, and subgraph containment. Our results mirror those of the classical Erd\H{o}s--R\'enyi random graph, which are recovered by specializing our problem to $d=0$, although we show that in many regards our setting is appreciably more nuanced. We survey what is known for this model and conclude with a variety of open questions., Comment: 12 pages

Examining the relationship between vulnerability of the built environment and community recovery is crucial for understanding disaster resilience. Yet, this relationship is rather neglected in the existing literature due to previous limitations in the availability of empirical datasets needed for such analysis. In this study, we combine fine-resolution flood damage claims data (composed of both insured and uninsured losses) and human mobility data (composed of millions of movement trajectories) during the 2017 Hurricane Harvey in Harris County, Texas, to specify the extent to which vulnerability of the built environment (i.e., flood property damage) affects community recovery (based on the speed of human mobility recovery) locally and regionally. We examine this relationship using a spatial lag, spatial reach, and spatial decay models to measure the extent of spillover effects of residential damage on community recovery. The findings show that: first, the severity of residential damage significantly affects the speed of community recovery. A greater extent of residential damage suppresses community recovery not only locally but also in the surrounding areas. Second, the spatial spillover effect of residential damage on community recovery speed decays with distance from the highly damaged areas. Third, spatial areas display heterogeneous spatial decay coefficients, which are associated with urban structure features such as the density of points-of-interest facilities and roads. These findings provide a novel data-driven characterization of the spatial diffusion of residential flood damage effects on community recovery and move us closer to a better understanding of complex spatial processes that shape community resilience to hazards. This study also provides valuable insights for emergency managers and public officials seeking to mitigate the non-local effects of residential damage.

We show that an isometric action of a torsion-free uniform lattice $\Gamma$ on hyperbolic space $\mathbb{H}^n$ can be metrically approximated by geometric actions of $\Gamma$ on $\mathrm{CAT}(0)$ cube complexes, provided that either $n$ is at most three, or the lattice is arithmetic of simplest type. This solves a conjecture of Futer and Wise. Our main tool is the study of a space of co-geodesic currents, consisting of invariant Radon measures supported on codimension-1 hyperspheres in the Gromov boundary of $\mathbb{H}^n$. By pairing co-geodesic currents and geodesic currents via an intersection number, we show that asymptotic convergence of geometric actions can be deduced from the convergence of their dual co-geodesic currents. For surface groups, our methods also imply approximation by cubulations for actions induced by non-positively curved Riemannian surfaces with singularities, Hitchin and maximal representations, and quasiFuchsian representations., Comment: 46 pages, 1 figure

The governance of online communities has been a critical issue since the first USENET groups, and a number of serious constitutions -- declarations of goals, values, and rights -- have emerged since the mid-1990s. More recently, decentralized autonomous organizations (DAOs) have begun to publish their own constitutions, manifestos, and other governance documents. There are two unique aspects to these documents: they (1) often govern significantly more resources than previously-observed online communities, and (2) are used in conjunction with smart contracts that can secure certain community rights and processes through code. In this article, we analyze 25 DAO constitutions, observe a number of common patterns, and provide a template and a set of recommendations to support the crafting and dissemination of future DAO constitutions. We conclude with a report on how our template and recommendations were then used within the actual constitutional drafting process of a major blockchain.

It is shown that if the wave function of a quantum system undergoes an arbitrary random transformation such that the diagonal elements of the density matrix in the decoherence basis associated with a preferred observable remain constant, then (i) the off-diagonal elements of the density matrix become smaller in magnitude, and (ii) the state of the system gains information about the preferred observable from its environment in the sense that the uncertainty of the observable is reduced in the transformed state. These results do not depend on the details of how the system-environment interaction generates the random state transformation, and together imply that decoherence leads in general to information gain, not information loss., Comment: 5 pages, version to appear in Physical Review Research

Climate change is a prevalent threat, and it is unlikely that current mitigation efforts will be enough to avoid unwanted impacts. One potential option to reduce climate change impacts is the use of stratospheric aerosol injection (SAI). Even if SAI is ultimately deployed, it might be initiated only after some temperature target is exceeded. The consequences of such a delay are assessed herein. This study compares two cases, with the same target global mean temperature of 1.5C above preindustrial, but start dates of 2035 or a delayed start in 2045. We make use of simulations in the Community Earth System Model version 2 with the Whole Atmosphere Coupled Chemistry Model version 6 (CESM2-WACCM6), using SAI under the SSP2-4.5 emissions pathway. We find that delaying the start of deployment (relative to the target temperature) necessitates lower net radiative forcing (-30%) and thus larger sulfur dioxide injection rates (+20%), even after surface temperatures converge, to compensate for the extra energy absorbed by the Earth system. However, many of the surface climate differences between the 2035 and 2045 start simulations appear to be small during the 10-25 years following the delayed SAI start, although longer simulations would be needed to assess any longer-term impacts in this model. In addition, irreversibilities and tipping points that might be triggered during the period of increased warming may not be adequately represented in the model but could change this conclusion in the real world.

The solution of a PDE over varying initial/boundary conditions on multiple domains is needed in a wide variety of applications, but it is computationally expensive if the solution is computed de novo whenever the initial/boundary conditions of the domain change. We introduce a general operator learning framework, called DIffeomorphic Mapping Operator learNing (DIMON) to learn approximate PDE solutions over a family of domains $\{\Omega_{\theta}}_\theta$, that learns the map from initial/boundary conditions and domain $\Omega_\theta$ to the solution of the PDE, or to specified functionals thereof. DIMON is based on transporting a given problem (initial/boundary conditions and domain $\Omega_{\theta}$) to a problem on a reference domain $\Omega_{0}$, where training data from multiple problems is used to learn the map to the solution on $\Omega_{0}$, which is then re-mapped to the original domain $\Omega_{\theta}$. We consider several problems to demonstrate the performance of the framework in learning both static and time-dependent PDEs on non-rigid geometries; these include solving the Laplace equation, reaction-diffusion equations, and a multiscale PDE that characterizes the electrical propagation on the left ventricle. This work paves the way toward the fast prediction of PDE solutions on a family of domains and the application of neural operators in engineering and precision medicine.

This paper examines the stability of the \`a trous algorithm under arbitrary iteration in the context of a more general study of shift-invariant filter banks. The main results describe sufficient conditions on the associated filters under which an infinitely iterated shift-invariant filter bank is stable. Moreover, it is shown that the stability of an infinitely iterated shift-invariant filter bank guarantees that of any associated finitely iterated shift-invariant filter bank, with uniform bounds. The reverse implication is shown to hold under an additional assumption on the low-pass filter. Finally, it is also shown that the separable product of stable one-dimensional shift-invariant filter banks produces a stable two-dimensional shift-invariant filter bank.

A frame $(x_j)_{j\in J}$ for a Hilbert space $H$ allows for a linear and stable reconstruction of any vector $x\in H$ from the linear measurements $(\langle x,x_j\rangle)_{j\in J}$. However, there are many situations where some information in the frame coefficients is lost. In applications where one is using sensors with a fixed dynamic range, any measurement above that range is registered as the maximum, and any measurement below that range is registered as the minimum. Depending on the context, recovering a vector from such measurements is called either declipping or saturation recovery. We initiate a frame theoretic approach to saturation recovery in a similar way to what [BCE06] did for phase retrieval. We characterize when saturation recovery is possible, show optimal frames for use with saturation recovery correspond to minimal multi-fold packings in projective space, and prove that the classical frame algorithm may be adapted to this non-linear problem to provide a reconstruction algorithm., Comment: 20 pages