Search

Inference-time techniques are emerging as highly effective tools to enhance large language model (LLM) capabilities. However, best practices for developing systems that combine these techniques remain underdeveloped due to our limited understanding of the utility of individual inference-time techniques and the interactions between them. Additionally, efficiently and automatically searching the space of model choices, inference-time techniques, and their compositions is challenging due to the large design space. To address these challenges, we introduce Archon, a modular framework for selecting, combining, and stacking layers of inference-time techniques to construct optimized LLM systems for target benchmarks. Rather than relying on a single LLM called once, we leverage a diverse set of LLMs and inference-time techniques, creating LLM systems greater than the sum of their parts. Archon defines an extensible design space, encompassing techniques such as generation ensembling, repeated sampling, ranking, fusion, critiquing, verification, and unit testing. It transforms the problem of building LLM systems into a hyperparameter optimization objective. Given the available LLMs, inference-time techniques, and compute budget, Archon utilizes hyperparameter search techniques to discover optimized architectures for target benchmark(s). We evaluate Archon architectures across a range of instruction-following, reasoning, and coding benchmarks, including MT-Bench, Arena-Hard-Auto, AlpacaEval 2.0, MixEval, MixEval Hard, MATH, and CodeContests. Archon architectures outperform frontier models, such as GPT-4o and Claude 3.5 Sonnet, on these benchmarks, achieving an average accuracy increase of 15.1 percentage points by using all available LLMs. We make our code and datasets available publicly on Github: https://github.com/ScalingIntelligence/Archon.

Using Large Language Models (LLMs) to produce robot programs from natural language has allowed for robot systems that can complete a higher diversity of tasks. However, LLM-generated programs may be faulty, either due to ambiguity in instructions, misinterpretation of the desired task, or missing information about the world state. As these programs run, the state of the world changes and they gather new information. When a failure occurs, it is important that they recover from the current world state and avoid repeating steps that they they previously completed successfully. We propose RoboRepair, a system which traces the execution of a program up until error, and then runs an LLM-produced recovery program that minimizes repeated actions. To evaluate the efficacy of our system, we create a benchmark consisting of eleven tasks with various error conditions that require the generation of a recovery program. We compare the efficiency of the recovery program to a plan built with an oracle that has foreknowledge of future errors., Comment: Under review at ACL Rolling Review

We propose a simple heuristic privacy analysis of noisy clipped stochastic gradient descent (DP-SGD) in the setting where only the last iterate is released and the intermediate iterates remain hidden. Namely, our heuristic assumes a linear structure for the model. We show experimentally that our heuristic is predictive of the outcome of privacy auditing applied to various training procedures. Thus it can be used prior to training as a rough estimate of the final privacy leakage. We also probe the limitations of our heuristic by providing some artificial counterexamples where it underestimates the privacy leakage. The standard composition-based privacy analysis of DP-SGD effectively assumes that the adversary has access to all intermediate iterates, which is often unrealistic. However, this analysis remains the state of the art in practice. While our heuristic does not replace a rigorous privacy analysis, it illustrates the large gap between the best theoretical upper bounds and the privacy auditing lower bounds and sets a target for further work to improve the theoretical privacy analyses. We also empirically support our heuristic and show existing privacy auditing attacks are bounded by our heuristic analysis in both vision and language tasks.

Uncertainty quantification is crucial to account for the imperfect predictions of machine learning algorithms for high-impact applications. Conformal prediction (CP) is a powerful framework for uncertainty quantification that generates calibrated prediction intervals with valid coverage. In this work, we study how CP intervals are affected by bias - the systematic deviation of a prediction from ground truth values - a phenomenon prevalent in many real-world applications. We investigate the influence of bias on interval lengths of two different types of adjustments -- symmetric adjustments, the conventional method where both sides of the interval are adjusted equally, and asymmetric adjustments, a more flexible method where the interval can be adjusted unequally in positive or negative directions. We present theoretical and empirical analyses characterizing how symmetric and asymmetric adjustments impact the "tightness" of CP intervals for regression tasks. Specifically for absolute residual and quantile-based non-conformity scores, we prove: 1) the upper bound of symmetrically adjusted interval lengths increases by $2|b|$ where $b$ is a globally applied scalar value representing bias, 2) asymmetrically adjusted interval lengths are not affected by bias, and 3) conditions when asymmetrically adjusted interval lengths are guaranteed to be smaller than symmetric ones. Our analyses suggest that even if predictions exhibit significant drift from ground truth values, asymmetrically adjusted intervals are still able to maintain the same tightness and validity of intervals as if the drift had never happened, while symmetric ones significantly inflate the lengths. We demonstrate our theoretical results with two real-world prediction tasks: sparse-view computed tomography (CT) reconstruction and time-series weather forecasting. Our work paves the way for more bias-robust machine learning systems., Comment: 17 pages, 6 figures, code available at: https://github.com/matthewyccheung/conformal-metric

Understanding local risks from extreme rainfall, such as flooding, requires both long records (to sample rare events) and high-resolution products (to assess localized hazards). Unfortunately, there is a dearth of long-record and high-resolution products that can be used to understand local risk and precipitation science. In this paper, we present a novel generative diffusion model that downscales (super-resolves) globally available Climate Prediction Center (CPC) gauge-based precipitation products and ERA5 reanalysis data to generate kilometer-scale precipitation estimates. Downscaling gauge-based precipitation from 55 km to 1 km while recovering extreme rainfall signals poses significant challenges. To enforce our model (named WassDiff) to produce well-calibrated precipitation intensity values, we introduce a Wasserstein Distance Regularization (WDR) term for the score-matching training objective in the diffusion denoising process. We show that WDR greatly enhances the model's ability to capture extreme values compared to diffusion without WDR. Extensive evaluation shows that WassDiff has better reconstruction accuracy and bias scores than conventional score-based diffusion models. Case studies of extreme weather phenomena, like tropical storms and cold fronts, demonstrate WassDiff's ability to produce appropriate spatial patterns while capturing extremes. Such downscaling capability enables the generation of extensive km-scale precipitation datasets from existing historical global gauge records and current gauge measurements in areas without high-resolution radar., Comment: 19 pages, 9 figures

A key component of dyadic spoken interactions is the contextually relevant non-verbal gestures, such as head movements that reflect a listener's response to the interlocutor's speech. Although significant progress has been made in the context of generating co-speech gestures, generating listener's response has remained a challenge. We introduce the task of generating continuous head motion response of a listener in response to the speaker's speech in real time. To this end, we propose a graph-based end-to-end crossmodal model that takes interlocutor's speech audio as input and directly generates head pose angles (roll, pitch, yaw) of the listener in real time. Different from previous work, our approach is completely data-driven, does not require manual annotations or oversimplify head motion to merely nods and shakes. Extensive evaluation on the dyadic interaction sessions on the IEMOCAP dataset shows that our model produces a low overall error (4.5 degrees) and a high frame rate, thereby indicating its deployability in real-world human-robot interaction systems. Our code is available at - https://github.com/bigzen/Active-Listener, Comment: 4+1 pages, 3 figures, 2 tables

Erbium-doped \ch{TiO2} materials are promising candidates for advancing quantum technologies, necessitating a thorough understanding of their electronic and crystal structures to tailor their properties and enhance coherence times. This study explored epitaxial erbium-doped rutile \ch{TiO2} films deposited on r-sapphire substrates using molecular beam epitaxy. Photoluminescence excitation spectroscopy demonstrated decreasing fluorescence lifetimes with erbium doping, indicating limited coherence times. Lattice distortions associated with \ch{Er^{3+}} were probed by X-ray absorption spectroscopy, indicating that erbium primarily occupies \ch{Ti^{4+}} sites and influences oxygen vacancies. Significant lattice distortions in the higher-order shells and full coordination around erbium suggest that additional defects are likely prevalent in these regions. These findings indicate that defects contribute to limited coherence times by introducing alternative decay pathways, leading to shorter fluorescence lifetimes., Comment: 8 pages, 13 figures, 3 tables

We studied the possible presence of dark matter (DM) in neutron stars (NSs) and the structural properties of the DM admixed NSs (DMANSs) in one of our recent works \cite{Guha:2024pnn}. The feeble interaction between the fermionic DM ($\chi$) with the hadronic matter is introduced through a dark scalar ($\phi$) and a dark vector ($\xi$) boson as mediators. The allowed range of the mass of the fermionic DM ($m_{\chi}$), for a particular range of DM Fermi momentum ($k_F^{\chi}$), was obtained in the same work \cite{Guha:2024pnn} with respect to the various astrophysical constraints on the structural properties of compact stars viz. the mass, radius and tidal deformability. The present work is dedicated to the calculation and study of non-radial oscillation of the DMANSs using Cowling approximation. We particularly investigate the effect of presence of DM on the fundamental ($f$) mode oscillation frequencies of the DMANSs utilizing the previously obtained range of $m_{\chi}$ for four different hadronic models. In this work we thoroughly investigate how the individual and combined effects of $m_{\chi}$ and $k_F^{\chi}$ affect the $f$-mode oscillation frequency. Within the framework of our DMANS models, for a particular value of $k_F^{\chi}$, the range of $f_{max}^{DMANS}$ with respect to the allowed range of $m_{\chi}$, is also obtained in the present work for four different hadronic models. Since in the present era, the 1.4 and 2.01 $M_{\odot}$ NSs are of special interest after the detection of GW170817 and PSR J0740+6620, we particularly investigate, for the four hadronic models, the range of $f_{1.4}^{DMANS}$ and $f_{2.01}^{DMANS}$ with respect to the acceptable range of $m_{\chi}$ corresponding to the range of $k_F^{\chi}$., Comment: Accepted for Publication is Phys. Rev. D; https://journals.aps.org/prd/accepted/cb07fQ7eG101a43b86a417d4b7d78185f8333ca5a

Quantum neural networks are deemed suitable to replace classical neural networks in their ability to learn and scale up network models using quantum-exclusive phenomena like superposition and entanglement. However, in the noisy intermediate scale quantum (NISQ) era, the trainability and expressibility of quantum models are yet under investigation. Medical image classification on the other hand, pertains well to applications in deep learning, particularly, convolutional neural networks. In this paper, we carry out a study of three hybrid classical-quantum neural network architectures and combine them using standard ensembling techniques on a breast cancer histopathological dataset. The best accuracy percentage obtained by an individual model is 85.59. Whereas, on performing ensemble, we have obtained accuracy as high as 86.72%, an improvement over the individual hybrid network as well as classical neural network counterparts of the hybrid network models., Comment: Accepted in the 3rd International Conference on Data Electronics and Computing

This work proposes a kinodynamic motion planning technique for collaborative object transportation by multiple mobile manipulators in dynamic environments. A global path planner computes a linear piecewise path from start to goal. A novel algorithm detects the narrow regions between the static obstacles and aids in defining the obstacle-free region to enhance the feasibility of the global path. We then formulate a local online motion planning technique for trajectory generation that minimizes the control efforts in a receding horizon manner. It plans the trajectory for finite time horizons, considering the kinodynamic constraints and the static and dynamic obstacles. The planning technique jointly plans for the mobile bases and the arms to utilize the locomotion capability of the mobile base and the manipulation capability of the arm efficiently. We use a convex cone approach to avoid self-collision of the formation by modifying the mobile manipulators admissible state without imposing additional constraints. Numerical simulations and hardware experiments showcase the efficiency of the proposed approach., Comment: Pre-print Under Review

Implicit neural representations (INRs) have demonstrated success in a variety of applications, including inverse problems and neural rendering. An INR is typically trained to capture one signal of interest, resulting in learned neural features that are highly attuned to that signal. Assumed to be less generalizable, we explore the aspect of transferability of such learned neural features for fitting similar signals. We introduce a new INR training framework, STRAINER that learns transferrable features for fitting INRs to new signals from a given distribution, faster and with better reconstruction quality. Owing to the sequential layer-wise affine operations in an INR, we propose to learn transferable representations by sharing initial encoder layers across multiple INRs with independent decoder layers. At test time, the learned encoder representations are transferred as initialization for an otherwise randomly initialized INR. We find STRAINER to yield extremely powerful initialization for fitting images from the same domain and allow for $\approx +10dB$ gain in signal quality early on compared to an untrained INR itself. STRAINER also provides a simple way to encode data-driven priors in INRs. We evaluate STRAINER on multiple in-domain and out-of-domain signal fitting tasks and inverse problems and further provide detailed analysis and discussion on the transferability of STRAINER's features. Our demo can be accessed at https://colab.research.google.com/drive/1fBZAwqE8C_lrRPAe-hQZJTWrMJuAKtG2?usp=sharing .

Non-invasive and continuous blood pressure (BP) monitoring is essential for the early prevention of many cardiovascular diseases. Estimating arterial blood pressure (ABP) from photoplethysmography (PPG) has emerged as a promising solution. However, existing deep learning approaches for PPG-to-ABP reconstruction (PAR) encounter certain information loss, impacting the precision of the reconstructed signal. To overcome this limitation, we introduce an invertible neural network for PPG to ABP reconstruction (INN-PAR), which employs a series of invertible blocks to jointly learn the mapping between PPG and its gradient with the ABP signal and its gradient. INN-PAR efficiently captures both forward and inverse mappings simultaneously, thereby preventing information loss. By integrating signal gradients into the learning process, INN-PAR enhances the network's ability to capture essential high-frequency details, leading to more accurate signal reconstruction. Moreover, we propose a multi-scale convolution module (MSCM) within the invertible block, enabling the model to learn features across multiple scales effectively. We have experimented on two benchmark datasets, which show that INN-PAR significantly outperforms the state-of-the-art methods in both waveform reconstruction and BP measurement accuracy.

A large number of multi-attribute group decisionmaking (MAGDM) have been widely introduced to obtain consensus results. However, most of the methodologies ignore the conflict among the experts opinions and only consider equal or variable priorities of them. Therefore, this study aims to propose an Evidential MAGDM method by assessing the inter-observational variability and handling uncertainty that emerges between the experts. The proposed framework has fourfold contributions. First, the basic probability assignment (BPA) generation method is introduced to consider the inherent characteristics of each alternative by computing the degree of belief. Second, the ordered weighted belief and plausibility measure is constructed to capture the overall intrinsic information of the alternative by assessing the inter-observational variability and addressing the conflicts emerging between the group of experts. An ordered weighted belief divergence measure is constructed to acquire the weighted support for each group of experts to obtain the final preference relationship. Finally, we have shown an illustrative example of the proposed Evidential MAGDM framework. Further, we have analyzed the interpretation of Evidential MAGDM in the real-world application for ensemble classifier feature fusion to diagnose retinal disorders using optical coherence tomography images.

When cell sheets fold during development, their apical or basal surfaces constrict and cell shapes approach the geometric singularity in which these surfaces vanish. Here, we reveal the mechanical consequences of this geometric singularity for tissue folding in a minimal vertex model of an epithelial monolayer. In simulations of the buckling of the epithelium under compression and numerical solutions of the corresponding continuum model, we discover an "unbuckling" bifurcation: At large compression, the buckling amplitude can decrease with increasing compression. By asymptotic solution of the continuum equations, we reveal that this bifurcation comes with a large stiffening of the epithelium. Our results thus provide the mechanical basis for absorption of compressive stresses by tissue folds such as the cephalic furrow during germband extension in Drosophila., Comment: 6 pages, 4 figures; Supplemental Material: 10 pages, 2 figures

Large Language Models (LLMs) are prone to generating factually incorrect information when responding to queries that involve numerical and statistical data or other timely facts. In this paper, we present an approach for enhancing the accuracy of LLMs by integrating them with Data Commons, a vast, open-source repository of public statistics from trusted organizations like the United Nations (UN), Center for Disease Control and Prevention (CDC) and global census bureaus. We explore two primary methods: Retrieval Interleaved Generation (RIG), where the LLM is trained to produce natural language queries to retrieve data from Data Commons, and Retrieval Augmented Generation (RAG), where relevant data tables are fetched from Data Commons and used to augment the LLM's prompt. We evaluate these methods on a diverse set of queries, demonstrating their effectiveness in improving the factual accuracy of LLM outputs. Our work represents an early step towards building more trustworthy and reliable LLMs that are grounded in verifiable statistical data and capable of complex factual reasoning., Comment: 39 pages - 25 page paper, 14 page Appendix, 7 figures, 9 tables

We consider an imaging system tasked with estimating the angular distance between two incoherently-emitting sub-Rayleigh-separated point sources, without any prior knowledge of the centroid or the constellation and with a fixed collected-photon budget. It was shown theoretically that splitting the optical recording time into two stages -- focal-plane direct imaging to obtain a pre-estimate of the centroid, and using that estimate to center a spatial-mode sorter followed by photon detection of the sorted modes -- can achieve 10 to 100 times lower mean squared error in estimating the separation. In this paper, we demonstrate this in proof-of-concept, using a programmable mode sorter we have built using multi-plane light conversion (MPLC) using a reflective spatial-light modulator (SLM) in an emulated experiment where we use a single coherent source to characterize the MPLC to electronically piece together the signature from two closely-separated quasi-monochromatic incoherent emitters., Comment: 7 pages, 7 figures in main paper, 3 pages, 8 figures in supplementary material

Automatic diagnosis techniques have evolved to identify age-related macular degeneration (AMD) by employing single modality Fundus images or optical coherence tomography (OCT). To classify ocular diseases, fundus and OCT images are the most crucial imaging modalities used in the clinical setting. Most deep learning-based techniques are established on a single imaging modality, which contemplates the ocular disorders to a specific extent and disregards other modality that comprises exhaustive information among distinct imaging modalities. This paper proposes a modality-specific multiscale color space embedding integrated with the attention mechanism based on transfer learning for classification (MCGAEc), which can efficiently extract the distinct modality information at various scales using the distinct color spaces. In this work, we first introduce the modality-specific multiscale color space encoder model, which includes diverse feature representations by integrating distinct characteristic color spaces on a multiscale into a unified framework. The extracted features from the prior encoder module are incorporated with the attention mechanism to extract the global features representation, which is integrated with the prior extracted features and transferred to the random forest classifier for the classification of AMD. To analyze the performance of the proposed MCGAEc method, a publicly available multi-modality dataset from Project Macula for AMD is utilized and compared with the existing models., Comment: 27th International Conference on Pattern Recognition (ICPR) 2024

Computed Tomography (CT) scans are the standard-of-care for the visualization and diagnosis of many clinical ailments, and are needed for the treatment planning of external beam radiotherapy. Unfortunately, the availability of CT scanners in low- and mid-resource settings is highly variable. Planar x-ray radiography units, in comparison, are far more prevalent, but can only provide limited 2D observations of the 3D anatomy. In this work we propose DIFR3CT, a 3D latent diffusion model, that can generate a distribution of plausible CT volumes from one or few (<10) planar x-ray observations. DIFR3CT works by fusing 2D features from each x-ray into a joint 3D space, and performing diffusion conditioned on these fused features in a low-dimensional latent space. We conduct extensive experiments demonstrating that DIFR3CT is better than recent sparse CT reconstruction baselines in terms of standard pixel-level (PSNR, SSIM) on both the public LIDC and in-house post-mastectomy CT datasets. We also show that DIFR3CT supports uncertainty quantification via Monte Carlo sampling, which provides an opportunity to measure reconstruction reliability. Finally, we perform a preliminary pilot study evaluating DIFR3CT for automated breast radiotherapy contouring and planning -- and demonstrate promising feasibility. Our code is available at https://github.com/yransun/DIFR3CT., Comment: 11 pages, 9 figures

In the social and health sciences, researchers often make causal inferences using sensitive variables. These researchers, as well as the data holders themselves, may be ethically and perhaps legally obligated to protect the confidentiality of study participants' data. It is now known that releasing any statistics, including estimates of causal effects, computed with confidential data leaks information about the underlying data values. Thus, analysts may desire to use causal estimators that can provably bound this information leakage. Motivated by this goal, we develop algorithms for estimating weighted average treatment effects with binary outcomes that satisfy the criterion of differential privacy. We present theoretical results on the accuracy of several differentially private estimators of weighted average treatment effects. We illustrate the empirical performance of these estimators using simulated data and a causal analysis using data on education and income.

Quantum illumination (QI) is the task of querying a scene using a transmitter probe whose quantum state is entangled with a reference beam retained in ideal storage, followed by optimally detecting the target-returned light together with the stored reference, to make decisions on characteristics of targets at stand-off range, at precision that exceeds what is achievable with a classical transmitter of the same brightness and otherwise identical conditions. Using tools from perturbation theory, we show that in the limit of low transmitter brightness, high loss, and high thermal background, there is a factor of four improvement in the Chernoff exponent of the error probability in discriminating any number of apriori-known reflective targets when using a Gaussian-state entangled QI probe, over using classical coherent-state illumination (CI). While this advantage was known for detecting the presence or absence of a target, it had not been proven for the generalized task of discriminating between arbitrary target libraries. In proving our result, we derive simple general analytic expressions for the lowest-order asymptotic expansions of the quantum Chernoff exponents for QI and CI in terms of the signal brightness, loss, thermal noise, and the modal expansion coefficients of the target-reflected light's radiant exitance profiles when separated by a spatial mode sorter after entering the entrance pupil of the receiver's aperture., Comment: 6 pages, 2 figures, presented at ISIT 2024

We give a general existence and convergence result for interacting particle systems on locally finite graphs with possibly unbounded degrees or jump rates. We allow the local state space to be Polish, and the jumps at a site to affect the states of its neighbours. The two common assumptions on interacting particle systems are uniform bounds on degrees and jump rates. However, in this paper, we relax these assumptions and allow for vertices with high degrees or rapid jumps. We introduce new assumptions that ensure that such vertices are placed sufficiently apart from each other and hence the process does not explode. Our proofs use graphical construction involving an analysis of certain subsets of the set of all self-avoiding walks on the $2$-step graph of the underlying graph. For some random graph models, if the jump rates are bounded by powers of vertex degrees, we give readily verifiable sufficient conditions on the underlying graph itself, under which our assumptions hold almost surely. These conditions involve exponential growth of the average number of self-avoiding walks from each vertex and that of moments of the vertex degrees. Using these conditions, we show the existence of interacting particle systems like sandpile models on random graphs such as the long-range percolation model and the geometric random graph -- models which lack uniform bounds on degrees and jump rates., Comment: 28 pages

Entanglement is essential for quantum information processing but is limited by noise. We address this by developing high-yield entanglement distillation protocols with several advancements. (1) We extend the 2-to-1 recurrence entanglement distillation protocol to higher-rate n-to-(n-1) protocols that can correct any single-qubit errors. These protocols are evaluated through numerical simulations focusing on fidelity and yield. We also outline a method to adapt any classical error-correcting code for entanglement distillation, where the code can correct both bit-flip and phase-flip errors by incorporating Hadamard gates. (2) We propose a constant-depth decoder for stabilizer codes that transforms logical states into physical ones using single-qubit measurements. This decoder is applied to entanglement distillation protocols, reducing circuit depth and enabling protocols derived from advanced quantum error-correcting codes. We demonstrate this by evaluating the circuit complexity for entanglement distillation protocols based on surface codes and quantum convolutional codes. (3) Our stabilizer entanglement distillation techniques advance quantum computing. We propose a fault-tolerant protocol for constant-depth encoding and decoding of arbitrary quantum states, applicable to quantum low-density parity-check (qLDPC) codes and surface codes. This protocol is feasible with state-of-the-art reconfigurable atom arrays and surpasses the limits of conventional logarithmic depth encoders. Overall, our study integrates stabilizer formalism, measurement-based quantum computing, and entanglement distillation, advancing both quantum communication and computing., Comment: 19 pages, 7 figures

Computer vision systems have witnessed rapid progress over the past two decades due to multiple advances in the field. As these systems are increasingly being deployed in high-stakes real-world applications, there is a dire need to ensure that they do not propagate or amplify any discriminatory tendencies in historical or human-curated data or inadvertently learn biases from spurious correlations. This paper presents a comprehensive survey on fairness that summarizes and sheds light on ongoing trends and successes in the context of computer vision. The topics we discuss include 1) The origin and technical definitions of fairness drawn from the wider fair machine learning literature and adjacent disciplines. 2) Work that sought to discover and analyze biases in computer vision systems. 3) A summary of methods proposed to mitigate bias in computer vision systems in recent years. 4) A comprehensive summary of resources and datasets produced by researchers to measure, analyze, and mitigate bias and enhance fairness. 5) Discussion of the field's success, continuing trends in the context of multimodal foundation and generative models, and gaps that still need to be addressed. The presented characterization should help researchers understand the importance of identifying and mitigating bias in computer vision and the state of the field and identify potential directions for future research., Comment: 20 pages, 4 figures

We present foundation language models developed to power Apple Intelligence features, including a ~3 billion parameter model designed to run efficiently on devices and a large server-based language model designed for Private Cloud Compute. These models are designed to perform a wide range of tasks efficiently, accurately, and responsibly. This report describes the model architecture, the data used to train the model, the training process, how the models are optimized for inference, and the evaluation results. We highlight our focus on Responsible AI and how the principles are applied throughout the model development.

In this article, we study whether the slope functions of two functional regression models in two samples are associated with any arbitrary transformation (barring constant and linear transformation) or not along the vertical axis. In order to address this issue, a statistical testing of the hypothesis problem is formalized, and the test statistic is formed based on the estimated second derivative of the unknown transformation. The asymptotic properties of the test statistics are investigated using some advanced techniques related to the empirical process. Moreover, to implement the test for small sample size data, a Bootstrap algorithm is proposed, and it is shown that the Bootstrap version of the test is as good as the original test for sufficiently large sample size. Furthermore, the utility of the proposed methodology is shown for simulated data sets, and DTI data is analyzed using this methodology., Comment: Some typos have been fixed

One of the most promising applications of quantum networks is entanglement assisted sensing. The field of quantum metrology exploits quantum correlations to improve the precision bound for applications such as precision timekeeping, field sensing, and biological imaging. When measuring multiple spatially distributed parameters, current literature focuses on quantum entanglement in the discrete variable case, and quantum squeezing in the continuous variable case, distributed amongst all of the sensors in a given network. However, it can be difficult to ensure all sensors pre-share entanglement of sufficiently high fidelity. This work probes the space between fully entangled and fully classical sensing networks by modeling a star network with probabilistic entanglement generation that is attempting to estimate the average of local parameters. The quantum Fisher information is used to determine which protocols best utilize entanglement as a resource for different network conditions. It is shown that without entanglement distillation there is a threshold fidelity below which classical sensing is preferable. For a network with a given number of sensors and links characterized by a certain initial fidelity and probability of success, this work outlines when and how to use entanglement, when to store it, and when it needs to be distilled., Comment: 22 pages, 9 Figures

AI progress is creating a growing range of risks and opportunities, but it is often unclear how they should be navigated. In many cases, the barriers and uncertainties faced are at least partly technical. Technical AI governance, referring to technical analysis and tools for supporting the effective governance of AI, seeks to address such challenges. It can help to (a) identify areas where intervention is needed, (b) identify and assess the efficacy of potential governance actions, and (c) enhance governance options by designing mechanisms for enforcement, incentivization, or compliance. In this paper, we explain what technical AI governance is, why it is important, and present a taxonomy and incomplete catalog of its open problems. This paper is intended as a resource for technical researchers or research funders looking to contribute to AI governance., Comment: Ben Bucknall and Anka Reuel contributed equally and share the first author position

The enormous scale of state-of-the-art foundation models has limited their accessibility to scientists, because customized experiments at large model sizes require costly hardware and complex engineering that is impractical for most researchers. To alleviate these problems, we introduce NNsight, an open-source Python package with a simple, flexible API that can express interventions on any PyTorch model by building computation graphs. We also introduce NDIF, a collaborative research platform providing researchers access to foundation-scale LLMs via the NNsight API. Code, documentation, and tutorials are available at https://www.nnsight.net., Comment: Code at https://nnsight.net

An ideal direct imaging coronagraph, which selectively rejects the fundamental mode of a telescope, has been shown to achieve the quantum information limits for exoplanet detection and localization. In this study, we experimentally implement this quantum-optimal coronagraph using spatial mode (de)multiplexing. Our benchtop system includes a forward and inverse pass through a free-space programmable spatial mode sorter, designed to isolate photons in a point spread function (PSF)-adapted basis. During the forward pass, the fundamental mode is rejected, effectively eliminating light from an on-axis point-like star. On the inverse pass, the remaining modes are coherently recombined, enabling direct imaging of a faint companion. We develop a probabilistic measurement model that accounts for combined effects of fundamental shot noise and experimental noise specific to our benchtop setup, such as modal cross-talk, dark noise, and ambient background illumination. We leverage this measurement model to formulate a maximum-likelihood estimator of the exoplanet position given an image captured with the coronagraph. Using this approach, we successfully localize an artificial exoplanet at sub-diffraction distances $(<\sigma)$ from its host star under a 1000:1 star-planet contrast ratio. Our system accurately localizes the exoplanet up to an absolute error $<0.03\sigma$ over the separation range $[0,\,0.6]\sigma$. Finally, we numerically evaluate the precision of our experimental coronagraph against state-of-the-art coronagraphs subject to comparable noise models., Comment: 11 pages, 10 figures

In active imaging protocols, information about a landscape is encoded into the spatial mode of a scattered photon. A common assumption is that the landscape is rigid; however, in principle it can be altered by radiation pressure, a concept that has found fruitful application in the field of quantum optomechanics. Here we explore active imaging of a mechanical resonator with an eye to generalizing the concept of radiation pressure backaction to spatially multimode light. As a thought experiment, we consider imaging the flexural modes of a membrane by sorting the spatial modes of a laser reflected from its surface. We show that backaction in this setting arises from spatial photon shot noise, an effect that cannot be observed in single-mode optomechanics. We also derive the imprecision-backaction product for coherent illumination in the limit of purely spatial backaction, revealing it to be equivalent to the standard quantum limit for purely dispersive, single-mode optomechanical coupling. Finally, we show that optomechanical correlations due to spatial backaction can give rise to two-mode entangled light. In conjunction with high-$Q$ nanomechanics, our findings point to new opportunities at the interface of quantum imaging and optomechanics, including sensors and networks enhanced by spatial mode entanglement., Comment: 10 pages, 5 figures

Microarcsecond resolutions afforded by an optical-NIR array with kilometer-baselines would enable breakthrough science. However significant technology barriers exist in transporting weakly coherent photon states over these distances: primarily photon loss and phase errors. Quantum telescopy, using entangled states to link spatially separated apertures, offers a possible solution to the loss of photons. We report on an initiative launched by NSF NOIRLab in collaboration with the Center for Quantum Networks and Arizona Quantum Initiative at the University of Arizona, Tucson, to explore these concepts further. A brief description of the quantum concepts and a possible technology roadmap towards a quantum-enhanced very long baseline optical-NIR interferometric array is presented. An on-sky demonstration of measuring spatial coherence of photons with apertures linked through the simplest Gottesman protocol over short baselines and with limited phase fluctuations is envisaged as the first step., Comment: Proceeding of SPIE Conference "Astronomical Telescopes + Instrumentation" (June 2024)

We present long-slit spectroscopic observations of 40 Galaxy On Top Of Quasars (GOTOQs) at ${0.37 \leqslant z \leqslant 1.01}$ using the South African Large Telescope. Using this and available photometric data, we measure the impact parameters of the foreground galaxies to be in the range of 3$-$16 kpc with a median value of 8.6 kpc. This is the largest sample of galaxies producing MgII absorption at such low impact parameters. These quasar-galaxy pairs are ideal for probing the disk-halo interface. At such impact parameters, we do not find any significant anti-correlation between rest equivalent width (REW) of CaII, MnII, FeII, MgII, and MgI absorptions and impact parameters. These sight lines are typically redder than those of strong MgII absorbers, with the color excess, E(B$-$V) for our sample ranging from $-$0.191 to 0.422, with a median value of 0.058. In the E(B$-$V) vs. W$_{3935}$ plane, GOTOQs occupy the same region as CaII absorbers. For a given E(B$-$V), we find larger W$_{3935}$ than what has been found in the Milky Way, probably due to a smaller dust-to-gas ratio in GOTOQs. Galaxy parameters could be measured for twelve cases, and their properties seem to follow the trends found for strong MgII absorbers. Measuring the host galaxy properties for the full sample using HST photometry or AO-assisted ground-based imaging is important to gain insights into the relationship between the stellar mass of galaxies and the metal line REW distributions at low impact parameters., Comment: Accepted for publication in MNRAS. 9 figures and 14 pages

3D building reconstruction from imaging data is an important task for many applications ranging from urban planning to reconnaissance. Modern Novel View synthesis (NVS) methods like NeRF and Gaussian Splatting offer powerful techniques for developing 3D models from natural 2D imagery in an unsupervised fashion. These algorithms generally require input training views surrounding the scene of interest, which, in the case of large buildings, is typically not available across all camera elevations. In particular, the most readily available camera viewpoints at scale across most buildings are at near-ground (e.g., with mobile phones) and aerial (drones) elevations. However, due to the significant difference in viewpoint between drone and ground image sets, camera registration - a necessary step for NVS algorithms - fails. In this work we propose a method, DRAGON, that can take drone and ground building imagery as input and produce a 3D NVS model. The key insight of DRAGON is that intermediate elevation imagery may be extrapolated by an NVS algorithm itself in an iterative procedure with perceptual regularization, thereby bridging the visual feature gap between the two elevations and enabling registration. We compiled a semi-synthetic dataset of 9 large building scenes using Google Earth Studio, and quantitatively and qualitatively demonstrate that DRAGON can generate compelling renderings on this dataset compared to baseline strategies., Comment: 12 pages, 9 figures, accepted to ICCP 2024

Generating ambient sounds is a challenging task due to data scarcity and often insufficient caption quality, making it difficult to employ large-scale generative models for the task. In this work, we tackle this problem by introducing two new models. First, we propose AutoCap, a high-quality and efficient automatic audio captioning model. By using a compact audio representation and leveraging audio metadata, AutoCap substantially enhances caption quality, reaching a CIDEr score of 83.2, marking a 3.2% improvement from the best available captioning model at four times faster inference speed. Second, we propose GenAu, a scalable transformer-based audio generation architecture that we scale up to 1.25B parameters. Using AutoCap to generate caption clips from existing audio datasets, we demonstrate the benefits of data scaling with synthetic captions as well as model size scaling. When compared to state-of-the-art audio generators trained at similar size and data scale, GenAu obtains significant improvements of 4.7% in FAD score, 22.7% in IS, and 13.5% in CLAP score, indicating significantly improved quality of generated audio compared to previous works. Moreover, we propose an efficient and scalable pipeline for collecting audio datasets, enabling us to compile 57M ambient audio clips, forming AutoReCap-XL, the largest available audio-text dataset, at 90 times the scale of existing ones. Our code, model checkpoints, and dataset are publicly available., Comment: Project Webpage: https://snap-research.github.io/GenAU/

Face detection is frequently attempted by using heavy pre-trained backbone networks like ResNet-50/101/152 and VGG16/19. Few recent works have also proposed lightweight detectors with customized backbones, novel loss functions and efficient training strategies. The novelty of this work lies in the design of a lightweight detector while training with only the commonly used loss functions and learning strategies. The proposed face detector grossly follows the established RetinaFace architecture. The first contribution of this work is the design of a customized lightweight backbone network (BLite) having 0.167M parameters with 0.52 GFLOPs. The second contribution is the use of two independent multi-task losses. The proposed lightweight face detector (FDLite) has 0.26M parameters with 0.94 GFLOPs. The network is trained on the WIDER FACE dataset. FDLite is observed to achieve 92.3\%, 89.8\%, and 82.2\% Average Precision (AP) on the easy, medium, and hard subsets of the WIDER FACE validation dataset, respectively., Comment: 10 pages, 14 figures

Underwater image quality is affected by fluorescence, low illumination, absorption, and scattering. Recent works in underwater image enhancement have proposed different deep network architectures to handle these problems. Most of these works have proposed a single network to handle all the challenges. We believe that deep networks trained for specific conditions deliver better performance than a single network learned from all degradation cases. Accordingly, the first contribution of this work lies in the proposal of an iterative framework where a single dominant degradation condition is identified and resolved. This proposal considers the following eight degradation conditions -- low illumination, low contrast, haziness, blurred image, presence of noise and color imbalance in three different channels. A deep network is designed to identify the dominant degradation condition. Accordingly, an appropriate deep network is selected for degradation condition-specific enhancement. The second contribution of this work is the construction of degradation condition specific datasets from good quality images of two standard datasets (UIEB and EUVP). This dataset is used to learn the condition specific enhancement networks. The proposed approach is found to outperform nine baseline methods on UIEB and EUVP datasets.

It has been shown that shared entanglement between two telescope sites can in principle be used to localize a point source by mimicking the standard phase-scanning interferometer, but without physically bringing the light from the distant telescopes together. In this paper, we show that a receiver that employs spatial-mode sorting at each telescope site, combined with pre-shared entanglement and local quantum operations can be used to mimic the most general multimode interferometer acting on light collected from the telescopes. As an example application to a quantitative passive-imaging problem, we show that the quantum-limited precision of estimating the angular separation between two stars can be attained by an instantiation of the aforesaid entanglement based receiver. We discuss how this entanglement assisted strategy can be used to achieve the quantum-limited precision of any complex quantitative imaging task involving any number of telescopes. We provide a blueprint of this general receiver that involves quantum transduction of starlight into quantum memory banks and spatial mode sorters deployed at each telescope site, and measurements that include optical detection as well as qubit gates and measurements on the quantum memories. We discuss the relative contributions of local mode sorting at telescope sites vis-a-vis distributed entanglement-assisted interferometry, to the overall quantum-limited information about the scene, based on the ratio of the baseline distance to the individual telescope diameter., Comment: 16 pages, 3 figures

Artificial Intelligence with its multifaceted technologies and integral role in global production significantly impacts gender dynamics particularly in gendered labor. This paper emphasizes the need to explore AIs broader impacts on gendered labor beyond its current emphasis on the generation and perpetuation of epistemic biases. We draw attention to how the AI industry as an integral component of the larger economic structure is transforming the nature of work. It is expanding the prevalence of platform based work models and exacerbating job insecurity particularly for women. Of critical concern is the increasing exclusion of women from meaningful engagement in the digital labor force. This issue often overlooked demands urgent attention from the AI research community. Understanding AIs multifaceted role in gendered labor requires a nuanced examination of economic transformation and its implications for gender equity. By shedding light on these intersections this paper aims to stimulate in depth discussions and catalyze targeted actions aimed at mitigating the gender disparities accentuated by AI driven transformations., Comment: Under review. An unindexed peer-reviewed working draft was accepted for presentation at IJCAI 2021 Workshop on AI for Social Good organized by Harvard CRCS

Multimodal interleaved datasets featuring free-form interleaved sequences of images and text are crucial for training frontier large multimodal models (LMMs). Despite the rapid progression of open-source LMMs, there remains a pronounced scarcity of large-scale, diverse open-source multimodal interleaved datasets. In response, we introduce MINT-1T, the most extensive and diverse open-source Multimodal INTerleaved dataset to date. MINT-1T comprises one trillion text tokens and 3.4 billion images, a 10x scale-up from existing open-source datasets. Additionally, we include previously untapped sources such as PDFs and ArXiv papers. As scaling multimodal interleaved datasets requires substantial engineering effort, sharing the data curation process and releasing the dataset greatly benefits the community. Our experiments show that LMMs trained on MINT-1T rival the performance of models trained on the previous leading dataset, OBELICS. Our data and code will be released at https://github.com/mlfoundations/MINT-1T.

We introduce DataComp for Language Models (DCLM), a testbed for controlled dataset experiments with the goal of improving language models. As part of DCLM, we provide a standardized corpus of 240T tokens extracted from Common Crawl, effective pretraining recipes based on the OpenLM framework, and a broad suite of 53 downstream evaluations. Participants in the DCLM benchmark can experiment with data curation strategies such as deduplication, filtering, and data mixing at model scales ranging from 412M to 7B parameters. As a baseline for DCLM, we conduct extensive experiments and find that model-based filtering is key to assembling a high-quality training set. The resulting dataset, DCLM-Baseline enables training a 7B parameter language model from scratch to 64% 5-shot accuracy on MMLU with 2.6T training tokens. Compared to MAP-Neo, the previous state-of-the-art in open-data language models, DCLM-Baseline represents a 6.6 percentage point improvement on MMLU while being trained with 40% less compute. Our baseline model is also comparable to Mistral-7B-v0.3 and Llama 3 8B on MMLU (63% & 66%), and performs similarly on an average of 53 natural language understanding tasks while being trained with 6.6x less compute than Llama 3 8B. Our results highlight the importance of dataset design for training language models and offer a starting point for further research on data curation., Comment: Project page: https://www.datacomp.ai/dclm/

Gottesman-Kitaev-Preskill (GKP) states have been demonstrated to pose significant advantages when utilized for fault-tolerant all optical continuous-variable quantum computing as well as for quantum communications links for entanglement distribution. However interfacing these systems to long-lived solid-state quantum memories has remained an open problem. Here we propose an interface between quantum memories and GKP qubit states based on a cavity-mediated controlled displacement gate. We characterize the quality of memory-GKP entanglement as a function of cavity parameters suggesting optimal regimes of operation for high-quality state transfer between either qubit states. We further extend this protocol to demonstrate the creation of GKP cluster states by avoiding the requirement of ancillary optical quadrature-squeezed light. Utilizing post-selected entanglement swapping operations for GKP qubits, we demonstrate the utility of our protocol for high-rate entanglement generation between quantum memories. Extensions and derivatives of our proposal could enable a wide variety of applications by utilizing the operational trade-offs for qubits encoded in memory and in the GKP basis., Comment: 17 pages; 8 figures; Comments are welcome!

Entangling quantum memories, mediated by optical-frequency or microwave channels, at high rates and fidelities is key for linking qubits across short and long ranges. All well-known protocols encode up to one qubit per optical mode, hence entangling one pair of memory qubits per transmitted mode over the channel, with probability $\eta$, the channel's transmissivity. The rate is proportional to $\eta$ ideal Bell states (ebits) per mode. The quantum capacity, $C(\eta) = -\log_2(1-{\eta})$ ebits per mode, which $\approx 1.44\eta$ for high loss, i.e., $\eta \ll 1$, thereby making these schemes near rate-optimal. However, $C(\eta) \to \infty$ as $\eta \to 1$, making the known schemes highly rate-suboptimal for shorter ranges. We show that a cavity-assisted memory-photon interface can be used to entangle matter memories with Gottesman-Kitaev-Preskill (GKP) photonic qudits, which along with dual-homodyne entanglement swaps that retain analog information, enables entangling memories at capacity-approaching rates at low loss. We benefit from loss resilience of GKP qudits, and their ability to encode multiple qubits in one mode. Our memory-photon interface further supports the preparation of needed ancilla GKP qudits. We expect our result to spur research in low-loss high-cooperativity cavity-coupled qubits with high-efficiency optical coupling, and demonstrations of high-rate short-range quantum links., Comment: 16 pages; 8 figures; Comments are welcome!

Stabilizer states are a prime resource for a number of applications in quantum information science, such as secret-sharing and measurement-based quantum computation. This motivates us to study the entanglement of noisy stabilizer states across a bipartition. We show that the spectra of the corresponding reduced states can be expressed in terms of properties of an associated stabilizer code. In particular, this allows us to show that the coherent information is related to the so-called syndrome entropy of the underlying code. We use this viewpoint to find stabilizer states that are resilient against noise, allowing for more robust entanglement distribution in near-term quantum networks. We specialize our results to the case of graph states, where the found connections with stabilizer codes reduces back to classical linear codes for dephasing noise. On our way we provide an alternative proof of the fact that every qubit stabilizer code is equivalent up to single-qubit Clifford gates to a graph code., Comment: 7 pages, 4 figures

Defects and dopant atoms in solid state materials are a promising platform for realizing single photon sources and quantum memories, which are the basic building blocks of quantum repeaters needed for long distance quantum networks. In particular, trivalent erbium (Er$^{3+}$) is of interest because it couples C-band telecom optical transitions with a spin-based memory platform. In order to produce quantum repeaters at the scale required for a quantum internet, it is imperative to integrate these necessary building blocks with mature and scalable semiconductor processes. In this work, we demonstrate the optical isolation of single Er$^{3+}$ ions in CMOS-compatible titanium dioxide (TiO$_2$) thin films monolithically integrated on a silicon-on-insulator (SOI) photonics platform. Our results demonstrate a first step toward the realization of a monolithically integrated and scalable quantum photonics package based on Er$^{3+}$ doped thin films.

Commercially impactful quantum algorithms such as quantum chemistry and Shor's algorithm require a number of qubits and gates far beyond the capacity of any existing quantum processor. Distributed architectures, which scale horizontally by networking modules, provide a route to commercial utility and will eventually surpass the capability of any single quantum computing module. Such processors consume remote entanglement distributed between modules to realize distributed quantum logic. Networked quantum computers will therefore require the capability to rapidly distribute high fidelity entanglement between modules. Here we present preliminary demonstrations of some key distributed quantum computing protocols on silicon T centres in isotopically-enriched silicon. We demonstrate the distribution of entanglement between modules and consume it to apply a teleported gate sequence, establishing a proof-of-concept for T centres as a distributed quantum computing and networking platform., Comment: 14 pages, 13 figures

Open-weight LLMs are particularly appealing choices to generate training data for fine-tuning Code LLMs on domain-specific service robot applications because they are cost-effective, customizable, and offer better privacy protection. However, unlike proprietary LLMs, open-weight models are more error-prone and often produce programs that violate domain-specific constraints. A promising solution is to incorporate a robot simulator with a well-defined environment to verify program correctness. Yet, these environments require pre-enumeration of relevant entities and their states, which limits the diversity of programs that can be effectively verified. In this work, we introduce ROBO-INSTRUCT that preserves the diversity of programs generated by an LLM while providing the correctness of simulator-based checking. ROBO-INSTRUCT introduces ROBOSIM to dynamically synthesize consistent simulation environments for each generated program. Moreover, ROBO-INSTRUCT handles subtler instruction-program inconsistencies that do not result in a constraint violation via INSTALIGN, an LLM-aided instruction-program alignment process. Given domain-specific APIs and a few seed examples, ROBO-INSTRUCT can leverage an 8B Llama3 model to generate a training dataset for fine-tuning a 7B CodeLlama model. Our fine-tuned model achieves a 28.75% improvement in pass@1 over the original base model and a 13.75% improvement compared to its SELF-INSTRUCT-finetuned counterparts, even surpassing the performance of a few proprietary LLMs, such as GPT-3.5-Turbo and Gemini-Pro.

We propose a new type of multi-bit and energy-efficient magnetic memory based on current-driven, field-free, and highly controlled domain wall motion. A meandering domain wall channel with precisely interspersed pinning regions provides the multi-bit capability of a magnetic tunnel junction. The magnetic free layer of the memory device has perpendicular magnetic anisotropy and interfacial Dzyaloshinskii-Moriya interaction, so that spin-orbit torques induce efficient domain wall motion. Using micromagnetic simulations, we find two pinning mechanisms that lead to different cell designs: two-way switching and four-way switching. The memory cell design choices and the physics behind these pinning mechanisms are discussed in detail. Furthermore, we show that switching reliability and speed may be significantly improved by replacing the ferromagnetic free layer with a synthetic antiferromagnetic layer. Switching behavior and material choices will be discussed for the two implementations., Comment: 24 pages

Stochastic two-player games model systems with an environment that is both adversarial and stochastic. In this paper, we study the expected value of quantitative prefix-independent objectives in stochastic games. We show a generic reduction from the expectation problem to linearly many instances of almost-sure satisfaction of threshold Boolean objectives. The result follows from partitioning the vertices of the game into so-called value classes where each class consists of vertices of the same value. Our procedure further entails that the memory required by both players to play optimally for the expectation problem is no more than the memory required by the players to play optimally for the almost-sure satisfaction problem for a corresponding threshold Boolean objective. We show the applicability of the framework to compute the expected window mean-payoff measure in stochastic games. The window mean-payoff measure strengthens the classical mean-payoff measure by computing the mean-payoff over a window of bounded length that slides along an infinite path. Two variants have been considered: in one variant, the maximum window length is fixed and given, while in the other, it is not fixed but is required to be bounded. For both variants, we show that the decision problem to check if the expected value is at least a given threshold is in UP $\cap$ coUP. The result follows from guessing the expected values of the vertices, partitioning them into value classes, and proving that a unique short certificate for the expected values exists. It also follows that the memory required by the players to play optimally is no more than that in non-stochastic two-player games with the corresponding window objectives., Comment: 30 pages, 4 figures. Fixed bugs in proofs. Updated the results and presentation to be applicable to prefix-independent objectives in general