Search

Wildfires propagate through intricate interactions between wind, fuel, and terrain, resulting in complex behaviors that pose challenges for accurate predictions. This study investigates the interaction between wind velocity topology and wildfire spread dynamics, aiming to enhance our understanding of wildfire spread patterns. We revisited the non-dimensionalizion of the governing combustion model by incorporating three distinct time scales. This approach revealed two new non-dimensional numbers, contrasting with the conventional non-dimensionalization that considers only a single time scale. Through scaling analysis, we analytically identified the critical determinants of transient wildfire behavior and established a state-neutral curve, indicating where initial wildfires extinguish for specific combinations of the identified non-dimensional numbers. Subsequently, a wildfire transport solver was developed using a finite difference method, integrating compact schemes and implicit-explicit Runge-Kutta methods. We explored the influence of stable and unstable manifolds in wind velocity on wildfire transport under steady wind conditions defined using a saddle-type fixed point flow, emphasizing the role of the non-dimensional numbers. Additionally, we considered the benchmark unsteady double-gyre flow and examined the effect of unsteady wind topology on wildfire propagation, and quantified the wildfire response to varying wind oscillation frequencies and amplitudes using a transfer function approach. The results were compared to Lagrangian coherent structures (LCS) used to characterize the correspondence of manifolds with wildfire propagation. The comprehensive approach of utilizing the manifolds computed from wind topology provides valuable insights into wildfire dynamics across diverse wind scenarios, offering a potential tool for improved predictive modeling and management strategies.

Multimodal foundation models offer a promising framework for robotic perception and planning by processing sensory inputs to generate actionable plans. However, addressing uncertainty in both perception (sensory interpretation) and decision-making (plan generation) remains a critical challenge for ensuring task reliability. We present a comprehensive framework to disentangle, quantify, and mitigate these two forms of uncertainty. We first introduce a framework for uncertainty disentanglement, isolating perception uncertainty arising from limitations in visual understanding and decision uncertainty relating to the robustness of generated plans. To quantify each type of uncertainty, we propose methods tailored to the unique properties of perception and decision-making: we use conformal prediction to calibrate perception uncertainty and introduce Formal-Methods-Driven Prediction (FMDP) to quantify decision uncertainty, leveraging formal verification techniques for theoretical guarantees. Building on this quantification, we implement two targeted intervention mechanisms: an active sensing process that dynamically re-observes high-uncertainty scenes to enhance visual input quality and an automated refinement procedure that fine-tunes the model on high-certainty data, improving its capability to meet task specifications. Empirical validation in real-world and simulated robotic tasks demonstrates that our uncertainty disentanglement framework reduces variability by up to 40% and enhances task success rates by 5% compared to baselines. These improvements are attributed to the combined effect of both interventions and highlight the importance of uncertainty disentanglement which facilitates targeted interventions that enhance the robustness and reliability of autonomous systems., Comment: Fine-tuned models, code, and datasets are available at https://tinyurl.com/uncertainty-disentanglement

Eye tracking technology offers great potential for improving road safety. It is already being built into vehicles, namely cars and trucks. When this technology is integrated into transit service vehicles, employees, i.e., bus drivers, will be subject to being eye tracked on their job. Although there is much research effort advancing algorithms for eye tracking in transportation, less is known about how end users perceive this technology, especially when interacting with it in an employer-mandated context. In this first study of its kind, we investigated transit bus operators' perceptions of eye tracking technology. From a methodological perspective, we introduce a mixed methods approach where participants experience the technology first-hand and then reflect on their experience while viewing a playback of the recorded data. Thematic analysis of the interview transcripts reveals interesting potential uses of eye tracking in this work context and surfaces transit operators' fears and concerns about this technology.

Motivated by engineering applications such as resource allocation in networks and inventory systems, we consider average-reward Reinforcement Learning with unbounded state space and reward function. Recent works studied this problem in the actor-critic framework and established finite sample bounds assuming access to a critic with certain error guarantees. We complement their work by studying Temporal Difference (TD) learning with linear function approximation and establishing finite-time bounds with the optimal $\mathcal{O}\left(1/\epsilon^2\right)$ sample complexity. These results are obtained using the following general-purpose theorem for non-linear Stochastic Approximation (SA). Suppose that one constructs a Lyapunov function for a non-linear SA with certain drift condition. Then, our theorem establishes finite-time bounds when this SA is driven by unbounded Markovian noise under suitable conditions. It serves as a black box tool to generalize sample guarantees on SA from i.i.d. or martingale difference case to potentially unbounded Markovian noise. The generality and the mild assumption of the setup enables broad applicability of our theorem. We illustrate its power by studying two more systems: (i) We improve upon the finite-time bounds of $Q$-learning by tightening the error bounds and also allowing for a larger class of behavior policies. (ii) We establish the first ever finite-time bounds for distributed stochastic optimization of high-dimensional smooth strongly convex function using cyclic block coordinate descent.

Informative path planning (IPP) is an important planning paradigm for various real-world robotic applications such as environment monitoring. IPP involves planning a path that can learn an accurate belief of the quantity of interest, while adhering to planning constraints. Traditional IPP methods typically require high computation time during execution, giving rise to reinforcement learning (RL) based IPP methods. However, the existing RL-based methods do not consider spatio-temporal environments which involve their own challenges due to variations in environment characteristics. In this paper, we propose DyPNIPP, a robust RL-based IPP framework, designed to operate effectively across spatio-temporal environments with varying dynamics. To achieve this, DyPNIPP incorporates domain randomization to train the agent across diverse environments and introduces a dynamics prediction model to capture and adapt the agent actions to specific environment dynamics. Our extensive experiments in a wildfire environment demonstrate that DyPNIPP outperforms existing RL-based IPP algorithms by significantly improving robustness and performing across diverse environment conditions., Comment: 8 pages, 4 figures, submitted to IEEE RA-L

Identifying dynamical systems characterized by nonlinear parameters presents significant challenges in deriving mathematical models that enhance understanding of physics. Traditional methods, such as Sparse Identification of Nonlinear Dynamics (SINDy) and symbolic regression, can extract governing equations from observational data; however, they also come with distinct advantages and disadvantages. This paper introduces a novel method within the SINDy framework, termed ADAM-SINDy, which synthesizes the strengths of established approaches by employing the ADAM optimization algorithm. This facilitates the simultaneous optimization of nonlinear parameters and coefficients associated with nonlinear candidate functions, enabling precise parameter estimation without requiring prior knowledge of nonlinear characteristics such as trigonometric frequencies, exponential bandwidths, or polynomial exponents, thereby addressing a key limitation of SINDy. Through an integrated global optimization, ADAM-SINDy dynamically adjusts all unknown variables in response to data, resulting in an adaptive identification procedure that reduces the sensitivity to the library of candidate functions. The performance of the ADAM-SINDy methodology is demonstrated across a spectrum of dynamical systems, including benchmark coupled nonlinear ordinary differential equations such as oscillators, chaotic fluid flows, reaction kinetics, pharmacokinetics, as well as nonlinear partial differential equations (wildfire transport). The results demonstrate significant improvements in identifying parameterized dynamical systems and underscore the importance of concurrently optimizing all parameters, particularly those characterized by nonlinear parameters. These findings highlight the potential of ADAM-SINDy to extend the applicability of the SINDy framework in addressing more complex challenges in dynamical system identification.

Many problems that cloud operators solve are computationally expensive, and operators often use heuristic algorithms (that are faster and scale better than optimal) to solve them more efficiently. Heuristic analyzers enable operators to find when and by how much their heuristics underperform. However, these tools do not provide enough detail for operators to mitigate the heuristic's impact in practice: they only discover a single input instance that causes the heuristic to underperform (and not the full set), and they do not explain why. We propose XPlain, a tool that extends these analyzers and helps operators understand when and why their heuristics underperform. We present promising initial results that show such an extension is viable.

Dust from core-collapse supernovae (CCSNe), specifically Type IIP SNe, has been suggested to be a significant source of the dust observed in high-redshift galaxies. CCSNe eject large amounts of newly formed heavy elements, which can condense into dust grains in the cooling ejecta. However, infrared (IR) observations of typical CCSNe generally measure dust masses that are too small to account for the dust production needed at high redshifts. Type IIn SNe, classified by their dense circumstellar medium (CSM), are also known to exhibit strong IR emission from warm dust, but the dust origin and heating mechanism have generally remained unconstrained because of limited observational capabilities in the mid-IR. Here, we present a JWST/MIRI Medium Resolution Spectrograph (MRS) spectrum of the Type IIn SN 2005ip nearly 17 years post-explosion. The Type IIn SN 2005ip is one of the longest-lasting and most well-studied SNe observed to date. Combined with a Spitzer mid-IR spectrum of SN 2005ip obtained in 2008, this data set provides a rare 15-year baseline, allowing for a unique investigation of the evolution of dust. The JWST spectrum shows a new high-mass dust component ($\gtrsim0.08$ M$_{\odot}$) that is not present in the earlier Spitzer spectrum. Our analysis shows dust likely formed over the past 15 years in the cold, dense shell (CDS), between the forward and reverse shocks. There is also a smaller mass of carbonaceous dust ($\gtrsim0.005$ M$_{\odot}$) in the ejecta. These observations provide new insights into the role of SN dust production, particularly within the CDS, and its potential contribution to the rapid dust enrichment of the early Universe.

Consider the stochastic heat equation \begin{equation*} \partial_t u_t(x)=\frac12 \partial^2_{xx}u_t(x) +b(u_t(x))+\dot{W}_{t}(x),\quad t\in(0,T],\, x\in [0,1], \end{equation*} where $b$ is a generalized function, and $\dot W$ is space-time white noise on ${\mathbb R}_+\times[0,1]$. If the drift $b$ is a sufficiently regular function, then it is well-known that any analytically weak solution to this equation is also analytically mild, and vice versa. We extend this result to drifts that are generalized functions, with an appropriate adaptation of the notions of mild and weak solutions. As a corollary of our results, we show that for $b\in L_p({\mathbb R})$, $p\ge1$, this equation has a unique analytically weak and mild solution, thus extending the classical results of Gy\"ongy and Pardoux (1993).

Large language models (LLMs) are increasingly applied to complex reasoning tasks that require executing several complex steps before receiving any reward. Properly assigning credit to these steps is essential for enhancing model performance. Proximal Policy Optimization (PPO), a state-of-the-art reinforcement learning (RL) algorithm used for LLM finetuning, employs value networks to tackle credit assignment. However, value networks face challenges in predicting the expected cumulative rewards accurately in complex reasoning tasks, often leading to high-variance updates and suboptimal performance. In this work, we systematically evaluate the efficacy of value networks and reveal their significant shortcomings in reasoning-heavy LLM tasks, showing that they barely outperform a random baseline when comparing alternative steps. To address this, we propose VinePPO, a straightforward approach that leverages the flexibility of language environments to compute unbiased Monte Carlo-based estimates, bypassing the need for large value networks. Our method consistently outperforms PPO and other RL-free baselines across MATH and GSM8K datasets with fewer gradient updates (up to 9x), less wall-clock time (up to 3.0x). These results emphasize the importance of accurate credit assignment in RL finetuning of LLM and demonstrate VinePPO's potential as a superior alternative.

Generative Artificial Intelligence (GenAI) is becoming ubiquitous in our daily lives. The increase in computational power and data availability has led to a proliferation of both single- and multi-modal models. As the GenAI ecosystem matures, the need for extensible and model-agnostic risk identification frameworks is growing. To meet this need, we introduce the Python Risk Identification Toolkit (PyRIT), an open-source framework designed to enhance red teaming efforts in GenAI systems. PyRIT is a model- and platform-agnostic tool that enables red teamers to probe for and identify novel harms, risks, and jailbreaks in multimodal generative AI models. Its composable architecture facilitates the reuse of core building blocks and allows for extensibility to future models and modalities. This paper details the challenges specific to red teaming generative AI systems, the development and features of PyRIT, and its practical applications in real-world scenarios.

Informative path planning (IPP) is a crucial task in robotics, where agents must design paths to gather valuable information about a target environment while adhering to resource constraints. Reinforcement learning (RL) has been shown to be effective for IPP, however, it requires environment interactions, which are risky and expensive in practice. To address this problem, we propose an offline RL-based IPP framework that optimizes information gain without requiring real-time interaction during training, offering safety and cost-efficiency by avoiding interaction, as well as superior performance and fast computation during execution -- key advantages of RL. Our framework leverages batch-constrained reinforcement learning to mitigate extrapolation errors, enabling the agent to learn from pre-collected datasets generated by arbitrary algorithms. We validate the framework through extensive simulations and real-world experiments. The numerical results show that our framework outperforms the baselines, demonstrating the effectiveness of the proposed approach., Comment: 7 pages, 6 figures, submitted to ICRA 2025

Ensuring robust performance on long-tail examples is an important problem for many real-world applications of machine learning, such as autonomous driving. This work focuses on the problem of identifying rare examples within a corpus of unlabeled data. We propose a simple and scalable data mining approach that leverages the knowledge contained within a large vision language model (VLM). Our approach utilizes a VLM to summarize the content of an image into a set of keywords, and we identify rare examples based on keyword frequency. We find that the VLM offers a distinct signal for identifying long-tail examples when compared to conventional methods based on model uncertainty. Therefore, we propose a simple and general approach for integrating signals from multiple mining algorithms. We evaluate the proposed method on two diverse tasks: 2D image classification, in which inter-class variation is the primary source of data diversity, and on 3D object detection, where intra-class variation is the main concern. Furthermore, through the detection task, we demonstrate that the knowledge extracted from 2D images is transferable to the 3D domain. Our experiments consistently show large improvements (between 10\% and 50\%) over the baseline techniques on several representative benchmarks: ImageNet-LT, Places-LT, and the Waymo Open Dataset.

Source enumeration, the task of estimating the number of sources from the signal received by the array of antennas, is a critical problem in array signal processing. Numerous methods have been proposed to estimate the number of sources under white or colored Gaussian noise. However, their performance degrades significantly in the presence of a limited number of observations and/or a large number of sources. In this work, we propose a method based on the distribution of angles that performs well in (a) independent Gaussian, (b) spatially colored Gaussian, and (c) heavy-tailed noise, even when the number of sources is large. We support the supremacy of our algorithm over state-of-the-art methods with extensive simulation results.

We consider the problem of estimating the asymptotic variance of a function defined on a Markov chain, an important step for statistical inference of the stationary mean. We design a novel recursive estimator that requires $O(1)$ computation at each step, does not require storing any historical samples or any prior knowledge of run-length, and has optimal $O(\frac{1}{n})$ rate of convergence for the mean-squared error (MSE) with provable finite sample guarantees. Here, $n$ refers to the total number of samples generated. Our estimator is based on linear stochastic approximation of an equivalent formulation of the asymptotic variance in terms of the solution of the Poisson equation. We generalize our estimator in several directions, including estimating the covariance matrix for vector-valued functions, estimating the stationary variance of a Markov chain, and approximately estimating the asymptotic variance in settings where the state space of the underlying Markov chain is large. We also show applications of our estimator in average reward reinforcement learning (RL), where we work with asymptotic variance as a risk measure to model safety-critical applications. We design a temporal-difference type algorithm tailored for policy evaluation in this context. We consider both the tabular and linear function approximation settings. Our work paves the way for developing actor-critic style algorithms for variance-constrained RL., Comment: 62 pages, 1 table, added additional references

A fully-discrete, nonlinearly-stable flux reconstruction (FD-NSFR) scheme is developed, which ensures robustness through entropy stability in both space and time for high-order flux reconstruction schemes. We extend the entropy-stable flux reconstruction semidiscretization of Cicchino et al. [1,2,3] with the relaxation Runge Kutta method to construct the FD-NSFR scheme. We focus our study on entropy-stable flux reconstruction methods, which allow a larger time step size than discontinuous Galerkin. In this work, we develop an FD-NSFR scheme that prevents temporal numerical entropy change in the broken Sobolev norm if the governing equations admit a convex entropy function that can be expressed in inner-product form. For governing equations with a general convex numerical entropy function, temporal entropy change in the physical $L_2$ norm is prevented. As a result, for general convex numerical entropy, the FD-NSFR scheme achieves fully-discrete entropy stability only when the DG correction function is employed. We use entropy-conserving and entropy-stable test cases for the Burgers', Euler, and Navier-Stokes equations to demonstrate that the FD-NSFR scheme prevents temporal numerical entropy change. The FD-NSFR scheme therefore allows for a larger time step size while maintaining the robustness offered by entropy-stable schemes. We find that the FD-NSFR scheme is able to recover both integrated quantities and solution contours at a higher target time-step size than the semi-discretely entropy-stable scheme, suggesting a robustness advantage for low-Mach turbulence simulations., Comment: 47 pages, 23 figures

Amidst the wide popularity of imitation learning algorithms in robotics, their properties regarding hyperparameter sensitivity, ease of training, data efficiency, and performance have not been well-studied in high-precision industry-inspired environments. In this work, we demonstrate the limitations and benefits of prominent imitation learning approaches and analyze their capabilities regarding these properties. We evaluate each algorithm on a complex bimanual manipulation task involving an over-constrained dynamics system in a setting involving multiple contacts between the manipulated object and the environment. While we find that imitation learning is well suited to solve such complex tasks, not all algorithms are equal in terms of handling environmental and hyperparameter perturbations, training requirements, performance, and ease of use. We investigate the empirical influence of these key characteristics by employing a carefully designed experimental procedure and learning environment. Paper website: https://bimanual-imitation.github.io/

This paper introduces LiNR, LinkedIn's large-scale, GPU-based retrieval system. LiNR supports a billion-sized index on GPU models. We discuss our experiences and challenges in creating scalable, differentiable search indexes using TensorFlow and PyTorch at production scale. In LiNR, both items and model weights are integrated into the model binary. Viewing index construction as a form of model training, we describe scaling our system for large indexes, incorporating full scans and efficient filtering. A key focus is on enabling attribute-based pre-filtering for exhaustive GPU searches, addressing the common challenge of post-filtering in KNN searches that often reduces system quality. We further provide multi-embedding retrieval algorithms and strategies for tackling cold start issues in retrieval. Our advancements in supporting larger indexes through quantization are also discussed. We believe LiNR represents one of the industry's first Live-updated model-based retrieval indexes. Applied to out-of-network post recommendations on LinkedIn Feed, LiNR has contributed to a 3% relative increase in professional daily active users. We envisage LiNR as a step towards integrating retrieval and ranking into a single GPU model, simplifying complex infrastructures and enabling end-to-end optimization of the entire differentiable infrastructure through gradient descent.

Recent innovations in language model training have demonstrated that it is possible to create highly performant models that are small enough to run on a smartphone. As these models are deployed in an increasing number of domains, it is critical to ensure that they are aligned with human preferences and safety considerations. In this report, we present our methodology for safety aligning the Phi-3 series of language models. We utilized a "break-fix" cycle, performing multiple rounds of dataset curation, safety post-training, benchmarking, red teaming, and vulnerability identification to cover a variety of harm areas in both single and multi-turn scenarios. Our results indicate that this approach iteratively improved the performance of the Phi-3 models across a wide range of responsible AI benchmarks. Finally, we include additional red teaming strategies and evaluations that were used to test the safety behavior of Phi-3.5-mini and Phi-3.5-MoE, which were optimized for multilingual capabilities.

Foundation models and vision-language pre-training have notably advanced Vision Language Models (VLMs), enabling multimodal processing of visual and linguistic data. However, their performance has been typically assessed on general scene understanding - recognizing objects, attributes, and actions - rather than cultural comprehension. This study introduces CulturalVQA, a visual question-answering benchmark aimed at assessing VLM's geo-diverse cultural understanding. We curate a collection of 2,378 image-question pairs with 1-5 answers per question representing cultures from 11 countries across 5 continents. The questions probe understanding of various facets of culture such as clothing, food, drinks, rituals, and traditions. Benchmarking VLMs on CulturalVQA, including GPT-4V and Gemini, reveals disparity in their level of cultural understanding across regions, with strong cultural understanding capabilities for North America while significantly lower performance for Africa. We observe disparity in their performance across cultural facets too, with clothing, rituals, and traditions seeing higher performances than food and drink. These disparities help us identify areas where VLMs lack cultural understanding and demonstrate the potential of CulturalVQA as a comprehensive evaluation set for gauging VLM progress in understanding diverse cultures., Comment: Accepted to EMNLP 2024 Main Conference

Model training requires significantly more memory, compared with inference. Parameter efficient fine-tuning (PEFT) methods provide a means of adapting large models to downstream tasks using less memory. However, existing methods such as adapters, prompt tuning or low-rank adaptation (LoRA) either introduce latency overhead at inference time or achieve subpar downstream performance compared with full fine-tuning. In this work we propose Random Subspace Adaptation (ROSA), a method that outperforms previous PEFT methods by a significant margin, while maintaining a zero latency overhead during inference time. In contrast to previous methods, ROSA is able to adapt subspaces of arbitrarily large dimension, better approximating full-finetuning. We demonstrate both theoretically and experimentally that this makes ROSA strictly more expressive than LoRA, without consuming additional memory during runtime. As PEFT methods are especially useful in the natural language processing domain, where models operate on scales that make full fine-tuning very expensive, we evaluate ROSA in two common NLP scenarios: natural language generation (NLG) and natural language understanding (NLU) with GPT-2 and RoBERTa, respectively. We show that on almost every GLUE task ROSA outperforms LoRA by a significant margin, while also outperforming LoRA on NLG tasks. Our code is available at https://github.com/rosa-paper/rosa

Rapid progress in general-purpose AI has sparked significant interest in "red teaming," a practice of adversarial testing originating in military and cybersecurity applications. AI red teaming raises many questions about the human factor, such as how red teamers are selected, biases and blindspots in how tests are conducted, and harmful content's psychological effects on red teamers. A growing body of HCI and CSCW literature examines related practices-including data labeling, content moderation, and algorithmic auditing. However, few, if any have investigated red teaming itself. Future studies may explore topics ranging from fairness to mental health and other areas of potential harm. We aim to facilitate a community of researchers and practitioners who can begin to meet these challenges with creativity, innovation, and thoughtful reflection., Comment: Updated with camera-ready version

The application of variational principles for analyzing problems in the physical sciences is widespread. Cantilever-like problems, where one end is subjected to a fixed value and the other end is free, have been less studied in terms of their stability despite their abundance. In this article, we develop the stability conditions for these problems by examining the second variation of the energy functional using the generalized Jacobi condition. This involves computing conjugate points that are determined by solving a set of initial value problems from the linearized equilibrium equations. We apply these conditions to investigate the nonlinear stability of intrinsically curved elastic cantilevers subject to a tip load. Kirchhoff rod theory is employed to model the elastic rod deformations. The role of intrinsic curvature in inducing complex nonlinear phenomena, such as snap-back instability, is particularly emphasized. This snap-back instability is demonstrated using various examples, highlighting its dependence on various system parameters. The presented examples illustrate the potential applications in the design of flexible soft robot arms and innovative mechanisms.

Large Language Models (LLMs) have seen widespread adoption due to their remarkable natural language capabilities. However, when deploying them in real-world settings, it is important to align LLMs to generate texts according to acceptable human standards. Methods such as Proximal Policy Optimization (PPO) and Direct Preference Optimization (DPO) have made significant progress in refining LLMs using human preference data. However, the privacy concerns inherent in utilizing such preference data have yet to be adequately studied. In this paper, we investigate the vulnerability of LLMs aligned using human preference datasets to membership inference attacks (MIAs), highlighting the shortcomings of previous MIA approaches with respect to preference data. Our study has two main contributions: first, we introduce a novel reference-based attack framework specifically for analyzing preference data called PREMIA (\uline{Pre}ference data \uline{MIA}); second, we provide empirical evidence that DPO models are more vulnerable to MIA compared to PPO models. Our findings highlight gaps in current privacy-preserving practices for LLM alignment.

Classification of different object surface material types can play a significant role in the decision-making algorithms for mobile robots and autonomous vehicles. RGB-based scene-level semantic segmentation has been well-addressed in the literature. However, improving material recognition using the depth modality and its integration with SLAM algorithms for 3D semantic mapping could unlock new potential benefits in the robotics perception pipeline. To this end, we propose a complementarity-aware deep learning approach for RGB-D-based material classification built on top of an object-oriented pipeline. The approach further integrates the ORB-SLAM2 method for 3D scene mapping with multiscale clustering of the detected material semantics in the point cloud map generated by the visual SLAM algorithm. Extensive experimental results with existing public datasets and newly contributed real-world robot datasets demonstrate a significant improvement in material classification and 3D clustering accuracy compared to state-of-the-art approaches for 3D semantic scene mapping., Comment: Accepted to IROS 2024

An image editing model should be able to perform diverse edits, ranging from object replacement, changing attributes or style, to performing actions or movement, which require many forms of reasoning. Current general instruction-guided editing models have significant shortcomings with action and reasoning-centric edits. Object, attribute or stylistic changes can be learned from visually static datasets. On the other hand, high-quality data for action and reasoning-centric edits is scarce and has to come from entirely different sources that cover e.g. physical dynamics, temporality and spatial reasoning. To this end, we meticulously curate the AURORA Dataset (Action-Reasoning-Object-Attribute), a collection of high-quality training data, human-annotated and curated from videos and simulation engines. We focus on a key aspect of quality training data: triplets (source image, prompt, target image) contain a single meaningful visual change described by the prompt, i.e., truly minimal changes between source and target images. To demonstrate the value of our dataset, we evaluate an AURORA-finetuned model on a new expert-curated benchmark (AURORA-Bench) covering 8 diverse editing tasks. Our model significantly outperforms previous editing models as judged by human raters. For automatic evaluations, we find important flaws in previous metrics and caution their use for semantically hard editing tasks. Instead, we propose a new automatic metric that focuses on discriminative understanding. We hope that our efforts : (1) curating a quality training dataset and an evaluation benchmark, (2) developing critical evaluations, and (3) releasing a state-of-the-art model, will fuel further progress on general image editing., Comment: NeurIPS 2024 (Dataset & Benchmarks)

Kidney cancer is a global health concern, and accurate assessment of patient frailty is crucial for optimizing surgical outcomes. This paper introduces AI Age Discrepancy, a novel metric derived from machine learning analysis of preoperative abdominal CT scans, as a potential indicator of frailty and postoperative risk in kidney cancer patients. This retrospective study of 599 patients from the 2023 Kidney Tumor Segmentation (KiTS) challenge dataset found that a higher AI Age Discrepancy is significantly associated with longer hospital stays and lower overall survival rates, independent of established factors. This suggests that AI Age Discrepancy may provide valuable insights into patient frailty and could thus inform clinical decision-making in kidney cancer treatment., Comment: 10 pages, 3 figures, 2 tables

A common, yet regular, decision made by people, whether healthy or with any health condition, is to decide what to have in meals like breakfast, lunch, and dinner, consisting of a combination of foods for appetizer, main course, side dishes, desserts, and beverages. However, often this decision is seen as a trade-off between nutritious choices (e.g., low salt and sugar) or convenience (e.g., inexpensive, fast to prepare/obtain, taste better). In this preliminary work, we present a data-driven approach for the novel meal recommendation problem that can explore and balance choices for both considerations while also reasoning about a food's constituents and cooking process. Beyond the problem formulation, our contributions also include a goodness measure, a recipe conversion method from text to the recently introduced multimodal rich recipe representation (R3) format, and learning methods using contextual bandits that show promising results., Comment: 6 pages (including references), 1 figure, 2 tables

As autonomous vehicles continue to revolutionize transportation, addressing challenges posed by adverse weather conditions, particularly during winter, becomes paramount for ensuring safe and efficient operations. One of the most important aspects of a road safety inspection during adverse weather is when a limited lane width can reduce the capacity of the road and raise the risk of serious accidents involving autonomous vehicles. In this research, a method for improving driving challenges on roads in winter conditions, with a model that segments and estimates the width of the road from the perspectives of Uncrewed aerial vehicles and autonomous vehicles. The proposed approach in this article is needed to empower self-driving cars with up-to-date and accurate insights, enhancing their adaptability and decision-making capabilities in winter landscapes.

Semantic and syntactic bootstrapping posit that children use their prior knowledge of one linguistic domain, say syntactic relations, to help later acquire another, such as the meanings of new words. Empirical results supporting both theories may tempt us to believe that these are different learning strategies, where one may precede the other. Here, we argue that they are instead both contingent on a more general learning strategy for language acquisition: joint learning. Using a series of neural visually-grounded grammar induction models, we demonstrate that both syntactic and semantic bootstrapping effects are strongest when syntax and semantics are learnt simultaneously. Joint learning results in better grammar induction, realistic lexical category learning, and better interpretations of novel sentence and verb meanings. Joint learning makes language acquisition easier for learners by mutually constraining the hypotheses spaces for both syntax and semantics. Studying the dynamics of joint inference over many input sources and modalities represents an important new direction for language modeling and learning research in both cognitive sciences and AI, as it may help us explain how language can be acquired in more constrained learning settings.

Polyurethane adhesives are extensively used in manufacturing of the lightweight components. These adhesives are hygroscopic in nature causing the material to degrade their mechanical properties due to ageing under environmental conditions. Polyurethane adhesive shows non-linear rate-dependent behaviour due to viscoelastic properties and assumes nearly incompressible deformation. Finite-strain viscoelastic model is formulated based on the continuum rheological model by combining spring and Maxwell elements in parallel. Nearly incompressible property is considered by decomposing the mechanical free energy into volumetric and isochoric parts. This work focuses on the investigation of the influence of moisture on the fracture behaviour. Phase-field damage model proved to be an efficient method to investigate the fracture behaviour. Therefore, the finite-strain viscoelastic material model is coupled with the phase-field model to investigate fracture behaviour. Material is assumed to be isotropic, therefore the total mechanical energy is considered for bulk degradation. Experiments are conducted on the samples aged for different humid conditions to understand tear strength. These test results are used to identify the phase-field damage material parameters from numerical investigation.

We consider policy optimization methods in reinforcement learning settings where the state space is arbitrarily large, or even countably infinite. The motivation arises from control problems in communication networks, matching markets, and other queueing systems. Specifically, we consider the popular Natural Policy Gradient (NPG) algorithm, which has been studied in the past only under the assumption that the cost is bounded and the state space is finite, neither of which holds for the aforementioned control problems. Assuming a Lyapunov drift condition, which is naturally satisfied in some cases and can be satisfied in other cases at a small cost in performance, we design a state-dependent step-size rule which dramatically improves the performance of NPG for our intended applications. In addition to experimentally verifying the performance improvement, we also theoretically show that the iteration complexity of NPG can be made independent of the size of the state space. The key analytical tool we use is the connection between NPG step-sizes and the solution to Poisson's equation. In particular, we provide policy-independent bounds on the solution to Poisson's equation, which are then used to guide the choice of NPG step-sizes., Comment: 24 pages; 3 figures

Aspect-Based Sentiment Analysis (ABSA) has experienced tremendous expansion and diversity due to various shared tasks spanning several languages and fields and organized via SemEval workshops and Germeval. Nonetheless, a few shortcomings still need to be addressed, such as the lack of low-resource language evaluations and the emphasis on sentence-level analysis. To thoroughly assess ABSA techniques in the context of complete reviews, this research presents a novel task, Review-Level Opinion Aspect Sentiment Target (ROAST). ROAST seeks to close the gap between sentence-level and text-level ABSA by identifying every ABSA constituent at the review level. We extend the available datasets to enable ROAST, addressing the drawbacks noted in previous research by incorporating low-resource languages, numerous languages, and a variety of topics. Through this effort, ABSA research will be able to cover more ground and get a deeper comprehension of the task and its practical application in a variety of languages and domains (https://github.com/RiTUAL-UH/ROAST-ABSA)., Comment: arXiv admin note: text overlap with arXiv:2309.13297

Ordinal Classification (OC) is a widely encountered challenge in Natural Language Processing (NLP), with applications in various domains such as sentiment analysis, rating prediction, and more. Previous approaches to tackle OC have primarily focused on modifying existing or creating novel loss functions that \textbf{explicitly} account for the ordinal nature of labels. However, with the advent of Pretrained Language Models (PLMs), it became possible to tackle ordinality through the \textbf{implicit} semantics of the labels as well. This paper provides a comprehensive theoretical and empirical examination of both these approaches. Furthermore, we also offer strategic recommendations regarding the most effective approach to adopt based on specific settings., Comment: Findings of ACL 2024

Interpretability is the study of explaining models in understandable terms to humans. At present, interpretability is divided into two paradigms: the intrinsic paradigm, which believes that only models designed to be explained can be explained, and the post-hoc paradigm, which believes that black-box models can be explained. At the core of this debate is how each paradigm ensures its explanations are faithful, i.e., true to the model's behavior. This is important, as false but convincing explanations lead to unsupported confidence in artificial intelligence (AI), which can be dangerous. This paper's position is that we should think about new paradigms while staying vigilant regarding faithfulness. First, by examining the history of paradigms in science, we see that paradigms are constantly evolving. Then, by examining the current paradigms, we can understand their underlying beliefs, the value they bring, and their limitations. Finally, this paper presents 3 emerging paradigms for interpretability. The first paradigm designs models such that faithfulness can be easily measured. Another optimizes models such that explanations become faithful. The last paradigm proposes to develop models that produce both a prediction and an explanation.

Solutions to the governing partial differential equations obtained from a discrete numerical scheme can have significant errors, especially near shocks when the discrete representation of the solution cannot fully capture the discontinuity in the solution. A recent approach to shock tracking [1, 2] has been to implicitly align the faces of mesh elements with the shock, yielding accurate solutions on coarse meshes. In engineering applications, the solution field is often used to evaluate a scalar functional of interest, such as lift or drag over an airfoil. While functionals are sensitive to errors in the flow solution, certain regions in the domain are more important for accurate evaluation of the functional than the rest. Using this fact, we formulate a goal-oriented implicit shock tracking approach that captures a segment of the shock that is important for evaluating the functional. Shock tracking is achieved using Lagrange-Newton-Krylov-Schur (LNKS) full space optimizer, with the objective of minimizing the adjoint-weighted residual error indicator. We also present a method to evaluate the sensitivity and the Hessian of the functional error. Using available block preconditioners for LNKS [3, 4] makes the full space approach scalable. The method is applied to test cases of two-dimensional advection and inviscid compressible flows to demonstrate functional-dependent shock tracking. Tracking the entire shock without using artificial dissipation results in the error converging at the orders of $\mathcal{O}(h^{p+1})$., Comment: 46 pages, 30 figures

Network slicing has emerged as an integral concept in 5G, aiming to partition the physical network infrastructure into isolated slices, customized for specific applications. We theoretically formulate the key performance metrics of an application, in terms of goodput and delivery delay, at a cost of network resources in terms of bandwidth. We explore an un-coded communication protocol that uses feedback-based repetitions, and a coded protocol, implementing random linear network coding and using coding-aware acknowledgments. We find that coding reduces the resource demands of a slice to meet the requirements for an application, thereby serving more applications efficiently. Coded slices thus free up resources for other slices, be they coded or not. Based on these results, we propose a hybrid approach, wherein coding is introduced selectively in certain network slices. This approach not only facilitates a smoother transition from un-coded systems to coded systems but also reduces costs across all slices. Theoretical findings in this paper are validated and expanded upon through real-time simulations of the network.

Large Language Models (LLM) have become a popular approach for implementing Retrieval Augmented Generation (RAG) systems, and a significant amount of effort has been spent on building good models and metrics. In spite of increased recognition of the need for rigorous evaluation of RAG systems, few tools exist that go beyond the creation of model output and automatic calculation. We present InspectorRAGet, an introspection platform for RAG evaluation. InspectorRAGet allows the user to analyze aggregate and instance-level performance of RAG systems, using both human and algorithmic metrics as well as annotator quality. InspectorRAGet is suitable for multiple use cases and is available publicly to the community. The demo video is available at https://youtu.be/MJhe8QIXcEc

Recent work has developed optimization procedures to find token sequences, called adversarial triggers, which can elicit unsafe responses from aligned language models. These triggers are believed to be universally transferable, i.e., a trigger optimized on one model can jailbreak other models. In this paper, we concretely show that such adversarial triggers are not universal. We extensively investigate trigger transfer amongst 13 open models and observe inconsistent transfer. Our experiments further reveal a significant difference in robustness to adversarial triggers between models Aligned by Preference Optimization (APO) and models Aligned by Fine-Tuning (AFT). We find that APO models are extremely hard to jailbreak even when the trigger is optimized directly on the model. On the other hand, while AFT models may appear safe on the surface, exhibiting refusals to a range of unsafe instructions, we show that they are highly susceptible to adversarial triggers. Lastly, we observe that most triggers optimized on AFT models also generalize to new unsafe instructions from five diverse domains, further emphasizing their vulnerability. Overall, our work highlights the need for more comprehensive safety evaluations for aligned language models.

Metastable states arise in a range of quantum systems and can be observed in various dynamical scenarios, including decay, bubble nucleation, and long-lived oscillations. The phenomenology of metastable states has been examined in quantum many-body systems, notably in 1D ferromagnetic Ising spin systems and superfluids. In this paper, we study long-lived oscillations of metastable and ground states in 1D antiferromagnetic neutral atom chains with long-range Rydberg interactions. We use a staggered local detuning field to achieve confinement. Using theoretical and numerical models, we identify novel spectral signatures of quasiparticle oscillations distinct to antiferromagnetic neutral atom systems and interpret them using a classical energy model of short-range meson repulsion. Finally, we evaluate the experimental accessibility of our proposed setup on current neutral-atom platforms and discuss experimental feasibility and constraints., Comment: 15 pages, 5 figures

Metastable states of quantum many-body systems with confinement offer a means to simulate false vacuum phenomenology, including non-equilibrium dynamical processes like decay by nucleation, in truncated limits. Recent work has examined the decay process in 1D ferromagnetic Ising spins and superfluids. In this paper, we study nucleation dynamics in 1D antiferromagnetic neutral atom chains with Rydberg interactions, using both numerical simulations and analytic modeling. We apply a staggered local detuning field to generate the metastable and ground states. Our efforts focus on two dynamical regimes: decay and annealing. In the first, we corroborate the phenomenological decay rate scaling and determine the associated parameter range for the decay process; in the second, we uncover and elucidate a procedure to anneal the metastable state from the initial to the final system, with intermediate nucleation events. We further propose experimental protocols to prepare the required states and perform quenches on near-term neutral atom quantum simulators, examining the experimental feasibility of our proposed setup and parameter regime., Comment: 15 pages, 8 figures

This paper explores the evolution of Radio Access Network (RAN) architectures and their integration into Non-Terrestrial Networks (NTN) to address escalating mobile traffic demands. Focusing on Low Earth Orbit (LEO) satellites as key components of NTN, we examine the feasibility of RAN function splits (FSs) in terms of fronthaul (FH) latency, elevation angle, and bandwidth (BW) across LEO satellites and ground stations (GS), alongside evaluating performance of Conditional Handover (CHO) procedures under diverse scenarios. By assessing performance metrics such as handover duration, disconnection time, and control traffic volume, we provide insights on several aspects such as stringent constraints for Low Layer Splits (LLSs), leading to longer delays during mobility procedures and increased control traffic across the feeder link in comparison with the case when gNodeB is onboard satellite. Despite challenges, LLSs demonstrate minimal onboard satellite computational requirements, promising reduced power consumption and payload weight. These findings underscore the architectural possibilities and challenges within the telecommunications industry, paving the way for future advancements in NTN RAN design and operation., Comment: 6 pages, 6 figures, EuCNC conference 2024

Large decoder-only language models (LLMs) are the state-of-the-art models on most of today's NLP tasks and benchmarks. Yet, the community is only slowly adopting these models for text embedding tasks, which require rich contextualized representations. In this work, we introduce LLM2Vec, a simple unsupervised approach that can transform any decoder-only LLM into a strong text encoder. LLM2Vec consists of three simple steps: 1) enabling bidirectional attention, 2) masked next token prediction, and 3) unsupervised contrastive learning. We demonstrate the effectiveness of LLM2Vec by applying it to 4 popular LLMs ranging from 1.3B to 8B parameters and evaluate the transformed models on English word- and sequence-level tasks. We outperform encoder-only models by a large margin on word-level tasks and reach a new unsupervised state-of-the-art performance on the Massive Text Embeddings Benchmark (MTEB). Moreover, when combining LLM2Vec with supervised contrastive learning, we achieve state-of-the-art performance on MTEB among models that train only on publicly available data (as of May 24, 2024). Our strong empirical results and extensive analysis demonstrate that LLMs can be effectively transformed into universal text encoders in a parameter-efficient manner without the need for expensive adaptation or synthetic GPT-4 generated data., Comment: Accepted to COLM 2024

Aquila, an analog quantum simulation platform developed by QuEra Computing, supports control of the position and coherent evolution of up to 256 neutral atoms. This study details novel experimental protocols designed for analog quantum simulators that generate Bell state entanglement far away from the blockade regime, construct a $Z_2$ state with a defect induced by an ancilla, and optimize the driving fields schedule to prepare excited states with enhanced fidelity. We additionally evaluate the effectiveness of readout error mitigation techniques in improving the fidelity of measurement results. All experiments were executed on Aquila from QuEra and facilitated by the AWS Braket interface. Our experimental results closely align with theoretical predictions and numerical simulations. The insights gained from this study showcase Aquila's capabilities in handling complex quantum simulations and computations, and also pave the way for new avenues of research in quantum information processing and physics that employ programmable analog hardware platforms., Comment: 7 pages, 6 figures, IEEE QCE24 - QTEM Best Paper Award - 1st Place

Sentences containing multiple semantic operators with overlapping scope often create ambiguities in interpretation, known as scope ambiguities. These ambiguities offer rich insights into the interaction between semantic structure and world knowledge in language processing. Despite this, there has been little research into how modern large language models treat them. In this paper, we investigate how different versions of certain autoregressive language models -- GPT-2, GPT-3/3.5, Llama 2 and GPT-4 -- treat scope ambiguous sentences, and compare this with human judgments. We introduce novel datasets that contain a joint total of almost 1,000 unique scope-ambiguous sentences, containing interactions between a range of semantic operators, and annotated for human judgments. Using these datasets, we find evidence that several models (i) are sensitive to the meaning ambiguity in these sentences, in a way that patterns well with human judgments, and (ii) can successfully identify human-preferred readings at a high level of accuracy (over 90% in some cases)., Comment: To be published in Transactions of the Association for Computational Linguistics

This paper addresses the issue of active speaker detection (ASD) in noisy environments and formulates a robust active speaker detection (rASD) problem. Existing ASD approaches leverage both audio and visual modalities, but non-speech sounds in the surrounding environment can negatively impact performance. To overcome this, we propose a novel framework that utilizes audio-visual speech separation as guidance to learn noise-free audio features. These features are then utilized in an ASD model, and both tasks are jointly optimized in an end-to-end framework. Our proposed framework mitigates residual noise and audio quality reduction issues that can occur in a naive cascaded two-stage framework that directly uses separated speech for ASD, and enables the two tasks to be optimized simultaneously. To further enhance the robustness of the audio features and handle inherent speech noises, we propose a dynamic weighted loss approach to train the speech separator. We also collected a real-world noise audio dataset to facilitate investigations. Experiments demonstrate that non-speech audio noises significantly impact ASD models, and our proposed approach improves ASD performance in noisy environments. The framework is general and can be applied to different ASD approaches to improve their robustness. Our code, models, and data will be released., Comment: 15 pages, 5 figures

Quantum error mitigation techniques mimic noiseless quantum circuits by running several related noisy circuits and combining their outputs in particular ways. How well such techniques work is thought to depend strongly on how noisy the underlying gates are. Weakly-entangling gates, like $R_{ZZ}(\theta)$ for small angles $\theta$, can be much less noisy than entangling Clifford gates, like CNOT and CZ, and they arise naturally in circuits used to simulate quantum dynamics. However, such weakly-entangling gates are non-Clifford, and are therefore incompatible with two of the most prominent error mitigation techniques to date: probabilistic error cancellation (PEC) and the related form of zero-noise extrapolation (ZNE). This paper generalizes these techniques to non-Clifford gates, and comprises two complementary parts. The first part shows how to effectively transform any given quantum channel into (almost) any desired channel, at the cost of a sampling overhead, by adding random Pauli gates and processing the measurement outcomes. This enables us to cancel or properly amplify noise in non-Clifford gates, provided we can first characterize such gates in detail. The second part therefore introduces techniques to do so for noisy $R_{ZZ}(\theta)$ gates. These techniques are robust to state preparation and measurement (SPAM) errors, and exhibit concentration and sensitivity--crucial features in many experiments. They are related to randomized benchmarking, and may also be of interest beyond the context of error mitigation. We find that while non-Clifford gates can be less noisy than related Cliffords, their noise is fundamentally more complex, which can lead to surprising and sometimes unwanted effects in error mitigation. Whether this trade-off can be broadly advantageous remains to be seen.