Search

Large Vision Language Models (LVLMs) have demonstrated impressive capabilities in video understanding, yet their adoption for Activities of Daily Living (ADL) remains limited by their inability to capture fine-grained interactions and spatial relationships. This limitation is particularly evident in ADL tasks, where understanding detailed human-object interaction and human-centric motion is crucial for applications such as elderly monitoring and cognitive assessment. To address this, we aim to leverage the complementary nature of egocentric views to enhance LVLM's understanding of exocentric ADL videos. Consequently, we propose an online ego2exo distillation approach to learn ego-augmented exo representations in LVLMs. While effective, this approach requires paired ego-exo training data, which is impractical to collect for real-world ADL scenarios. Consequently, we develop EgoMimic, a skeleton-guided method that can generate mimicked ego views from exocentric videos. We find that the exo representations of our ego-augmented LVLMs successfully learn to extract ego-perspective cues, demonstrated through comprehensive evaluation on six ADL benchmarks and our proposed EgoPerceptionMCQ benchmark designed specifically to assess egocentric understanding from exocentric videos. Code, models, and data will be open-sourced at https://github.com/dominickrei/EgoExo4ADL.

We study the linear stability of inviscid steady parallel flow of an ideal gas in a channel of finite width. Compressible isothermal two-dimensional monochromatic perturbations are considered. The eigenvalue problem governing density and velocity perturbations is a compressible version of Rayleigh's equation and involves two parameters: a flow Mach number $M$ and the perturbation wavenumber $k$. For an odd background velocity profile, there is a $\mathbb{Z}_2 \times \mathbb{Z}_2$ symmetry and growth rates $\gamma$ come in symmetrically placed 4-tuples in the complex eigenplane. Specializing to uniform background vorticity Couette flow, we find an infinite tower of noninflectional eigenmodes and derive stability theorems and bounds on growth rates. We show that eigenmodes are neutrally stable for small $k$ and small $M$ but that they otherwise display an infinite sequence of stability transitions with increasing $k$ or $M$. Using a search algorithm based on the Fredholm alternative, we find that the transitions are associated to level crossings between neighboring eigenmodes. Repeated level crossings result in windows of instability. For a given eigenmode, they are arranged in a zebra-like striped pattern on the $k$-$M$ plane. A canonical square-root power law form for $\gamma(k,M)$ in the vicinity of a stability transition is identified. In addition to the discrete spectrum, we find a continuous spectrum of eigenmodes that are always neutrally stable but fail to be smooth across critical layers., Comment: 33 pages and 13 figure files

Radio remains a pervasive medium for mass information dissemination, with AM/FM stations reaching more Americans than either smartphone-based social networking or live television. Increasingly, radio broadcasts are also streamed online and accessed over the Internet. We present WavePulse, a framework that records, documents, and analyzes radio content in real-time. While our framework is generally applicable, we showcase the efficacy of WavePulse in a collaborative project with a team of political scientists focusing on the 2024 Presidential Elections. We use WavePulse to monitor livestreams of 396 news radio stations over a period of three months, processing close to 500,000 hours of audio streams. These streams were converted into time-stamped, diarized transcripts and analyzed to track answer key political science questions at both the national and state levels. Our analysis revealed how local issues interacted with national trends, providing insights into information flow. Our results demonstrate WavePulse's efficacy in capturing and analyzing content from radio livestreams sourced from the Web. Code and dataset can be accessed at \url{https://wave-pulse.io}., Comment: 22 Pages: 10 main + 12 appendix, 24 figures. Access code and dataset at https://wave-pulse.io

Celiac disease is an autoimmune disorder triggered by the consumption of gluten. It causes damage to the villi, the finger-like projections in the small intestine that are responsible for nutrient absorption. Additionally, the crypts, which form the base of the villi, are also affected, impairing the regenerative process. The deterioration in villi length, computed as the villi-to-crypt length ratio, indicates the severity of celiac disease. However, manual measurement of villi-crypt length can be both time-consuming and susceptible to inter-observer variability, leading to inconsistencies in diagnosis. While some methods can perform measurement as a post-hoc process, they are prone to errors in the initial stages. This gap underscores the need for pathologically driven solutions that enhance measurement accuracy and reduce human error in celiac disease assessments. Our proposed method, MeasureNet, is a pathologically driven polyline detection framework incorporating polyline localization and object-driven losses specifically designed for measurement tasks. Furthermore, we leverage segmentation model to provide auxiliary guidance about crypt location when crypt are partially visible. To ensure that model is not overdependent on segmentation mask we enhance model robustness through a mask feature mixup technique. Additionally, we introduce a novel dataset for grading celiac disease, consisting of 750 annotated duodenum biopsy images. MeasureNet achieves an 82.66% classification accuracy for binary classification and 81% accuracy for multi-class grading of celiac disease. Code: https://github.com/dair-iitd/MeasureNet

In many real-world applications, continuous machine learning (ML) systems are crucial but prone to data drift, a phenomenon where discrepancies between historical training data and future test data lead to significant performance degradation and operational inefficiencies. Traditional drift adaptation methods typically update models using ensemble techniques, often discarding drifted historical data, and focus primarily on either covariate drift or concept drift. These methods face issues such as high resource demands, inability to manage all types of drifts effectively, and neglecting the valuable context that historical data can provide. We contend that explicitly incorporating drifted data into the model training process significantly enhances model accuracy and robustness. This paper introduces an advanced framework that integrates the strengths of data-centric approaches with adaptive management of both covariate and concept drift in a scalable and efficient manner. Our framework employs sophisticated data segmentation techniques to identify optimal data batches that accurately reflect test data patterns. These data batches are then utilized for training on test data, ensuring that the models remain relevant and accurate over time. By leveraging the advantages of both data segmentation and scalable drift management, our solution ensures robust model accuracy and operational efficiency in large-scale ML deployments. It also minimizes resource consumption and computational overhead by selecting and utilizing relevant data subsets, leading to significant cost savings. Experimental results on classification task on real-world and synthetic datasets show our approach improves model accuracy while reducing operational costs and latency. This practical solution overcomes inefficiencies in current methods, providing a robust, adaptable, and scalable approach., Comment: Accepted in CODS-COMAD 2024

The NTK is a widely used tool in the theoretical analysis of deep learning, allowing us to look at supervised deep neural networks through the lenses of kernel regression. Recently, several works have investigated kernel models for self-supervised learning, hypothesizing that these also shed light on the behaviour of wide neural networks by virtue of the NTK. However, it remains an open question to what extent this connection is mathematically sound -- it is a commonly encountered misbelief that the kernel behaviour of wide neural networks emerges irrespective of the loss function it is trained on. In this paper, we bridge the gap between the NTK and self-supervised learning, focusing on two-layer neural networks trained under the Barlow Twins loss. We prove that the NTK of Barlow Twins indeed becomes constant as the width of the network approaches infinity. Our analysis technique is different from previous works on the NTK and may be of independent interest. Overall, our work provides a first rigorous justification for the use of classic kernel theory to understand self-supervised learning of wide neural networks. Building on this result, we derive generalization error bounds for kernelized Barlow Twins and connect them to neural networks of finite width.

This article provides an expository account of training dynamics in the Deep Linear Network (DLN) from the perspective of the geometric theory of dynamical systems. Rigorous results by several authors are unified into a thermodynamic framework for deep learning. The analysis begins with a characterization of the invariant manifolds and Riemannian geometry in the DLN. This is followed by exact formulas for a Boltzmann entropy, as well as stochastic gradient descent of free energy using a Riemannian Langevin Equation. Several links between the DLN and other areas of mathematics are discussed, along with some open questions.

The absorption and emission spectrum arising from the vibrational motion of a molecule is mostly in the infrared region. These fingerprint absorptions of polar bonds enable us to acquire bond-specific chemical information from specimens. However, the mode mismatch between the atomic-scale dimensions of the chemical bonds and the resonance wavelength limits the direct detection of tiny amounts of samples such as self-assembled monolayers or biological membranes. To overcome this limitation, surface-enhanced infrared absorption spectroscopy (SEIRA) has been proposed to enhance infrared absorption directly via local field enhancement. Here, we report on the perfect absorption of molecular vibration enabled by critical coupling in the metamaterials. Our molecular metamaterial design consists of a thin polymer layer sandwiched between a structured metal layer on top and a continuous metal layer at the bottom that supports the gap plasmon mode. The measured and simulated infrared spectra of the molecular metamaterial show broad and narrow absorption bands corresponding to the metamaterial and molecular vibration modes. We show that by tuning the structure's molecular film thickness and periodicity, vibrational absorption can be enhanced to near unity. We also show that for a particular periodicity of the array, metamaterial resonance can be completely suppressed, and only molecular vibrational absorption is excited, giving rise to an extremely narrow absorption band.

Neural networks trained with stochastic gradient descent exhibit an inductive bias towards simpler decision boundaries, typically converging to a narrow family of functions, and often fail to capture more complex features. This phenomenon raises concerns about the capacity of deep models to adequately learn and represent real-world datasets. Traditional approaches such as explicit regularization, data augmentation, architectural modifications, etc., have largely proven ineffective in encouraging the models to learn diverse features. In this work, we investigate the impact of pre-training models with noisy labels on the dynamics of SGD across various architectures and datasets. We show that pretraining promotes learning complex functions and diverse features in the presence of noise. Our experiments demonstrate that pre-training with noisy labels encourages gradient descent to find alternate minima that do not solely depend upon simple features, rather learns more complex and broader set of features, without hurting performance., Comment: Accepted at Workshop on Scientific Methods for Understanding Deep Learning, NeurIPS 2024

Digital payment systems have revolutionized financial transactions, offering unparalleled convenience and accessibility to users worldwide. However, the increasing popularity of these platforms has also attracted malicious actors seeking to exploit their vulnerabilities for financial gain. To address this challenge, robust and adaptable scam detection mechanisms are crucial for maintaining the trust and safety of digital payment ecosystems. This paper presents a comprehensive approach to scam detection, focusing on the Unified Payments Interface (UPI) in India, Google Pay (GPay) as a specific use case. The approach leverages Large Language Models (LLMs) to enhance scam classification accuracy and designs a digital assistant to aid human reviewers in identifying and mitigating fraudulent activities. The results demonstrate the potential of LLMs in augmenting existing machine learning models and improving the efficiency, accuracy, quality, and consistency of scam reviews, ultimately contributing to a safer and more secure digital payment landscape. Our evaluation of the Gemini Ultra model on curated transaction data showed a 93.33% accuracy in scam classification. Furthermore, the model demonstrated 89% accuracy in generating reasoning for these classifications. A promising fact, the model identified 32% new accurate reasons for suspected scams that human reviewers had not included in the review notes., Comment: 10 pages, 7 figures

We derive robust long-time a-priori estimates for the Navier-Stokes equation in a two-dimensional infinite strip which are uniform in the Reynolds number. These estimates provide several new scale invariant upper bounds for the size of the mixing layer in the Kelvin-Helmholtz instability.

Recently, language models like Llama 3.1 Instruct have become increasingly capable of agentic behavior, enabling them to perform tasks requiring short-term planning and tool use. In this study, we apply refusal-vector ablation to Llama 3.1 70B and implement a simple agent scaffolding to create an unrestricted agent. Our findings imply that these refusal-vector ablated models can successfully complete harmful tasks, such as bribing officials or crafting phishing attacks, revealing significant vulnerabilities in current safety mechanisms. To further explore this, we introduce a small Safe Agent Benchmark, designed to test both harmful and benign tasks in agentic scenarios. Our results imply that safety fine-tuning in chat models does not generalize well to agentic behavior, as we find that Llama 3.1 Instruct models are willing to perform most harmful tasks without modifications. At the same time, these models will refuse to give advice on how to perform the same tasks when asked for a chat completion. This highlights the growing risk of misuse as models become more capable, underscoring the need for improved safety frameworks for language model agents.

Data curation is the problem of how to collect and organize samples into a dataset that supports efficient learning. Despite the centrality of the task, little work has been devoted towards a large-scale, systematic comparison of various curation methods. In this work, we take steps towards a formal evaluation of data curation strategies and introduce SELECT, the first large-scale benchmark of curation strategies for image classification. In order to generate baseline methods for the SELECT benchmark, we create a new dataset, ImageNet++, which constitutes the largest superset of ImageNet-1K to date. Our dataset extends ImageNet with 5 new training-data shifts, each approximately the size of ImageNet-1K itself, and each assembled using a distinct curation strategy. We evaluate our data curation baselines in two ways: (i) using each training-data shift to train identical image classification models from scratch (ii) using the data itself to fit a pretrained self-supervised representation. Our findings show interesting trends, particularly pertaining to recent methods for data curation such as synthetic data generation and lookup based on CLIP embeddings. We show that although these strategies are highly competitive for certain tasks, the curation strategy used to assemble the original ImageNet-1K dataset remains the gold standard. We anticipate that our benchmark can illuminate the path for new methods to further reduce the gap. We release our checkpoints, code, documentation, and a link to our dataset at https://github.com/jimmyxu123/SELECT., Comment: NeurIPS 2024, Datasets and Benchmarks Track

We consider the framework of distributed aggregation of Distributed Energy Resources (DERs) in power networks to provide ancillary services to the power grid. Existing aggregation schemes work under the assumption of trust and honest behavior of the DERs and can suffer when that is not the case. In this article, we develop a distributed detection scheme that allows the DERs to detect and isolate the maliciously behaving DERs. We propose a model for the maliciously behaving DERs and show that the proposed distributed scheme leads to the detection of the malicious DERs. Further, augmented with the distributed power apportioning algorithm the proposed scheme provides a framework for resilient distributed power apportioning for ancillary service dispatch in power networks. A controller-hardware-in-the-loop (CHIL) experimental setup is developed to evaluate the performance of the proposed resilient distributed power apportioning scheme on an 8-commercial building distribution network (Central Core) connected to a 55 bus distribution network (External Power Network) based on the University of Minnesota Campus. A diversity of DERs and loads are included in the network to generalize the applicability of the framework. The experimental results corroborate the efficacy of the proposed resilient distributed power apportioning for ancillary service dispatch in power networks., Comment: 7 pages, 3 figures

To mitigate risks from AI systems, we need to assess their capabilities accurately. This is especially difficult in cases where capabilities are only rarely displayed. Phuong et al. propose two methods that aim to obtain better estimates of the probability of an AI agent successfully completing a given task. The milestone method decomposes tasks into subtasks, aiming to improve overall success rate estimation, while the expert best-of-N method leverages human guidance as a proxy for the model's independent performance. Our analysis of these methods as Monte Carlo estimators reveals that while both effectively reduce variance compared to naive Monte Carlo sampling, they also introduce bias. Experimental results demonstrate that the milestone method underestimates true solve rates for many real-world tasks due to its constraining assumptions. The expert best-of-N method exhibits even more severe underestimation across all tasks, attributed to an inherently flawed re-weighting factor. To enhance the accuracy of capability estimates of AI agents on difficult tasks, we suggest future work should leverage the rich literature on Monte Carlo Estimators., Comment: Accepted (Poster) to SoLaR 2024

Coherent control of single photon sources is a key requirement for the advancement of photonic quantum technologies. Among them, nanowire-based quantum dot sources are popular due to their potential for on-chip hybrid integration. Here we demonstrate on-demand single-photon generation ($g^{(2)}(0)(X^{*}) =0.078$ and $g^{(2)}(0)(X)= 0.03$) from resonantly excited InAsP/InP nanowire quantum dots and observe Rabi oscillations in the dot emission, indicating successful coherent manipulation of the excitonic states in the nanowire. We also measure a low emission time jitter for resonant excitation as compared to above-band excitation. This work addresses the long-standing challenge of resonantly exciting nanowire-quantum dots. It paves the way for hybrid quantum photonic integration, enabling spin-photon entanglement and matter memories on-chip.

We set up the Wheeler-DeWitt (WDW) equation for late gravitational collapse. The fact that the gravitational collapse and the expanding/ collapsing universe can be described within the realm of the Robertson-Walker metric renders the corresponding WDW equation for collapsing matter a timeless Schr\"odinger equation. We explore the consequences of such an equation and find the density to be quantized in terms of the Planck density. Apart from that, the wave function as a solution of the WDW equation shows that the initial singularity is avoided. We concentrate on different factor orderings in the kinetic term of the equation and show how after splitting off an exponential ansatz, new polynomials entering the solution can be constructed. This enables us to conclude that the factor ordering changes the details of the solution and interpretation, but overall on a qualitative level the results remain the same. We also probe into the effects of a positive cosmological constant. It offers the possibility of a tunneling scenario at the cosmological horizon., Comment: 22 pages, 5 figures

The irregular moons orbit the giant planets on distant, inclined, and eccentric trajectories, in sharp contrast with the coplanar and quasicircular orbits of the regular moons. The origin of these irregular moons is still an open question, but these moons have a lot in common with the objects beyond Neptune (trans-Neptunian objects -- TNOs), suggestive of a common origin. Here, we show that the close flyby of a star may be the connecting element. A stellar flyby can simultaneously reproduce the complex TNO dynamics quantitatively while explaining the origin of the irregular moons and the colour distributions of both populations. This flyby would have catapulted 7.2% of the original TNO population into the region of the planets, many on retrograde orbits. Most injected TNOs would have been subsequently ejected from the solar system (85%). However, a considerable fraction would have had the potential to be captured by the planets. The exclusively distant origin of the injected TNOs may also explain the lack of very red irregular moons., Comment: 5 pages, 4 figures

Unlike the Solar System planets, thousands of smaller bodies beyond Neptune orbit the Sun on eccentric ($e >$ 0.1) and ($i>$ 3$^\circ$) orbits. While migration of the giant planets during the early stages of Solar System evolution can induce substantial scattering of trans-Neptunian objects (TNO), this process cannot account for the small number of distant TNOs ($r_p >$ 60 au) outside the planets' reach. The alternative scenario of the close flyby of another star can instead produce all these TNO features simultaneously, but the possible parameter space for such an encounter is vast. Here, we compare observed TNO properties with thousands of flyby simulations to determine the specific properties of a flyby that reproduces all the different dynamical TNO populations, their location and their relative abundance and find that a 0.8$^{+0.1}_{-0.1}$ $M_{\odot}$ star passing at a distance of $ r_p =$ 110 $\mathbf{\pm}$ 10 au, inclined by $i$ = 70$^\circ$ $^{+5}_{-10}$ gives a near-perfect match. This flyby also replicates the retrograde TNO population, which has proved difficult to explain. Such a flyby is reasonably frequent; at least 140 million solar-type stars in the Milky Way are likely to have experienced a similar one. In light of these results, we predict that the upcoming Vera Rubin telescope will reveal that distant and retrograde TNOs are relatively common., Comment: 15 pages, 3 figures Open access version of original Nature Astronomy available under 10.1038/s41550-024-02349-x

Modern artificial intelligence (AI) systems are powered by foundation models. This paper presents a new set of foundation models, called Llama 3. It is a herd of language models that natively support multilinguality, coding, reasoning, and tool usage. Our largest model is a dense Transformer with 405B parameters and a context window of up to 128K tokens. This paper presents an extensive empirical evaluation of Llama 3. We find that Llama 3 delivers comparable quality to leading language models such as GPT-4 on a plethora of tasks. We publicly release Llama 3, including pre-trained and post-trained versions of the 405B parameter language model and our Llama Guard 3 model for input and output safety. The paper also presents the results of experiments in which we integrate image, video, and speech capabilities into Llama 3 via a compositional approach. We observe this approach performs competitively with the state-of-the-art on image, video, and speech recognition tasks. The resulting models are not yet being broadly released as they are still under development.

Prompt engineering is an iterative procedure often requiring extensive manual effort to formulate suitable instructions for effectively directing large language models (LLMs) in specific tasks. Incorporating few-shot examples is a vital and effective approach to providing LLMs with precise instructions, leading to improved LLM performance. Nonetheless, identifying the most informative demonstrations for LLMs is labor-intensive, frequently entailing sifting through an extensive search space. In this demonstration, we showcase a human-in-the-loop tool called APE (Active Prompt Engineering) designed for refining prompts through active learning. Drawing inspiration from active learning, APE iteratively selects the most ambiguous examples for human feedback, which will be transformed into few-shot examples within the prompt. The demo recording can be found with the submission or be viewed at https://youtu.be/OwQ6MQx53-Y., Comment: 3 pages, Proceedings of the Fifth Workshop on Data Science with Human-in-the-Loop (DaSH 2024)

The quantum three-rotor problem concerns the dynamics of 3 equally massive particles moving on a circle subject to pairwise attractive cosine potentials and can model coupled Josephson junctions. Classically, it displays order-chaos-order behavior with increasing energy. The quantum system admits a dimensionless coupling with semiclassical behavior at strong coupling. We study stationary states with periodic `relative' wave functions. Perturbative and harmonic approximations capture the spectrum at weak coupling and that of low-lying states at strong coupling. More generally, the cumulative distribution of energy levels obtained by numerical diagonalization is well-described by a Weyl-like semiclassical estimate. However, the system has an $S_3 \times Z_2$ symmetry that is obscured when working with relative angles. By exploiting a basis for invariant states, we obtain the spectrum restricted to the identity representation. To uncover universal quantum hallmarks of chaos, we partition the spectrum into energy windows where the classical motion is regular, mixed or chaotic and unfold each separately. At strong coupling, we find striking signatures of transitions between regularity and chaos: spacing distributions morph from Poisson to Wigner-Dyson while the number variance shifts from linear to logarithmic behavior at small lengths. Some nonuniversal features are also examined. For instance, for strong coupling, the number variance saturates and oscillates at large lengths while the spectral form factor displays a nonuniversal peak at short times. Moreover, deviations from Poisson spacings at asymptotically low and high energies are well-explained by quantum harmonic and free-rotor spectra projected to the identity representation at strong and weak coupling. Interestingly, the degeneracy of free-rotor levels admits an elegant formula that we deduce using properties of Eisenstein primes., Comment: 17 pages, 24 figure files, 2 tikz figures. To appear in Phys. Rev. E

Precise phase control of microwaves and millimeter waves is critical for today's wireless communication and signal processing electronics. For decades, achieving even modest phase-shifting resolution has demanded a complex mix of transistor switches and passive electromagnetic structures. These true-phase delay, true-time-delay or quasi-true-time-delay circuits operate near resonances that heavily attenuate and distort signals, limiting transmission bandwidth. Despite numerous attempts, passive phase shifters have remained lossy and incompatible with high-channel-capacity beamforming, irrespective of the semiconductor fabrication process. In this article, we introduce a mechanism whereby waveguide reflectors based on coupled resonances can be reprogrammed to evade loss. Central to their operation is that loss from internal resonances is confined to low frequencies, while at high frequencies, their reflectivity is maximized and broadband phase variations, induced by those resonances, still persist. Since performance in the low-loss post-resonance spectrum is largely agnostic to the number of switches, ultra-fine digital tuning is possible. This breaks the historical tradeoff between loss and precision to achieve resolution surpassing state-of-the-art integrated circuits by over three orders (bits) of magnitude. The device does not distort the transmitted signal and consumes no power. Moreover, the chip occupies a sub-wavelength footprint in a Complementary Metal Oxide Semiconductor platform. This makes it an optimal candidate for seamless integration in on-chip multi-gigabit data links, radio astronomy transceivers and control hardware for millimeter-wave qubits., Comment: 12 main article pages, 4 main figures, 4 extended figures, 15 Supplementary information pages

Several studies have investigated human interaction using modern tracking techniques for face-to-face encounters across various settings and age groups. However, little attention has been given to understanding how individual characteristics relate to social behavior. This is particularly important in younger age groups due to its potential effects on early childhood development. In this study, conducted during the Complexity 72h Workshop, we analyze human social interactions in a French preschool, where children's face-to-face interactions were monitored using proximity sensors over an academic year. We use metadata from parent surveys and preschool linguistic tests, covering demographic information and home habits, to examine the interplay between individual characteristics and contact patterns. Using a mixture of approaches, from random forest classifiers to network-based metrics at both dyadic and higher-order (group) levels, we identify sex, age, language scores, and number of siblings as the variables displaying the most significant associations with interaction patterns. We explore these variables' relationships to interactions within and outside classrooms and across mixed and single-grade classes. At the group level, we investigate how group affinity affects group persistence. We also find that higher-order network centrality (hypercoreness) is higher among children with siblings, indicating different group embedding despite similar total contact duration. This study aligns with existing literature on early social development and highlights the importance of integrating individual traits into the study of human interactions. Focusing on 2-5-year-olds offers insights into emerging social preferences during critical phases of cognitive development. Future research could use these findings to enhance mechanistic models of complex social systems by incorporating individual traits., Comment: This work is the output of the Complexity72h workshop, held at the Universidad Carlos III de Madrid in Legan\'es, Spain, 24-28 June 2024

Open clusters are ideal tools for tracing the abundances of different elements because their stars are expected to have the same age, distance, and metallicity. Therefore, they serve as very powerful tracers for investigating the cosmic origins of elements. This paper expands on a recent study by us, where the element Fluorine was studied in seven previously open clusters, adding six open clusters as well as eight field stars. The primary objective is to determine the abundance of fluorine (F) to gain insight into its production and evolution. The magnesium (Mg) abundances were derived to categorize the field stars into high and low alpha disk populations. Additionally, cerium (Ce) abundances are determined to better understand the interplay between F and s-process elements. The spectra were obtained from the high-resolution near-infra-red GIANO-B instrument at the Telescopio Nazionale Galileo (TNG). For the derivation of the stellar parameters and abundances, the Python version of Spectroscopy Made Easy (PySME) was used. OH, CN, and CO molecular lines and band heads along with Fe I lines were used to determine the stellar parameters in the H-band region. Two HF lines in the K-band ({\lambda}{\lambda} 2.28, 2.33 {\mu}m), three K-band Mg I lines ({\lambda}{\lambda} 2.10, 2.11, 2.15 {\mu}m), and two Ce II lines in the H-band ({\lambda}{\lambda} 1.66, and 1.71 {\mu}m) were used to derive the abundances of F, Mg, and Ce, respectively. F, Mg, and Ce abundances were derived for 14 stars from 6 OCs, as well as 8 field stars. The F and Ce abundances were investigated as a function of metallicity, age, and Galactocentric distances. Our results indicate that asymptotic giant branch stars and massive stars, including a subset of fast rotators (whose rotation speed likely increases as metallicity decreases), are necessary to explain the cosmic origin of F., Comment: 12 pages, 4 figures, accepted by Astronomy and Astrophysics(A&A)

The translation of Metric Interval Temporal Logic (MITL) to timed automata is a topic that has been extensively studied. A key challenge here is the conversion of future modalities into equivalent automata. Typical conversions equip the automata with a guess-and-check mechanism to ascertain the truth of future modalities. Guess-and-check can be naturally implemented via alternation. However, since timed automata tools do not handle alternation, existing methods perform an additional step of converting the alternating timed automata into timed automata. This de-alternation step proceeds by an intricate finite abstraction of the space of configurations of the alternating automaton. Recently, a model of generalized timed automata (GTA) has been proposed. The model comes with several powerful additional features, and yet, the best known zone-based reachability algorithms for timed automata have been extended to the GTA model, with the same complexity for all the zone operations. We provide a new concise translation from MITL to GTA. In particular, for the timed until modality, our translation offers an exponential improvement w.r.t. the state-of-the-art. Thanks to this conversion, MITL model checking reduces to checking liveness for GTAs. However, no liveness algorithm is known for GTAs. Due to the presence of future clocks, there is no finite time-abstract bisimulation (region equivalence) for GTAs, whereas liveness algorithms for timed automata crucially rely on the presence of the finite region equivalence. As our second contribution, we provide a new zone-based algorithm for checking Buchi non-emptiness in GTAs, which circumvents this fundamental challenge.

Legged robots are physically capable of navigating a diverse variety of environments and overcoming a wide range of obstructions. For example, in a search and rescue mission, a legged robot could climb over debris, crawl through gaps, and navigate out of dead ends. However, the robot's controller needs to respond intelligently to such varied obstacles, and this requires handling unexpected and unusual scenarios successfully. This presents an open challenge to current learning methods, which often struggle with generalization to the long tail of unexpected situations without heavy human supervision. To address this issue, we investigate how to leverage the broad knowledge about the structure of the world and commonsense reasoning capabilities of vision-language models (VLMs) to aid legged robots in handling difficult, ambiguous situations. We propose a system, VLM-Predictive Control (VLM-PC), combining two key components that we find to be crucial for eliciting on-the-fly, adaptive behavior selection with VLMs: (1) in-context adaptation over previous robot interactions and (2) planning multiple skills into the future and replanning. We evaluate VLM-PC on several challenging real-world obstacle courses, involving dead ends and climbing and crawling, on a Go1 quadruped robot. Our experiments show that by reasoning over the history of interactions and future plans, VLMs enable the robot to autonomously perceive, navigate, and act in a wide range of complex scenarios that would otherwise require environment-specific engineering or human guidance., Comment: 27 pages

The emerging field of polaritonic chemistry explores the behavior of molecules under strong coupling with cavity modes. Despite recent developments in ab initio polaritonic methods for simulating polaritonic chemistry under electronic strong coupling, their capabilities are limited, especially in cases where the molecule also features strong electronic correlation. To bridge this gap, we have developed a novel method for cavity QED calculations utilizing the Density Matrix Renormalization Group (DMRG) algorithm in conjunction with the Pauli-Fierz Hamiltonian. Our approach is applied to investigate the effect of the cavity on the S0 -S1 transition of n-oligoacenes, with n ranging from 2 to 5, encompassing 22 fully correlated {\pi} orbitals in the largest pentacene molecule. Our findings indicate that the influence of the cavity intensifies with larger acenes. Additionally, we demonstrate that, unlike the full determinantal representation, DMRG efficiently optimizes and eliminates excess photonic degrees of freedom, resulting in an asymptotically constant computational cost as the photonic basis increases.

Transformer architectures such as Vision Transformers (ViT) have proven effective for solving visual perception tasks. However, they suffer from two major limitations; first, the quadratic complexity of self-attention limits the number of tokens that can be processed, and second, Transformers often require large amounts of training data to attain state-of-the-art performance. In this paper, we propose a new multi-head self-attention (MHSA) variant named Fibottention, which can replace MHSA in Transformer architectures. Fibottention is data-efficient and computationally more suitable for processing large numbers of tokens than the standard MHSA. It employs structured sparse attention based on dilated Fibonacci sequences, which, uniquely, differ across attention heads, resulting in inception-like diverse features across heads. The spacing of the Fibonacci sequences follows the Wythoff array, which minimizes the redundancy of token interactions aggregated across different attention heads, while still capturing sufficient complementary information through token pair interactions. These sparse attention patterns are unique among the existing sparse attention and lead to an $O(N \log N)$ complexity, where $N$ is the number of tokens. Leveraging only 2-6% of the elements in the self-attention heads, Fibottention embedded into popular, state-of-the-art Transformer architectures can achieve significantly improved predictive performance for domains with limited data such as image classification, video understanding, and robot learning tasks, and render reduced computational complexity. We further validated the improved diversity of feature representations resulting from different self-attention heads, and our model design against other sparse attention mechanisms., Comment: The complete implementation, including source code and evaluation scripts, is publicly available at: https://github.com/Charlotte-CharMLab/Fibottention

Deterministic two-way transducers capture the class of regular functions. The efficiency of composing two-way transducers has a direct implication in algorithmic problems related to reactive synthesis, where transformation specifications are converted into equivalent transducers. These specifications are presented in a modular way, and composing the resultant machines simulates the full specification. An important result by Dartois et al. shows that composition of two-way transducers enjoy a polynomial composition when the underlying transducer is reversible, that is, if they are both deterministic and co-deterministic. This is a major improvement over general deterministic two-way transducers, for which composition causes a doubly exponential blow-up in the size of the inputs in general. Moreover, they show that reversible two-way transducers have the same expressiveness as deterministic two-way transducers. However, the question of expressiveness of reversible transducers over infinite words is still open. In this article, we introduce the class of reversible two-way transducers over infinite words and show that they enjoy the same expressive power as deterministic two-way transducers over infinite words. This is done through a non-trivial, effective construction inducing a single exponential blow-up in the set of states. Further, we also prove that composing two reversible two-way transducers over infinite words incurs only a polynomial complexity, thereby providing foundations for efficient procedure for composition of transducers over infinite words.

Current Large Language Vision Models (LLVMs) trained on web videos perform well in general video understanding but struggle with fine-grained details, complex human-object interactions (HOI), and view-invariant representation learning essential for Activities of Daily Living (ADL). This limitation stems from a lack of specialized ADL video instruction-tuning datasets and insufficient modality integration to capture discriminative action representations. To address this, we propose a semi-automated framework for curating ADL datasets, creating ADL-X, a multiview, multimodal RGBS instruction-tuning dataset. Additionally, we introduce LLAVIDAL, an LLVM integrating videos, 3D skeletons, and HOIs to model ADL's complex spatiotemporal relationships. For training LLAVIDAL a simple joint alignment of all modalities yields suboptimal results; thus, we propose a Multimodal Progressive (MMPro) training strategy, incorporating modalities in stages following a curriculum. We also establish ADL MCQ and video description benchmarks to assess LLVM performance in ADL tasks. Trained on ADL-X, LLAVIDAL achieves state-of-the-art performance across ADL benchmarks. Code and data will be made publicly available at: https://adl-x.github.io/.

Two-way quantum computing (2WQC) represents a novel approach to quantum computing that introduces a CPT version of state preparation. This paper analyses the influence of this approach on Grover's algorithm and compares the behaviour of typical Grover and its 2WQC version in the presence of noise in the system. Our findings indicate that, in an ideal scenario without noise, the 2WQC Grover algorithm exhibits a constant complexity of $\mathcal{O}(1)$. In the presence of noise, the 2WQC Grover algorithm demonstrates greater resilience to different noise models than the standard Grover's algorithm., Comment: Preliminary version of the article, we welcome your comments and suggestions

In this paper, we consider classic randomized low diameter decomposition procedures for planar graphs that obtain connected clusters which are cohesive in that close-by pairs of nodes are assigned to the same cluster with high probability. We require the additional aspect of individual fairness - pairs of nodes at comparable distances should be separated with comparable probability. We show that classic decomposition procedures do not satisfy this property. We present novel algorithms that achieve various trade-offs between this property and additional desiderata of connectivity of the clusters and optimality in the number of clusters. We show that our individual fairness bounds may be difficult to improve by tying the improvement to resolving a major open question in metric embeddings. We finally show the efficacy of our algorithms on real planar networks modeling congressional redistricting., Comment: To appear in ICML 2024

Research on jailbreaking has been valuable for testing and understanding the safety and security issues of large language models (LLMs). In this paper, we introduce Iterative Refinement Induced Self-Jailbreak (IRIS), a novel approach that leverages the reflective capabilities of LLMs for jailbreaking with only black-box access. Unlike previous methods, IRIS simplifies the jailbreaking process by using a single model as both the attacker and target. This method first iteratively refines adversarial prompts through self-explanation, which is crucial for ensuring that even well-aligned LLMs obey adversarial instructions. IRIS then rates and enhances the output given the refined prompt to increase its harmfulness. We find that IRIS achieves jailbreak success rates of 98% on GPT-4, 92% on GPT-4 Turbo, and 94% on Llama-3.1-70B in under 7 queries. It significantly outperforms prior approaches in automatic, black-box, and interpretable jailbreaking, while requiring substantially fewer queries, thereby establishing a new standard for interpretable jailbreaking methods., Comment: Accepted to EMNLP 2024 Main Conference

Over the past year, there has been a robust debate about the benefits and risks of open sourcing foundation models. However, this discussion has often taken place at a high level of generality or with a narrow focus on specific technical attributes. In part, this is because defining open source for foundation models has proven tricky, given its significant differences from traditional software development. In order to inform more practical and nuanced decisions about opening AI systems, including foundation models, this paper presents a framework for grappling with openness across the AI stack. It summarizes previous work on this topic, analyzes the various potential reasons to pursue openness, and outlines how openness varies in different parts of the AI stack, both at the model and at the system level. In doing so, its authors hope to provide a common descriptive framework to deepen a nuanced and rigorous understanding of openness in AI and enable further work around definitions of openness and safety in AI.