Search

Robot haircare systems could provide a controlled and personalized environment that is respectful of an individual's sensitivities and may offer a comfortable experience. We argue that because of hair and hairstyles' often unique importance in defining and expressing an individual's identity, we should approach the development of assistive robot haircare systems carefully while considering various practical and ethical concerns and risks. In this work, we specifically list and discuss the consideration of hair type, expression of the individual's preferred identity, cost accessibility of the system, culturally-aware robot strategies, and the associated societal risks. Finally, we discuss the planned studies that will allow us to better understand and address the concerns and considerations we outlined in this work through interactions with both haircare experts and end-users. Through these practical and ethical considerations, this work seeks to systematically organize and provide guidance for the development of inclusive and ethical robot haircare systems., Comment: 3rd Workshop on Inclusive HRI

Personalized search represents a problem where retrieval models condition on historical user interaction data in order to improve retrieval results. However, personalization is commonly perceived as opaque and not amenable to control by users. Further, personalization necessarily limits the space of items that users are exposed to. Therefore, prior work notes a tension between personalization and users' ability for discovering novel items. While discovery of novel items in personalization setups may be resolved through search result diversification, these approaches do little to allow user control over personalization. Therefore, in this paper, we introduce an approach for controllable personalized search. Our model, CtrlCE presents a novel cross-encoder model augmented with an editable memory constructed from users historical items. Our proposed memory augmentation allows cross-encoder models to condition on large amounts of historical user data and supports interaction from users permitting control over personalization. Further, controllable personalization for search must account for queries which don't require personalization, and in turn user control. For this, we introduce a calibrated mixing model which determines when personalization is necessary. This allows system designers using CtrlCE to only obtain user input for control when necessary. In multiple datasets of personalized search, we show CtrlCE to result in effective personalization as well as fulfill various key goals for controllable personalized search., Comment: Work in progress

This study presents an approach to developing a model for classifying abnormalities in video capsule endoscopy (VCE) frames. Given the challenges of data imbalance, we implemented a tiered augmentation strategy using the albumentations library to enhance minority class representation. Additionally, we addressed learning complexities by progressively structuring training tasks, allowing the model to differentiate between normal and abnormal cases and then gradually adding more specific classes based on data availability. Our pipeline, developed in PyTorch, employs a flexible architecture enabling seamless adjustments to classification complexity. We tested our approach using ResNet50 and a custom ViT-CNN hybrid model, with training conducted on the Kaggle platform. This work demonstrates a scalable approach to abnormality classification in VCE.

Image Quality of MRI brain scans is strongly influenced by within scanner head movements and the resulting image artifacts alter derived measures like brain volume and cortical thickness. Automated image quality assessment is key to controlling for confounding effects of poor image quality. In this study, we systematically test for the influence of image quality on univariate statistics and machine learning classification. We analyzed group effects of sex/gender on local brain volume and made predictions of sex/gender using logistic regression, while correcting for brain size. From three large publicly available datasets, two age and sex-balanced samples were derived to test the generalizability of the effect for pooled sample sizes of n=760 and n=1094. Results of the Bonferroni corrected t-tests over 3747 gray matter features showed a strong influence of low-quality data on the ability to find significant sex/gender differences for the smaller sample. Increasing sample size and more so image quality showed a stark increase in detecting significant effects in univariate group comparisons. For the classification of sex/gender using logistic regression, both increasing sample size and image quality had a marginal effect on the Area under the Receiver Operating Characteristic Curve for most datasets and subsamples. Our results suggest a more stringent quality control for univariate approaches than for multivariate classification with a leaning towards higher quality for classical group statistics and bigger sample sizes for machine learning applications in neuroimaging.

Our study introduces ResNet-L2 (RL2), a novel metric for evaluating generative models and image quality in histopathology, addressing limitations of traditional metrics, such as Frechet inception distance (FID), when the data is scarce. RL2 leverages ResNet features with a normalizing flow to calculate RMSE distance in the latent space, providing reliable assessments across diverse histopathology datasets. We evaluated the performance of RL2 on degradation types, such as blur, Gaussian noise, salt-and-pepper noise, and rectangular patches, as well as diffusion processes. RL2's monotonic response to increasing degradation makes it well-suited for models that assess image quality, proving a valuable advancement for evaluating image generation techniques in histopathology. It can also be used to discard low-quality patches while sampling from a whole slide image. It is also significantly lighter and faster compared to traditional metrics and requires fewer images to give stable metric value., Comment: 7 pages, 5 figures

Accurate survival prediction is essential for personalized cancer treatment. However, genomic data - often a more powerful predictor than pathology data - is costly and inaccessible. We present the cross-modal genomic feature translation and alignment network for enhanced survival prediction from histopathology images (PathoGen-X). It is a deep learning framework that leverages both genomic and imaging data during training, relying solely on imaging data at testing. PathoGen-X employs transformer-based networks to align and translate image features into the genomic feature space, enhancing weaker imaging signals with stronger genomic signals. Unlike other methods, PathoGen-X translates and aligns features without projecting them to a shared latent space and requires fewer paired samples. Evaluated on TCGA-BRCA, TCGA-LUAD, and TCGA-GBM datasets, PathoGen-X demonstrates strong survival prediction performance, emphasizing the potential of enriched imaging models for accessible cancer prognosis.

We report the discovery of the repeating fast radio burst source FRB 20240209A using the CHIME/FRB telescope. We have detected 22 bursts from this repeater between February and July 2024, six of which were also recorded at the Outrigger station KKO. The 66-km long CHIME-KKO baseline can provide single-pulse FRB localizations along one dimension with $2^{\prime\prime}$ accuracy. The high declination of $\sim$86 degrees for this repeater allowed its detection with a rotating range of baseline vectors, enabling the combined localization region size to be constrained to $1^{\prime\prime}\times2^{\prime\prime}$. We present deep Gemini observations that, combined with the FRB localization, enabled a robust association of FRB 20240209A to the outskirts of a luminous galaxy (P(O|x) = 0.99; $L \approx 5.3 \times 10^{10}\,L_{\odot}$). FRB 20240209A has a projected physical offset of $40 \pm 5$ kpc from the center of its host galaxy, making it the FRB with the largest host galaxy offset to date. When normalized by the host galaxy size, the offset of FRB 20240209A is comparable to that of FRB 20200120E, the only FRB source known to originate in a globular cluster. We consider several explanations for the large offset, including a progenitor that was kicked from the host galaxy or in situ formation in a low-luminosity satellite galaxy of the putative host, but find the most plausible scenario to be a globular cluster origin. This, coupled with the quiescent, elliptical nature of the host as demonstrated in our companion paper, provide strong evidence for a delayed formation channel for the progenitor of the FRB source., Comment: Submitted to AAS Journals

In recent years, there has been a trend in the field of Reinforcement Learning (RL) towards large action models trained offline on large-scale datasets via sequence modeling. Existing models are primarily based on the Transformer architecture, which result in powerful agents. However, due to slow inference times, Transformer-based approaches are impractical for real-time applications, such as robotics. Recently, modern recurrent architectures, such as xLSTM and Mamba, have been proposed that exhibit parallelization benefits during training similar to the Transformer architecture while offering fast inference. In this work, we study the aptitude of these modern recurrent architectures for large action models. Consequently, we propose a Large Recurrent Action Model (LRAM) with an xLSTM at its core that comes with linear-time inference complexity and natural sequence length extrapolation abilities. Experiments on 432 tasks from 6 domains show that LRAM compares favorably to Transformers in terms of performance and speed.

Many uncontrollable disease outbreaks of the past exposed several vulnerabilities in the healthcare systems worldwide. While advancements in technology assisted in the rapid creation of the vaccinations, there needs to be a pressing focus on the prevention and prediction of such massive outbreaks. Early detection and intervention of an outbreak can drastically reduce its impact on public health while also making the healthcare system more resilient. The complexity of disease transmission dynamics, influence of various directly and indirectly related factors and limitations of traditional approaches are the main bottlenecks in taking preventive actions. Specifically, this paper implements deep learning algorithms and LLM's to predict the severity of infectious disease outbreaks. Utilizing the historic data of several diseases that have spread in India and the climatic data spanning the past decade, the insights from our research aim to assist in creating a robust predictive system for any outbreaks in the future., Comment: 16 pages, 4 figures

GPT-4o is an autoregressive omni model that accepts as input any combination of text, audio, image, and video, and generates any combination of text, audio, and image outputs. It's trained end-to-end across text, vision, and audio, meaning all inputs and outputs are processed by the same neural network. GPT-4o can respond to audio inputs in as little as 232 milliseconds, with an average of 320 milliseconds, which is similar to human response time in conversation. It matches GPT-4 Turbo performance on text in English and code, with significant improvement on text in non-English languages, while also being much faster and 50\% cheaper in the API. GPT-4o is especially better at vision and audio understanding compared to existing models. In line with our commitment to building AI safely and consistent with our voluntary commitments to the White House, we are sharing the GPT-4o System Card, which includes our Preparedness Framework evaluations. In this System Card, we provide a detailed look at GPT-4o's capabilities, limitations, and safety evaluations across multiple categories, focusing on speech-to-speech while also evaluating text and image capabilities, and measures we've implemented to ensure the model is safe and aligned. We also include third-party assessments on dangerous capabilities, as well as discussion of potential societal impacts of GPT-4o's text and vision capabilities.

Machine learning (ML) models benefit from large datasets. Collecting data in biomedical domains is costly and challenging, hence, combining datasets has become a common practice. However, datasets obtained under different conditions could present undesired site-specific variability. Data harmonization methods aim to remove site-specific variance while retaining biologically relevant information. This study evaluates the effectiveness of popularly used ComBat-based methods for harmonizing data in scenarios where the class balance is not equal across sites. We find that these methods struggle with data leakage issues. To overcome this problem, we propose a novel approach PrettYharmonize, designed to harmonize data by pretending the target labels. We validate our approach using controlled datasets designed to benchmark the utility of harmonization. Finally, using real-world MRI and clinical data, we compare leakage-prone methods with PrettYharmonize and show that it achieves comparable performance while avoiding data leakage, particularly in site-target-dependence scenarios.

Formal verification provides mathematical guarantees that a software is correct. Design-level verification tools ensure software specifications are correct, but they do not expose defects in actual implementations. For this purpose, engineers use code-level tools. However, such tools struggle to scale to large software. The process of "Unit Proofing" mitigates this by decomposing the software and verifying each unit independently. We examined AWS's use of unit proofing and observed that current approaches are manual and prone to faults that mask severe defects. We propose a research agenda for a unit proofing framework, both methods and tools, to support software engineers in applying unit proofing effectively and efficiently. This will enable engineers to discover code-level defects early., Comment: 6 pages, 2 figures

Recent advances in diffusion models have significantly enhanced their ability to generate high-quality images and videos, but they have also increased the risk of producing unsafe content. Existing unlearning/editing-based methods for safe generation remove harmful concepts from models but face several challenges: (1) They cannot instantly remove harmful concepts without training. (2) Their safe generation capabilities depend on collected training data. (3) They alter model weights, risking degradation in quality for content unrelated to toxic concepts. To address these, we propose SAFREE, a novel, training-free approach for safe T2I and T2V, that does not alter the model's weights. Specifically, we detect a subspace corresponding to a set of toxic concepts in the text embedding space and steer prompt embeddings away from this subspace, thereby filtering out harmful content while preserving intended semantics. To balance the trade-off between filtering toxicity and preserving safe concepts, SAFREE incorporates a novel self-validating filtering mechanism that dynamically adjusts the denoising steps when applying the filtered embeddings. Additionally, we incorporate adaptive re-attention mechanisms within the diffusion latent space to selectively diminish the influence of features related to toxic concepts at the pixel level. In the end, SAFREE ensures coherent safety checking, preserving the fidelity, quality, and safety of the output. SAFREE achieves SOTA performance in suppressing unsafe content in T2I generation compared to training-free baselines and effectively filters targeted concepts while maintaining high-quality images. It also shows competitive results against training-based methods. We extend SAFREE to various T2I backbones and T2V tasks, showcasing its flexibility and generalization. SAFREE provides a robust and adaptable safeguard for ensuring safe visual generation., Comment: The first two authors contributed equally; Project page: https://safree-safe-t2i-t2v.github.io/

Machine Learning guided data augmentation may support the development of technologies in the physical sciences, such as nuclear fusion tokamaks. Here we endeavor to study the problem of detecting disruptions i.e. plasma instabilities that can cause significant damages, impairing the reliability and efficiency required for their real world viability. Machine learning (ML) prediction models have shown promise in detecting disruptions for specific tokamaks, but they often struggle in generalizing to the diverse characteristics and dynamics of different machines. This limits the effectiveness of ML models across different tokamak designs and operating conditions, which is a critical barrier to scaling fusion technology. Given the success of data augmentation in improving model robustness and generalizability in other fields, this study explores the use of a novel time series viewmaker network to generate diverse augmentations or "views" of training data. Our results show that incorporating views during training improves AUC and F2 scores on DisruptionBench tasks compared to standard or no augmentations. This approach represents a promising step towards developing more broadly applicable ML models for disruption avoidance, which is essential for advancing fusion technology and, ultimately, addressing climate change through reliable and sustainable energy production.

Wildfire suppression is a complex task that poses high risks to humans. Using robotic teams for wildfire suppression enhances the safety and efficiency of detecting, monitoring, and extinguishing fires. We propose a control architecture based on task hierarchical control for the autonomous steering of a system of flying robots in wildfire suppression. We incorporate a novel line-of-sight obstacle avoidance method that calculates the best viewpoints and ensures an occlusion-free view for the suppression robot during the mission. Path integral control generates optimal trajectories towards the goals. We conduct an ablation study to assess the effectiveness of our approach by comparing it to scenarios where these key components are excluded, in order to validate the approach in simulations using Matlab and Unity. The results demonstrate significant performance improvements, with 44.0 % increase in effectiveness with the new line-of-sight obstacle avoidance task and up to 39.6 % improvement when using path integral control., Comment: 6 pages, 5 figures, submitted to joint ACC and L-CSS

Fast radio bursts (FRBs) are unique probes of extragalactic ionized baryonic structure as each signal, through its burst properties, holds information about the ionized matter it encounters along its sightline. FRB 20200723B is a burst with a scattering timescale of $\tau_\mathrm{400\,MHz} >$1 second at 400 MHz and a dispersion measure of DM $\sim$ 244 pc cm$^{-3}$. Observed across the entire CHIME/FRB frequency band, it is the single-component burst with the largest scattering timescale yet observed by CHIME/FRB. The combination of its high scattering timescale and relatively low dispersion measure present an uncommon opportunity to use FRB 20200723B to explore the properties of the cosmic web it traversed. With an $\sim$arcminute-scale localization region, we find the most likely host galaxy is NGC 4602 (with PATH probability $P(O|x)=0.985$), which resides $\sim$30 Mpc away within a sheet filamentary structure on the outskirts of the Virgo Cluster. We place an upper limit on the average free electron density of this filamentary structure of $\langle n_e \rangle < 4.6^{+9.6}_{-2.0} \times 10^{-5}$ cm$^{-3}$, broadly consistent with expectations from cosmological simulations. We investigate whether the source of scattering lies within the same galaxy as the FRB, or at a farther distance from an intervening structure along the line of sight. Comparing with Milky Way pulsar observations, we suggest the scattering may originate from within the host galaxy of FRB 20200723B., Comment: 20 pages, 6 figures, submitted. Comments welcome!

In-context learning (ICL) is the ability of a model to learn a new task by observing a few exemplars in its context. While prevalent in NLP, this capability has recently also been observed in Reinforcement Learning (RL) settings. Prior in-context RL methods, however, require entire episodes in the agent's context. Given that complex environments typically lead to long episodes with sparse rewards, these methods are constrained to simple environments with short episodes. To address these challenges, we introduce Retrieval-Augmented Decision Transformer (RA-DT). RA-DT employs an external memory mechanism to store past experiences from which it retrieves only sub-trajectories relevant for the current situation. The retrieval component in RA-DT does not require training and can be entirely domain-agnostic. We evaluate the capabilities of RA-DT on grid-world environments, robotics simulations, and procedurally-generated video games. On grid-worlds, RA-DT outperforms baselines, while using only a fraction of their context length. Furthermore, we illuminate the limitations of current in-context RL methods on complex environments and discuss future directions. To facilitate future research, we release datasets for four of the considered environments.

Data science pipelines inform and influence many daily decisions, from what we buy to who we work for and even where we live. When designed incorrectly, these pipelines can easily propagate social inequity and harm. Traditional solutions are technical in nature; e.g., mitigating biased algorithms. In this vision paper, we introduce a novel lens for promoting responsible data science using theories of behavior change that emphasize not only technical solutions but also the behavioral responsibility of practitioners. By integrating behavior change theories from cognitive psychology with data science workflow knowledge and ethics guidelines, we present a new perspective on responsible data science. We present example data science interventions in machine learning and visual data analysis, contextualized in behavior change theories that could be implemented to interrupt and redirect potentially suboptimal or negligent practices while reinforcing ethically conscious behaviors. We conclude with a call to action to our community to explore this new research area of behavior change interventions for responsible data science., Comment: 23 pages, 4 figures, to be published in CSCW 2025

How can we precisely estimate a large language model's (LLM) accuracy on questions belonging to a specific topic within a larger question-answering dataset? The standard direct estimator, which averages the model's accuracy on the questions in each subgroup, may exhibit high variance for subgroups (topics) with small sample sizes. Synthetic regression modeling, which leverages the model's accuracy on questions about other topics, may yield biased estimates that are too unreliable for large subgroups. We prescribe a simple yet effective solution: an empirical Bayes (EB) estimator that balances direct and regression estimates for each subgroup separately, improving the precision of subgroup-level estimates of model performance. Our experiments on multiple datasets show that this approach consistently provides more precise estimates of the LLM performance compared to the direct and regression approaches, achieving substantial reductions in the mean squared error. Confidence intervals for EB estimates also have near-nominal coverage and are narrower compared to those for the direct estimator. Additional experiments on tabular and vision data validate the benefits of this EB approach., Comment: To appear at EMNLP 2024

We present observations of CO(1--0) and CO(2--1) lines from the Institut de radioastronomie millim\'etrique (IRAM) 30m telescope toward 20 nearby, optically luminous type 2 quasars (QSO2s) and observations of [C II] 158$\mu$m line from the Stratospheric Observatory For Infrared Astronomy (SOFIA) for 5 QSO2s in the CO sample and 5 type 1 quasars (QSO1s). In the traditional evolutionary scenario explaining different types of QSOs, obscured QSO2s emerge from gas-rich mergers observed as luminous infrared galaxies (LIRGs) and then turn into unobscured QSO1s as the black holes clear out the obscuring material in a blow-out phase. We test the validity of this theoretical prediction by comparing the gas fractions and star formation efficiencies among LIRGs and QSOs. We find that CO luminosity, CO-derived gas masses and gas fractions in QSO1s are consistent with those estimated for QSO2s, while LIRGs exhibit a closer resemblance to QSO2s in terms of CO-derived gas masses and gas fractions, and [C II] luminosity. However, comparisons between [C II] luminosity and star formation tracers such as the CO and infrared luminosity imply additional sources of [C II] emission in QSO1s likely tracing neutral atomic or ionized gas. All three types of galaxies have statistically indistinguishable distributions of star formation efficiency. Our results are consistent with part of the evolutionary scenario where nearby QSO2s could emerge from LIRGs, but they are unlikely to be the precursors of nearby QSO1s., Comment: 32 pages, 10 figures, 7 tables; the complete set of Figure 1 is appended to the end of document. Under review (revisions submitted) by ApJ

Deep reinforcement learning (DRL) has shown success in diverse domains such as robotics, computer games, and recommendation systems. However, like any other software system, DRL-based software systems are susceptible to faults that pose unique challenges for debugging and diagnosing. These faults often result in unexpected behavior without explicit failures and error messages, making debugging difficult and time-consuming. Therefore, automating the monitoring and diagnosis of DRL systems is crucial to alleviate the burden on developers. In this paper, we propose RLExplorer, the first fault diagnosis approach for DRL-based software systems. RLExplorer automatically monitors training traces and runs diagnosis routines based on properties of the DRL learning dynamics to detect the occurrence of DRL-specific faults. It then logs the results of these diagnoses as warnings that cover theoretical concepts, recommended practices, and potential solutions to the identified faults. We conducted two sets of evaluations to assess RLExplorer. Our first evaluation of faulty DRL samples from Stack Overflow revealed that our approach can effectively diagnose real faults in 83% of the cases. Our second evaluation of RLExplorer with 15 DRL experts/developers showed that (1) RLExplorer could identify 3.6 times more defects than manual debugging and (2) RLExplorer is easily integrated into DRL applications., Comment: Accepted for publication in The International Conference on Software Maintenance and Evolution (ICSME 2024)

Multimodal learning seeks to combine data from multiple input sources to enhance the performance of different downstream tasks. In real-world scenarios, performance can degrade substantially if some input modalities are missing. Existing methods that can handle missing modalities involve custom training or adaptation steps for each input modality combination. These approaches are either tied to specific modalities or become computationally expensive as the number of input modalities increases. In this paper, we propose Masked Modality Projection (MMP), a method designed to train a single model that is robust to any missing modality scenario. We achieve this by randomly masking a subset of modalities during training and learning to project available input modalities to estimate the tokens for the masked modalities. This approach enables the model to effectively learn to leverage the information from the available modalities to compensate for the missing ones, enhancing missing modality robustness. We conduct a series of experiments with various baseline models and datasets to assess the effectiveness of this strategy. Experiments demonstrate that our approach improves robustness to different missing modality scenarios, outperforming existing methods designed for missing modalities or specific modality combinations.

In this paper, we propose a novel and generalizable zero-shot knowledge transfer framework that distills expert sports navigation strategies from web videos into robotic systems with adversarial constraints and out-of-distribution image trajectories. Our pipeline enables diffusion-based imitation learning by reconstructing the full 3D task space from multiple partial views, warping it into 2D image space, closing the planning loop within this 2D space, and transfer constrained motion of interest back to task space. Additionally, we demonstrate that the learned policy can serve as a local planner in conjunction with position control. We apply this framework in the wheelchair tennis navigation problem to guide the wheelchair into the ball-hitting region. Our pipeline achieves a navigation success rate of 97.67% in reaching real-world recorded tennis ball trajectories with a physical robot wheelchair, and achieve a success rate of 68.49% in a real-world, real-time experiment on a full-sized tennis court., Comment: This manuscript has been submitted to 2025 IEEE International Conference on Robotics & Automation (ICRA)

Interactions between active individuals in animal collectives lead to emergent responses that remain elusive in synthetic soft matter. Here, shape-morphing polymers are used to create bio-inspired transient solids that self-assemble with controlled mechanical properties and disassemble on demand. Dilute-suspensions of magnetic, heat-responsive liquid crystal elastomer ribbons mechanically interlock, inducing reversible aggregation. A mathematical model is developed that sheds light on the role of topological mechanisms in aggregation. Aggregation was favored for ribbons with moderate curvature at 25C above crosslinking temperature as compared to flat ribbons or higher curvature ribbons at higher temperatures. The ribbon suspensions reversibly transition between fluid- and solid-like states, exhibiting up to 6 orders-of-magnitude increase in the storage moduli of the entangled aggregates compared with the liquid dispersions. Controlled dissociation is induced by imparting kinetic energy to the individual ribbons at high magnetic field rotation speeds (> 200 RPM). Ribbon shape and the medium in which dissociation occurs were shown to govern disassembly. Imparting dynamic collective behaviors into synthetic systems may enable a range of potential applications from bio-inspired soft robotics to injectable biomaterials., Comment: 16 pages, 6 figures

We present Gemini near-infrared integral field spectrograph (NIFS) K-band observations of the central 400 pc of NGC 1266, a nearby (D$\approx$30 Mpc) post-starburst galaxy with a powerful multi-phase outflow and a shocked ISM. We detect 7 H$_2$ ro-vibrational emission lines excited thermally to $T$$\sim$2000 K, and weak Br$\gamma$ emission, consistent with a fast C-shock. With these bright H$_2$ lines, we observe the spatial structure of the shock with an unambiguous tracer for the first time. The Br$\gamma$ emission is concentrated in the central $\lesssim$100 pc, indicating that any remaining star-formation in NGC 1266 is in the nucleus while the surrounding cold molecular gas has little on-going star-formation. Though it is unclear what fraction of this Br$\gamma$ emission is from star-formation or the AGN, assuming it is entirely due to star-formation we measure an instantaneous star-formation rate of 0.7 M$_\odot$ yr$^{-1}$, though the star-formation rate may be significantly higher in the presence of additional extinction. NGC 1266 provides a unique laboratory to study the complex interactions between AGN, outflows, shocks, and star-formation, all of which are necessary to unravel the evolution of the post-starburst phase., Comment: ApJ accepted

Large Language Models (LLM) have emerged as a tool for robots to generate task plans using common sense reasoning. For the LLM to generate actionable plans, scene context must be provided, often through a map. Recent works have shifted from explicit maps with fixed semantic classes to implicit open vocabulary maps based on queryable embeddings capable of representing any semantic class. However, embeddings cannot directly report the scene context as they are implicit, requiring further processing for LLM integration. To address this, we propose an explicit text-based map that can represent thousands of semantic classes while easily integrating with LLMs due to their text-based nature by building upon large-scale image recognition models. We study how entities in our map can be localized and show through evaluations that our text-based map localizations perform comparably to those from open vocabulary maps while using two to four orders of magnitude less memory. Real-robot experiments demonstrate the grounding of an LLM with the text-based map to solve user tasks.

We characterize the squared prediction risk of ensemble estimators obtained through subagging (subsample bootstrap aggregating) regularized M-estimators and construct a consistent estimator for the risk. Specifically, we consider a heterogeneous collection of $M \ge 1$ regularized M-estimators, each trained with (possibly different) subsample sizes, convex differentiable losses, and convex regularizers. We operate under the proportional asymptotics regime, where the sample size $n$, feature size $p$, and subsample sizes $k_m$ for $m \in [M]$ all diverge with fixed limiting ratios $n/p$ and $k_m/n$. Key to our analysis is a new result on the joint asymptotic behavior of correlations between the estimator and residual errors on overlapping subsamples, governed through a (provably) contractible nonlinear system of equations. Of independent interest, we also establish convergence of trace functionals related to degrees of freedom in the non-ensemble setting (with $M = 1$) along the way, extending previously known cases for square loss and ridge, lasso regularizers. When specialized to homogeneous ensembles trained with a common loss, regularizer, and subsample size, the risk characterization sheds some light on the implicit regularization effect due to the ensemble and subsample sizes $(M,k)$. For any ensemble size $M$, optimally tuning subsample size yields sample-wise monotonic risk. For the full-ensemble estimator (when $M \to \infty$), the optimal subsample size $k^\star$ tends to be in the overparameterized regime $(k^\star \le \min\{n,p\})$, when explicit regularization is vanishing. Finally, joint optimization of subsample size, ensemble size, and regularization can significantly outperform regularizer optimization alone on the full data (without any subagging)., Comment: 72 pages, 14 figures, 6 tables

We present the CO(1-0) maps of 28 infrared-bright galaxies from the Great Observatories All-Sky Luminous Infrared Galaxy Survey (GOALS) taken with the Combined Array for Research in Millimeter Astronomy (CARMA). We detect 100GHz continuum in 16 of 28 galaxies, which trace both active galactic nuclei (AGNs) and compact star-forming cores. The GOALS galaxies show a variety of molecular gas morphologies, though in the majority of cases, the average velocity fields show a gradient consistent with rotation. We fit the full continuum SEDs of each of the source using either MAGPHYS or SED3FIT (if there are signs of an AGN) to derive the total stellar mass, dust mass, and star formation rates of each object. We adopt a value determined from luminous and ultraluminous infrared galaxies (LIRGs and ULIRGs) of $\alpha_{\rm CO}=1.5^{+1.3}_{-0.8}~M_\odot$ (K km s$^{-1}$ pc$^2)^{-1}$, which leads to more physical values for $f_{\rm mol}$ and the gas-to-dust ratio. Mergers tend to have the highest gas-to-dust ratios. We assume the cospatiality of the molecular gas and star formation, and plot the sample on the Schmidt-Kennicutt relation, we find that they preferentially lie above the line set by normal star-forming galaxies. This hyper-efficiency is likely due to the increased turbulence in these systems, which decreases the freefall time compared to star-forming galaxies, leading to "enhanced" star formation efficiency. Line wings are present in a non-negligible subsample (11/28) of the CARMA GOALS sources and are likely due to outflows driven by AGNs or star formation, gas inflows, or additional decoupled gas components., Comment: 29 pages, 4 tables, 11 figures, Accepted by the Astrophysical Journal

In an effort to develop a rapid, reliable, and cost-effective method for predicting the structure of single-phase high entropy alloys, a Graph Neural Network (ALIGNN-FF) based approach was introduced. This method was successfully tested on 132 different high entropy alloys, and the results were analyzed and compared with density functional theory and valence electron concentration calculations. Additionally, the effects of various factors, including lattice parameters and the number of supercells with unique atomic configurations, on the prediction accuracy were investigated. The ALIGNN-FF based approach was subsequently used to predict the structure of a novel cobalt-free 3d high entropy alloy, and the result was experimentally verified., Comment: 16 pages, 8 Figures. To be published in Results Materials

In 2022, the Boston Opportunity Agenda, the Birth to Eight Collaborative and the City of Boston's Office of Early Childhood partnered in the development of this survey of hundreds of early education professionals in the city of Boston, and then expanded its reach to total more than 600 respondents across the state. In it, educators and administrators shared their demographics and educational background, as well as elements of their work experience, wages, and more. Taken together, the data portray a diverse workforce of dedicated professionals who also must manage low wages and long hours in a system that seeks to attract thousands of new workers to replace retiring workers and meet the need for more available early education seats.

We study the implicit regularization effects induced by (observation) weighting of pretrained features. For weight and feature matrices of bounded operator norms that are infinitesimally free with respect to (normalized) trace functionals, we derive equivalence paths connecting different weighting matrices and ridge regularization levels. Specifically, we show that ridge estimators trained on weighted features along the same path are asymptotically equivalent when evaluated against test vectors of bounded norms. These paths can be interpreted as matching the effective degrees of freedom of ridge estimators fitted with weighted features. For the special case of subsampling without replacement, our results apply to independently sampled random features and kernel features and confirm recent conjectures (Conjectures 7 and 8) of the authors on the existence of such paths in Patil et al. We also present an additive risk decomposition for ensembles of weighted estimators and show that the risks are equivalent along the paths when the ensemble size goes to infinity. As a practical consequence of the path equivalences, we develop an efficient cross-validation method for tuning and apply it to subsampled pretrained representations across several models (e.g., ResNet-50) and datasets (e.g., CIFAR-100)., Comment: 19 pages for main and 19 pages for appendix

The current standard for detecting human epidermal growth factor receptor 2 (HER2) status in breast cancer patients relies on HER2 amplification, identified through fluorescence in situ hybridization (FISH) or immunohistochemistry (IHC). However, hematoxylin and eosin (H\&E) tumor stains are more widely available, and accurately predicting HER2 status using H\&E could reduce costs and expedite treatment selection. Deep Learning algorithms for H&E have shown effectiveness in predicting various cancer features and clinical outcomes, including moderate success in HER2 status prediction. In this work, we employed a customized weak supervision classification technique combined with MoCo-v2 contrastive learning to predict HER2 status. We trained our pipeline on 182 publicly available H&E Whole Slide Images (WSIs) from The Cancer Genome Atlas (TCGA), for which annotations by the pathology team at Yale School of Medicine are publicly available. Our pipeline achieved an Area Under the Curve (AUC) of 0.85 across four different test folds. Additionally, we tested our model on 44 H&E slides from the TCGA-BRCA dataset, which had an HER2 score of 2+ and included corresponding HER2 status and FISH test results. These cases are considered equivocal for IHC, requiring an expensive FISH test on their IHC slides for disambiguation. Our pipeline demonstrated an AUC of 0.81 on these challenging H&E slides. Reducing the need for FISH test can have significant implications in cancer treatment equity for underserved populations.

This paper addresses the pressing need for an accurate solar energy prediction model, which is crucial for efficient grid integration. We explore the influence of the Air Quality Index and weather features on solar energy generation, employing advanced Machine Learning and Deep Learning techniques. Our methodology uses time series modeling and makes novel use of power transform normalization and zero-inflated modeling. Various Machine Learning algorithms and Conv2D Long Short-Term Memory model based Deep Learning models are applied to these transformations for precise predictions. Results underscore the effectiveness of our approach, demonstrating enhanced prediction accuracy with Air Quality Index and weather features. We achieved a 0.9691 $R^2$ Score, 0.18 MAE, 0.10 RMSE with Conv2D Long Short-Term Memory model, showcasing the power transform technique's innovation in enhancing time series forecasting for solar energy generation. Such results help our research contribute valuable insights to the synergy between Air Quality Index, weather features, and Deep Learning techniques for solar energy prediction., Comment: Accepted at AISD2024 : Second International Workshop on Artificial Intelligence: Empowering Sustainable Development

Control landscape phase transitions (CLPTs) occur as abrupt changes in the cost function landscape upon varying a control parameter, and can be revealed by non-analytic points in statistical order parameters. A prime example are quantum speed limits (QSL) which mark the onset of controllability as the protocol duration is increased. Here we lay the foundations of an analytical theory for CLPTs by developing Dyson, Magnus, and cumulant expansions for the cost function that capture the behavior of CLPTs with a controlled precision. Using linear and quadratic stability analysis, we reveal that CLPTs can be associated with different types of instabilities of the optimal protocol. This allows us to explicitly relate CLPTs to critical structural rearrangements in the extrema of the control landscape: utilizing path integral methods from statistical field theory, we trace back the critical scaling of the order parameter at the QSL to the topological and geometric properties of the set of optimal protocols, such as the number of connected components and its dimensionality. We verify our predictions by introducing a numerical sampling algorithm designed to explore this optimal set via a homotopic stochastic update rule. We apply this new toolbox explicitly to analyze CLPTs in the single- and two-qubit control problems whose landscapes are analytically tractable, and compare the landscapes for bang-bang and continuous protocols. Our work provides the first steps towards a systematic theory of CLPTs and paves the way for utilizing statistical field theory techniques for generic complex control landscapes., Comment: 38 pages, 24 figures

We study undominated mechanisms with transfers for regulating a monopolist who privately observes the marginal cost of production. We show that in any undominated mechanism, there is a quantity floor, which depends only on the primitives, and the regulator's operation decision is stochastic only if the monopolist produces at the quantity floor. We provide a near-complete characterization of the set of undominated mechanisms and use it to (a) provide a foundation for deterministic mechanisms, (b) show that the efficient mechanism is dominated, and (c) derive a max-min optimal regulatory mechanism.

This paper addresses the limitations of traditional keyword-based search in understanding user intent and introduces a novel hybrid search approach that leverages the strengths of non-semantic search engines, Large Language Models (LLMs), and embedding models. The proposed system integrates keyword matching, semantic vector embeddings, and LLM-generated structured queries to deliver highly relevant and contextually appropriate search results. By combining these complementary methods, the hybrid approach effectively captures both explicit and implicit user intent.The paper further explores techniques to optimize query execution for faster response times and demonstrates the effectiveness of this hybrid search model in producing comprehensive and accurate search outcomes.

The integration of topology into photonics has generated a new design framework for constructing robust and unidirectional waveguides, which are not feasible with traditional photonic devices. Here, we overcome current barriers to the successful integration of quantum emitters such as quantum dots (QDs) into valley-Hall (VH) topological waveguides, utilising photonic defects at the topological interface to stabilise the local charge environment and inverse design for efficient topological-conventional mode conversion. By incorporating QDs within defects of VH-photonic crystals, we demonstrate the first instances of single-photon resonant fluorescence and resonant transmission spectroscopy of a quantum emitter at a topological waveguide interface. Our results bring together topological photonics with optical nonlinear effects at the single-photon level, offering a new avenue to investigate the interaction between topology and quantum nonlinear systems.

Entanglement is essential for quantum information processing but is limited by noise. We address this by developing high-yield entanglement distillation protocols with several advancements. (1) We extend the 2-to-1 recurrence entanglement distillation protocol to higher-rate n-to-(n-1) protocols that can correct any single-qubit errors. These protocols are evaluated through numerical simulations focusing on fidelity and yield. We also outline a method to adapt any classical error-correcting code for entanglement distillation, where the code can correct both bit-flip and phase-flip errors by incorporating Hadamard gates. (2) We propose a constant-depth decoder for stabilizer codes that transforms logical states into physical ones using single-qubit measurements. This decoder is applied to entanglement distillation protocols, reducing circuit depth and enabling protocols derived from advanced quantum error-correcting codes. We demonstrate this by evaluating the circuit complexity for entanglement distillation protocols based on surface codes and quantum convolutional codes. (3) Our stabilizer entanglement distillation techniques advance quantum computing. We propose a fault-tolerant protocol for constant-depth encoding and decoding of arbitrary quantum states, applicable to quantum low-density parity-check (qLDPC) codes and surface codes. This protocol is feasible with state-of-the-art reconfigurable atom arrays and surpasses the limits of conventional logarithmic depth encoders. Overall, our study integrates stabilizer formalism, measurement-based quantum computing, and entanglement distillation, advancing both quantum communication and computing., Comment: 19 pages, 7 figures

Single-shot magnetic resonance (MR) imaging acquires the entire k-space data in a single shot and it has various applications in whole-body imaging. However, the long acquisition time for the entire k-space in single-shot fast spin echo (SSFSE) MR imaging poses a challenge, as it introduces T2-blur in the acquired images. This study aims to enhance the reconstruction quality of SSFSE MR images by (a) optimizing the trajectory for measuring the k-space, (b) acquiring fewer samples to speed up the acquisition process, and (c) reducing the impact of T2-blur. The proposed method adheres to physics constraints due to maximum gradient strength and slew-rate available while optimizing the trajectory within an end-to-end learning framework. Experiments were conducted on publicly available fastMRI multichannel dataset with 8-fold and 16-fold acceleration factors. An experienced radiologist's evaluation on a five-point Likert scale indicates improvements in the reconstruction quality as the ACL fibers are sharper than comparative methods.

Machine-learning models can be trained to predict the converged electron charge density of a density functional theory (DFT) calculation. In general, the value of the density at a given point in space is invariant under global translations and rotations having that point as a centre. Hence, one can construct locally invariant machine-learning density predictors. However, the widely used projector augmented wave (PAW) implementation of DFT requires the evaluation of the one-center augmentation contributions, that are not rotationally invariant. Building on our recently proposed Jacobi-Legendre charge-density scheme, we construct a covariant Jacobi-Legendre model capable of predicting the local occupancies needed to compose the augmentation charge density. Our formalism is then applied to the prediction of the energy barrier for the 1H-to-1T phase transition of two-dimensional MoS$_2$. With extremely modest training, the model is capable of performing a non-self-consistent nudged elastic band calculation at virtually the same accuracy as a fully DFT-converged one, thus saving thousands of self-consistent DFT steps. Furthermore, at variance with machine-learning force fields, the charge density is here available for any nudged elastic band image, so that we can trace the evolution of the electronic structure across the phase transition., Comment: 12 pages, 4 figures

Radiance field methods such as Neural Radiance Fields (NeRFs) or 3D Gaussian Splatting (3DGS), have revolutionized graphics and novel view synthesis. Their ability to synthesize new viewpoints with photo-realistic quality, as well as capture complex volumetric and specular scenes, makes them an ideal visualization for robotic teleoperation setups. Direct camera teleoperation provides high-fidelity operation at the cost of maneuverability, while reconstruction-based approaches offer controllable scenes with lower fidelity. With this in mind, we propose replacing the traditional reconstruction-visualization components of the robotic teleoperation pipeline with online Radiance Fields, offering highly maneuverable scenes with photorealistic quality. As such, there are three main contributions to state of the art: (1) online training of Radiance Fields using live data from multiple cameras, (2) support for a variety of radiance methods including NeRF and 3DGS, (3) visualization suite for these methods including a virtual reality scene. To enable seamless integration with existing setups, these components were tested with multiple robots in multiple configurations and were displayed using traditional tools as well as the VR headset. The results across methods and robots were compared quantitatively to a baseline of mesh reconstruction, and a user study was conducted to compare the different visualization methods. For videos and code, check out https://leggedrobotics.github.io/rffr.github.io/., Comment: 8 pages, 10 figures, Accepted to IROS 2024

This work addresses the problem of finding a consensus point in the state space ($\mathbb{R}^2$) for a multi-agent system that is comprised of $N$ identical double integrator agents. It is assumed that each agent operates under constrained control input (i.e., $|u_i(t)| \leq 1$ $\forall i = 1, \hdots N$). Further, a fixed fuel budget is also assumed i.e., the total amount of cumulative input that can be expended is limited by $\int_0^{t_f}|u(t)|dt \le \beta$. First, the attainable set $\mathcal{A}(t,x_0,\beta)$ at time $t$, which is the set of all states that an agent can attain starting from initial conditions $x_0$ under the fuel budget constraints at time $t$ is computed for every agent. This attainable set is a convex set for all $t\ge0$. Then the minimum time to consensus is the minimum time $\bar{t}$ at which attainable sets of all agents intersect, and the consensus point is the point of intersection. A closed-form expression for the minimum time consensus point is provided for the case of three agents. Then, using Helly's theorem, the intersection will be non-empty at a time when all the $N \choose 3$ triplets of agents have non-empty intersection. The computation of minimum time consensus for all $N \choose 3$ triplets is performed independently and can be distributed among all the $N$ agents. Finally, the overall minimum time to consensus is given by the triplet that has the highest minimum time to consensus. Further, the intersection of all the attainable sets of this triplet gives the minimum time consensus point for all $N$ agents.

In this work, we explore the usage of the Frequency Transformation for reducing the domain shift between the source and target domain (e.g., synthetic image and real image respectively) towards solving the Domain Adaptation task. Most of the Unsupervised Domain Adaptation (UDA) algorithms focus on reducing the global domain shift between labelled source and unlabelled target domains by matching the marginal distributions under a small domain gap assumption. UDA performance degrades for the cases where the domain gap between source and target distribution is large. In order to bring the source and the target domains closer, we propose a novel approach based on traditional image processing technique Class Aware Frequency Transformation (CAFT) that utilizes pseudo label based class consistent low-frequency swapping for improving the overall performance of the existing UDA algorithms. The proposed approach, when compared with the state-of-the-art deep learning based methods, is computationally more efficient and can easily be plugged into any existing UDA algorithm to improve its performance. Additionally, we introduce a novel approach based on absolute difference of top-2 class prediction probabilities (ADT2P) for filtering target pseudo labels into clean and noisy sets. Samples with clean pseudo labels can be used to improve the performance of unsupervised learning algorithms. We name the overall framework as CAFT++. We evaluate the same on the top of different UDA algorithms across many public domain adaptation datasets. Our extensive experiments indicate that CAFT++ is able to achieve significant performance gains across all the popular benchmarks., Comment: Accepted at the International Journal of Computer Vision

This paper presents Hardware Description Language Generative Pre-trained Transformers (HDL-GPT), a novel approach that leverages the vast repository of open-source High Definition Language (HDL) codes to train superior quality large code models. The core premise of this paper is the hypothesis that high-quality HDL is all you need to create models with exceptional performance and broad zero-shot generalization abilities. The paper elucidates the methods employed for the curation and augmentation of large corpora from open-source HDL code, transforming highly variable quality data into high-quality data through careful prompting and context maintenance. We demonstrate that the careful selection, filtering, and augmentation of data across HDLs can yield powerful models that surpass current state-of-the-art models. We also explore the impact of different fine-tuning methods on the quality of results. We describe experimental results across a range of fine-tuned SOTA LLMs, substantiating our claims. We demonstrate improvements of 50% to 200% over SOTA HDL models on current benchmarks in tasks ranging from HDL circuit explanations, code generation, formal and simulation testbench creation, triaging bugs, and fixing them. HDL-GPT opens new avenues for the development of advanced model training techniques for circuit design tasks., Comment: DAC 2024 Invited Paper

Metaverse applications desire to communicate with semantically identified objects among a diverse set of cyberspace entities, such as cameras for collecting images from, sensors for sensing environment, and users collaborating with each other, all could be nearby or far away, in a timely and secure way. However, supporting the above function faces networking challenges. Today's metaverse implementations are, by and large, use secure transport connections to communicate with cloud servers instead of letting participating entities communicate directly. In this paper, we use the design and implementation of NDN Workspace, a web-based, multi-user collaborative app to showcase a new way to networking that supports many-to-many secure data exchanges among communicating entities directly. NDN Workspace users establish trust relations among each other, exchange URI-identified objects directly, and can collaborate through intermittent connectivity, all in the absence of cloud servers. Its data-centric design offers an exciting new approach to metaverse app development.

This position paper explores how to support the Web's evolution through an underlying data-centric approach that better matches the data-orientedness of modern and emerging applications. We revisit the original vision of the Web as a hypermedia system that supports document composability and application interoperability via name-based data access. We propose the use of secure web objects (SWO), a data-oriented communication approach that can reduce complexity, centrality, and inefficiency, particularly for collaborative and local-first applications, such as the Metaverse and other collaborative applications. SWO are named, signed, application-defined objects that are secured independently of their containers or communications channels, an approach that leverages the results from over a decade-long data-centric networking research. This approach does not require intermediation by aggregators of identity, storage, and other services that are common today. We present a brief design overview, illustrated through prototypes for two editors of shared hypermedia documents: one for 3D and one for LaTeX. We also discuss our findings and suggest a roadmap for future research., Comment: 9 pages

We propose a straightforward yet highly effective few-shot fine-tuning strategy for adapting the Segment Anything (SAM) to anatomical segmentation tasks in medical images. Our novel approach revolves around reformulating the mask decoder within SAM, leveraging few-shot embeddings derived from a limited set of labeled images (few-shot collection) as prompts for querying anatomical objects captured in image embeddings. This innovative reformulation greatly reduces the need for time-consuming online user interactions for labeling volumetric images, such as exhaustively marking points and bounding boxes to provide prompts slice by slice. With our method, users can manually segment a few 2D slices offline, and the embeddings of these annotated image regions serve as effective prompts for online segmentation tasks. Our method prioritizes the efficiency of the fine-tuning process by exclusively training the mask decoder through caching mechanisms while keeping the image encoder frozen. Importantly, this approach is not limited to volumetric medical images, but can generically be applied to any 2D/3D segmentation task. To thoroughly evaluate our method, we conducted extensive validation on four datasets, covering six anatomical segmentation tasks across two modalities. Furthermore, we conducted a comparative analysis of different prompting options within SAM and the fully-supervised nnU-Net. The results demonstrate the superior performance of our method compared to SAM employing only point prompts (approximately 50% improvement in IoU) and performs on-par with fully supervised methods whilst reducing the requirement of labeled data by at least an order of magnitude., Comment: 9 pages, Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision. 2024

An optically high-quality single crystal of sodium mesitylene sulfonate crystal was successfully grown by a slow evaporation method using methanol as solvent at room temperature. Single-crystal XRD has characterized the material and belongs to a monoclinic structure with a C2 space group. Functional groups were determined using Fourier-transformed infrared spectroscopy. The optical quality of the generated crystal was evaluated using UV-Vis NIR spectral analysis, which is transparent in the range of 300-1500 nm. We report the optical properties using terahertz time-domain spectroscopy (THz-TDS) and THz generation using crystal.

Large Language Models(LLMs) have shown exceptional abilities, yet training these models can be quite challenging. There is a strong dependence on the quality of data and finding the best instruction tuning set. Further, the inherent limitations in training methods create substantial difficulties to train relatively smaller models with 7B and 13B parameters. In our research, we suggest an improved training method for these models by utilising knowledge from larger models, such as a mixture of experts (8x7B) architectures. The scale of these larger models allows them to capture a wide range of variations from data alone, making them effective teachers for smaller models. Moreover, we implement a novel post-training domain alignment phase that employs domain-specific expert models to boost domain-specific knowledge during training while preserving the model's ability to generalise. Fine-tuning Mistral 7B and 2x7B with our method surpasses the performance of state-of-the-art language models with more than 7B and 13B parameters: achieving up to $7.9$ in MT-Bench and $93.04\%$ on AlpacaEval., Comment: 7 pages, 4 figures