Search

Multimodal AI has demonstrated superior performance over unimodal approaches by leveraging diverse data sources for more comprehensive analysis. However, applying this effectiveness in healthcare is challenging due to the limited availability of public datasets. Federated learning presents an exciting solution, allowing the use of extensive databases from hospitals and health centers without centralizing sensitive data, thus maintaining privacy and security. Yet, research in multimodal federated learning, particularly in scenarios with missing modalities a common issue in healthcare datasets remains scarce, highlighting a critical area for future exploration. Toward this, we propose a novel method for multimodal federated learning with missing modalities. Our contribution lies in a novel cross-modal data augmentation by retrieval, leveraging the small publicly available dataset to fill the missing modalities in the clients. Our method learns the parameters in a federated manner, ensuring privacy protection and improving performance in multiple challenging multimodal benchmarks in the medical domain, surpassing several competitive baselines. Code Available: https://github.com/bhattarailab/CAR-MFL, Comment: Accepted at MICCAI 2024

We present and analyze the extensive optical broadband photometry of the Type II SN 2023ixf up to one year after explosion. We find that, when compared to two pre-existing model grids, the pseudo-bolometric light curve is consistent with drastically different combinations of progenitor and explosion properties. This may be an effect of known degeneracies in Type IIP light-curve models. We independently compute a large grid of ${\tt MESA+STELLA}$ single-star progenitor and light-curve models with various zero-age main-sequence masses, mass-loss efficiencies, and convective efficiencies. Using the observed progenitor variability as an additional constraint, we select stellar models consistent with the pulsation period and explode them according to previously established scaling laws to match plateau properties. Our hydrodynamic modeling indicates that SN 2023ixf is most consistent with a moderate-energy ($E_{\rm exp}\approx7\times10^{50}$ erg) explosion of an initially high-mass red supergiant progenitor ($\gtrsim 17\ M_{\odot}$) that lost a significant amount of mass in its prior evolution, leaving a low-mass hydrogen envelope ($\lesssim 3\ M_{\odot}$) at the time of explosion, with a radius $\gtrsim 950\ R_{\odot}$ and a synthesized $^{56}$Ni mass of $0.07\ M_{\odot}$. We posit that previous mass transfer in a binary system may have stripped the envelope of SN 2023ixf's progenitor. The analysis method with pulsation period presented in this work offers a way to break degeneracies in light-curve modeling in the future, particularly with the upcoming Vera C.~Rubin Observatory Legacy Survey of Space and Time, when a record of progenitor variability will be more common., Comment: 18 pages, 7 figures, submitted to ApJ. Comments welcome

With the advent of multi-modal large language models (MLLMs), datasets used for visual question answering (VQA) and referring expression comprehension have seen a resurgence. However, the most popular datasets used to evaluate MLLMs are some of the earliest ones created, and they have many known problems, including extreme bias, spurious correlations, and an inability to permit fine-grained analysis. In this paper, we pioneer evaluating recent MLLMs (LLaVA 1.5, LLaVA-NeXT, BLIP2, InstructBLIP, GPT-4V, and GPT-4o) on datasets designed to address weaknesses in earlier ones. We assess three VQA datasets: 1) TDIUC, which permits fine-grained analysis on 12 question types; 2) TallyQA, which has simple and complex counting questions; and 3) DVQA, which requires optical character recognition for chart understanding. We also study VQDv1, a dataset that requires identifying all image regions that satisfy a given query. Our experiments reveal the weaknesses of many MLLMs that have not previously been reported. Our code is integrated into the widely used LAVIS framework for MLLM evaluation, enabling the rapid assessment of future MLLMs. Project webpage: https://kevinlujian.github.io/MLLM_Evaluations/

SN 2023ixf was discovered in M101 within a day of explosion and rapidly classified as a Type II supernova with flash features. Here we present ultraviolet (UV) spectra obtained with the Hubble Space Telescope 14, 19, 24, and 66 days after explosion. Interaction between the supernova ejecta and circumstellar material (CSM) is seen in the UV throughout our observations in the flux of the first three epochs and asymmetric MgII emission at day 66. We compare our observations to CMFGEN supernova models which include CSM interaction ($\dot{M}<10^{-3}$ $M_{\odot}$/yr) and find that the power from CSM interaction is decreasing with time, from $L_{\rm sh}\approx5\times10^{42}$ erg/s at day 14 to $L_{sh}\approx1\times10^{40}$ erg/s at day 66. We examine the contribution of individual atomic species to the spectra at day 14 and 19, showing that the majority of the features are dominated by iron, nickel, magnesium, and chromium absorption in the ejecta. The UV spectral energy distribution of SN 2023ixf sits between that of supernovae which show no definitive signs of CSM interaction and those with persistent signatures assuming the same progenitor radius and metallicity. Finally, we show that the evolution and asymmetric shape of the MgII emission are not unique to SN 2023ixf. These observations add to the early measurements of dense, confined CSM interaction, tracing the mass-loss history of SN 2023ixf to $\sim33$ yr prior to explosion and the density profile to a radius of $\sim5.7\times10^{15}$ cm. They show the relatively short evolution from quiescent red supergiant wind to high mass loss., Comment: Submitted to ApJ, comments welcome

We introduce the Masked Humanoid Controller (MHC) for whole-body tracking of target trajectories over arbitrary subsets of humanoid state variables. This enables the realization of whole-body motions from diverse sources such as video, motion capture, and VR, while ensuring balance and robustness against disturbances. The MHC is trained in simulation using a carefully designed curriculum that imitates partially masked motions from a library of behaviors spanning pre-trained policy rollouts, optimized reference trajectories, re-targeted video clips, and human motion capture data. We showcase simulation experiments validating the MHC's ability to execute a wide variety of behavior from partially-specified target motions. Moreover, we also highlight sim-to-real transfer as demonstrated by real-world trials on the Digit humanoid robot. To our knowledge, this is the first instance of a learned controller that can realize whole-body control of a real-world humanoid for such diverse multi-modal targets., Comment: Website: https://masked-humanoid.github.io/mhc/

We leverage generative large language models for language learning applications, focusing on estimating the difficulty of foreign language texts and simplifying them to lower difficulty levels. We frame both tasks as prediction problems and develop a difficulty classification model using labeled examples, transfer learning, and large language models, demonstrating superior accuracy compared to previous approaches. For simplification, we evaluate the trade-off between simplification quality and meaning preservation, comparing zero-shot and fine-tuned performances of large language models. We show that meaningful text simplifications can be obtained with limited fine-tuning. Our experiments are conducted on French texts, but our methods are language-agnostic and directly applicable to other foreign languages., Comment: 10 pages, 4 figures

General-purpose artificial intelligence (AI) systems are built on massive swathes of public web data, assembled into corpora such as C4, RefinedWeb, and Dolma. To our knowledge, we conduct the first, large-scale, longitudinal audit of the consent protocols for the web domains underlying AI training corpora. Our audit of 14,000 web domains provides an expansive view of crawlable web data and how codified data use preferences are changing over time. We observe a proliferation of AI-specific clauses to limit use, acute differences in restrictions on AI developers, as well as general inconsistencies between websites' expressed intentions in their Terms of Service and their robots.txt. We diagnose these as symptoms of ineffective web protocols, not designed to cope with the widespread re-purposing of the internet for AI. Our longitudinal analyses show that in a single year (2023-2024) there has been a rapid crescendo of data restrictions from web sources, rendering ~5%+ of all tokens in C4, or 28%+ of the most actively maintained, critical sources in C4, fully restricted from use. For Terms of Service crawling restrictions, a full 45% of C4 is now restricted. If respected or enforced, these restrictions are rapidly biasing the diversity, freshness, and scaling laws for general-purpose AI systems. We hope to illustrate the emerging crises in data consent, for both developers and creators. The foreclosure of much of the open web will impact not only commercial AI, but also non-commercial AI and academic research., Comment: 41 pages (13 main), 5 figures, 9 tables

In the quest toward realizing novel quantum matter in ultracold molecular gases, we perform a numerical study of evaporative cooling in ultracold gases of microwave-shielded polar fermionic molecules. Our Monte Carlo simulations incorporate accurate two-body elastic and inelastic scattering cross sections, realistic modeling of the optical dipole trap, and the influence of Pauli blocking at low temperatures. The simulations are benchmarked against data from evaporation studies performed with ultracold NaK molecules, showing excellent agreement. We further explore the prospects for optimizing the evaporation efficiency by varying the ramp rate and duration of the evaporation trajectory. Our simulation shows that it is possible to reach $< 10\%$ of the Fermi temperature under optimal conditions even in the presence of two-body molecular losses., Comment: 12 pages, 9 figures, 2 tables

We present multiwavelength observations of the Swift short $\gamma$-ray burst GRB 231117A, localized to an underlying galaxy at redshift $z = 0.257$ at a small projected offset ($\sim 2~$kpc). We uncover long-lived X-ray (Chandra) and radio/millimeter (VLA, MeerKAT, and ALMA) afterglow emission, detected to $\sim 37~$days and $\sim 20~$days (rest frame), respectively. We measure a wide jet ($\sim 10.4^\circ$) and relatively high circumburst density ($\sim 0.07~{\rm cm}^{-3}$) compared to the short GRB population. Our data cannot be easily fit with a standard forward shock model, but they are generally well fit with the incorporation of a refreshed forward shock and a reverse shock at $< 1~$day. We incorporate GRB 231117A into a larger sample of 132 X-ray detected events, 71 of which were radio-observed (17 cm-band detections), for a systematic study of the distributions of redshifts, jet and afterglow properties, galactocentric offsets, and local environments of events with and without detected radio afterglows. Compared to the entire short GRB population, the majority of radio-detected GRBs are at relatively low redshifts ($z < 0.6$) and have high circumburst densities ($> 10^{-2}~{\rm cm}^{-3}$), consistent with their smaller ($< 8~$kpc) projected galactocentric offsets. We additionally find that 70% of short GRBs with opening angle measurements were radio-detected, indicating the importance of radio afterglows in jet measurements, especially in the cases of wide ($> 10^\circ$) jets where observational evidence of collimation may only be detectable at radio wavelengths. Owing to improved observing strategies and the emergence of sensitive radio facilities, the number of radio-detected short GRBs has quadrupled in the past decade., Comment: 30 pages, 11 figures, submitted to ApJ

Seamless interaction between AI agents and humans using natural language remains a key goal in AI research. This paper addresses the challenges of developing interactive agents capable of understanding and executing grounded natural language instructions through the IGLU competition at NeurIPS. Despite advancements, challenges such as a scarcity of appropriate datasets and the need for effective evaluation platforms persist. We introduce a scalable data collection tool for gathering interactive grounded language instructions within a Minecraft-like environment, resulting in a Multi-Modal dataset with around 9,000 utterances and over 1,000 clarification questions. Additionally, we present a Human-in-the-Loop interactive evaluation platform for qualitative analysis and comparison of agent performance through multi-turn communication with human annotators. We offer to the community these assets referred to as IDAT (IGLU Dataset And Toolkit) which aim to advance the development of intelligent, interactive AI agents and provide essential resources for further research.

Payload transport over flat terrain via multi-wheel robot carriers is well-understood, highly effective, and configurable. In this paper, our goal is to provide similar effectiveness and configurability for transport over rough terrain that is more suitable for legs rather than wheels. For this purpose, we consider multi-biped robot carriers, where wheels are replaced by multiple bipedal robots attached to the carrier. Our main contribution is to design a decentralized controller for such systems that can be effectively applied to varying numbers and configurations of rigidly attached bipedal robots without retraining. We present a reinforcement learning approach for training the controller in simulation that supports transfer to the real world. Our experiments in simulation provide quantitative metrics showing the effectiveness of the approach over a wide variety of simulated transport scenarios. In addition, we demonstrate the controller in the real-world for systems composed of two and three Cassie robots. To our knowledge, this is the first example of a scalable multi-biped payload transport system., Comment: Submitted to CoRL 2024, Project website: decmbc.github.io

Variability can be the pathway to understanding the physical processes in astrophysical jets, however, the high-cadence observations required to test particle acceleration models are still missing. Here we report on the first attempt to produce continuous, >24 hour polarization light curves of blazars using telescopes distributed across the globe and the rotation of the Earth to avoid the rising Sun. Our campaign involved 16 telescopes in Asia, Europe, and North America. We observed BL Lacertae and CGRaBS J0211+1051 for a combined 685 telescope hours. We find large variations in the polarization degree and angle for both sources in sub-hour timescales as well as a ~180 degree rotation of the polarization angle in CGRaBS J0211+1051 in less than two days. We compared our high-cadence observations to Particle-In-Cell magnetic reconnection and turbulent plasma simulations. We find that although the state of the art simulation frameworks can produce a large fraction of the polarization properties, they do not account for the entirety of the observed polarization behavior in blazar jets., Comment: 20 pages, 15 figures, 2 tables, accepted for publication in A&A. The data used in the paper are available here: https://doi.org/10.7910/DVN/IETSXS

Measuring Deeply Virtual Compton Scattering on the neutron is one of the necessary steps to understand the structure of the nucleon in terms of Generalized Parton Distributions (GPDs). Neutron targets play a complementary role to transversely polarized proton targets in the determination of the GPD $E$. This poorly known and poorly constrained GPD is essential to obtain the contribution of the quarks' angular momentum to the spin of the nucleon. DVCS on the neutron was measured for the first time selecting the exclusive final state by detecting the neutron, using the Jefferson Lab longitudinally polarized electron beam, with energies up to 10.6 GeV, and the CLAS12 detector. The extracted beam-spin asymmetries, combined with DVCS observables measured on the proton, allow a clean quark-flavor separation of the imaginary parts of the GPDs $H$ and $E$., Comment: 7 pages, 6 figures

The Game of Life (GoL), one well known 2D cellular automaton, does not typically ensure interesting long-term phenotypic dynamics. Therefore, while being Turing complete, GoL cannot be said to be open-ended. In this work, we extend GoL with the opportunity for local mutations, thus enabling a heterogeneous life-like cellular automaton guided by an evolutionary inner loop. Additionally, we introduce the concept of cell ageing to ensure that cell aliveness (activated by inheritance with variation, and controlled by ageing) and actual cell computation (governed by life-like rules on local neighborhoods) are kept conceptually separated. We conduct an experimental campaign to identify suitable parameters that produce long-term phenotypic dynamics and favor genotypic innovations., Comment: 16 pages, 9 Figures

Cavity-enhanced emission of electrically controlled semiconductor quantum dots is essential in developing bright quantum devices for real-world quantum photonic applications. Combining the circular Bragg grating (CBG) approach with a PIN-diode structure, we propose and implement an innovative concept for ridge-based electrically-contacted CBG resonators. Through fine-tuning of device parameters in numerical simulations and deterministic nanoprocessing, we produced electrically controlled single quantum dot CBG resonators with excellent electro-optical emission properties. These include multiple wavelength-tunable emission lines and a photon extraction efficiency (PEE) of up to (30.4$\pm$3.4)%, where refined numerical optimization based on experimental findings suggests a substantial improvement, promising PEE >50%. Additionally, the developed quantum light sources yield single-photon purity reaching (98.8$\pm$0.2)% [post-selected: (99.5$\pm$0.3)%] and a photon indistinguishability of (25.8$\pm$2.1)% [post-selected: (92.8$\pm$4.8)%]. Our results pave the way for high-performance quantum devices with combined cavity enhancement and deterministic charge-environment controls, advancing the development of photonic quantum information systems such as complex quantum repeater networks.

Conformal phased arrays provide multiple degrees of freedom to the scan angle, which is typically limited by antenna aperture in rigid arrays. Silicon-based RF signal processing offers reliable, reconfigurable, multi-functional, and compact control for conformal phased arrays that can be used for on-the-move communication. While the lightweight, compactness, and shape-changing properties of the conformal phased arrays are attractive, these features result in dynamic deformation of the array during motion leading to significant dynamic beam pointing errors. We propose a silicon-based, compact, reconfigurable solution to self-correct these dynamic deformation-induced beam pointing errors. Furthermore, additive printing is leveraged to enhance the flexibility of the conformal phased arrays, as the printed conductive ink is more flexible than bulk copper and can be easily deposited on flexible sheets using different printing tools, providing an environmentally-friendly solution for large-scale production. The inks such as conventional silver inks are expensive and copper-based printable inks suffer from spontaneous metal oxidation that alters trace impedance and degrades beamforming performance. This work uses a low-cost molecular copper decomposition ink with reliable RF properties at different temperature and strain to print the proposed intelligent conformal phased array operating at 2.1 GHz. Proof-of-concept prototype $2\times2$ array self-corrects the deformation induces beampointing error with an error $<1.25^\circ$. The silicon based array processing part occupying only 2.58 mm$^2$ area and 83 mW power per tile.

Gaussian Splatting (GS) has proven to be highly effective in novel view synthesis, achieving high-quality and real-time rendering. However, its potential for reconstructing detailed 3D shapes has not been fully explored. Existing methods often suffer from limited shape accuracy due to the discrete and unstructured nature of Gaussian splats, which complicates the shape extraction. While recent techniques like 2D GS have attempted to improve shape reconstruction, they often reformulate the Gaussian primitives in ways that reduce both rendering quality and computational efficiency. To address these problems, our work introduces a rasterized approach to render the depth maps and surface normal maps of general 3D Gaussian splats. Our method not only significantly enhances shape reconstruction accuracy but also maintains the computational efficiency intrinsic to Gaussian Splatting. It achieves a Chamfer distance error comparable to NeuraLangelo on the DTU dataset and maintains similar computational efficiency as the original 3D GS methods. Our method is a significant advancement in Gaussian Splatting and can be directly integrated into existing Gaussian Splatting-based methods.

Object detection forms a key component in Unmanned Aerial Vehicles (UAVs) for completing high-level tasks that depend on the awareness of objects on the ground from an aerial perspective. In that scenario, adversarial patch attacks on an onboard object detector can severely impair the performance of upstream tasks. This paper proposes a novel model-agnostic defense mechanism against the threat of adversarial patch attacks in the context of UAV-based object detection. We formulate adversarial patch defense as an occlusion removal task. The proposed defense method can neutralize adversarial patches located on objects of interest, without exposure to adversarial patches during training. Our lightweight single-stage defense approach allows us to maintain a model-agnostic nature, that once deployed does not require to be updated in response to changes in the object detection pipeline. The evaluations in digital and physical domains show the feasibility of our method for deployment in UAV object detection pipelines, by significantly decreasing the Attack Success Ratio without incurring significant processing costs. As a result, the proposed defense solution can improve the reliability of object detection for UAVs., Comment: submitted to IROS 2024

We present high-cadence photometric and spectroscopic observations of supernova (SN) 2024ggi, a Type II SN with flash spectroscopy features which exploded in the nearby galaxy NGC 3621 at $\sim$7 Mpc. The light-curve evolution over the first 30 hours can be fit by two power law indices with a break after 22 hours, rising from $M_V \approx -12.95$ mag at +0.66 days to $M_V \approx -17.91$ mag after 7 days. In addition, the densely sampled color curve shows a strong blueward evolution over the first few days and then behaves as a normal SN II with a redward evolution as the ejecta cool. Such deviations could be due to interaction with circumstellar material (CSM). Early high- and low-resolution spectra clearly show high-ionization flash features from the first spectrum to +3.42 days after the explosion. From the high-resolution spectra, we calculate the CSM velocity to be 37 $\pm~4~\mathrm{km\,s^{-1}} $. We also see the line strength evolve rapidly from 1.22 to 1.49 days in the earliest high-resolution spectra. Comparison of the low-resolution spectra with CMFGEN models suggests that the pre-explosion mass-loss rate of SN 2024ggi falls in a range of $10^{-3}$ to $10^{-2}$ M$_{\odot}$ yr$^{-1}$, which is similar to that derived for SN 2023ixf. However, the rapid temporal evolution of the narrow lines in the spectra of SN 2024ggi ($R_\mathrm{CSM} \sim 2.7 \times 10^{14} \mathrm{cm}$) could indicate a smaller spatial extent of the CSM than in SN 2023ixf ($R_\mathrm{CSM} \sim 5.4 \times 10^{14} \mathrm{cm}$) which in turn implies lower total CSM mass for SN 2024ggi., Comment: 23 pages, 15 figures, 4 tables, accepted for publication in ApJL

The complexity of quantum states under dynamical evolution can be investigated by studying the spread with time of the state over a pre-defined basis. It is known that this complexity is minimised by choosing the Krylov basis, thus defining the spread complexity. We study the dynamics of spread complexity for quantum maps using the Arnoldi iterative procedure. The main illustrative quantum many-body model we use is the periodically kicked Ising spin-chain with non-integrable deformations, a chaotic system where we look at both local and non-local interactions. In the various cases we find distinctive behaviour of the Arnoldi coefficients and spread complexity for regular vs. chaotic dynamics: suppressed fluctuations in the Arnoldi coefficients as well as larger saturation value in spread complexity in the chaotic case. We compare the behaviour of the Krylov measures with that of standard spectral diagnostics of chaos. We also study the effect of changing the driving frequency on the complexity saturation., Comment: 23 pages, 13 figures, comments welcome

Deep Neural Networks (DNNs) are known to be vulnerable to adversarial examples. Further, these adversarial examples are found to be transferable from the source network in which they are crafted to a black-box target network. As the trend of using deep learning on embedded devices grows, it becomes relevant to study the transferability properties of adversarial examples among compressed networks. In this paper, we consider quantization as a network compression technique and evaluate the performance of transfer-based attacks when the source and target networks are quantized at different bitwidths. We explore how algorithm specific properties affect transferability by considering various adversarial example generation algorithms. Furthermore, we examine transferability in a more realistic scenario where the source and target networks may differ in bitwidth and other model-related properties like capacity and architecture. We find that although quantization reduces transferability, certain attack types demonstrate an ability to enhance it. Additionally, the average transferability of adversarial examples among quantized versions of a network can be used to estimate the transferability to quantized target networks with varying capacity and architecture.

Towards the need for automated and precise AI-based analysis of medical images, we present RT-utils, a specialized Python library tuned for the manipulation of radiotherapy (RT) structures stored in DICOM format. RT-utils excels in converting the polygon contours into binary masks, ensuring accuracy and efficiency. By converting DICOM RT structures into standardized formats such as NumPy arrays and SimpleITK Images, RT-utils optimizes inputs for computational solutions such as AI-based automated segmentation techniques or radiomics analysis. Since its inception in 2020, RT-utils has been used extensively with a focus on simplifying complex data processing tasks. RT-utils offers researchers a powerful solution to enhance workflows and drive significant advancements in medical imaging.

We present photometric and spectroscopic observations of SN 2023fyq, a type Ibn supernova in the nearby galaxy NGC 4388 (D$\simeq$18~Mpc). In addition, we trace long-standing precursor emission at the position of SN 2023fyq using data from DLT40, ATLAS, ZTF, ASAS-SN, Swift, and amateur astronomer Koichi Itagaki. Precursor activity is observed up to nearly three years before the supernova explosion, with a relatively rapid rise in the final 100 days. The double-peaked post-explosion light curve reaches a luminosity of $\sim10^{43}~\rm erg\,s^{-1}$. The strong intermediate-width He lines observed in the nebular spectrum of SN 2023fyq imply the interaction is still active at late phases. We found that the precursor activity in SN 2023fyq is best explained by the mass transfer in a binary system involving a low-mass He star and a compact companion. An equatorial disk is likely formed in this process ($\sim$0.6$\rm M_{\odot}$), and the interaction of SN ejecta with this disk powers the main peak of the supernova. The early SN light curve reveals the presence of dense extended material ($\sim$0.3$\rm M_{\odot}$) at $\sim$3000$\rm R_{\odot}$ ejected weeks before the SN explosion, likely due to final-stage core silicon burning or runaway mass transfer resulting from binary orbital shrinking, leading to rapid rising precursor emission within $\sim$30 days prior to explosion. The final explosion could be triggered either by the core-collapse of the He star or by the merger of the He star with a compact object. SN 2023fyq, along with SN 2018gjx and SN 2015G, forms a unique class of Type Ibn SNe which originate in binary systems and are likely to exhibit detectable long-lasting pre-explosion outbursts with magnitudes ranging from $-$10 to $-$13., Comment: submitted to ApJ

A necessary capability for humanoid robots is the ability to stand and walk while rejecting natural disturbances. Recent progress has been made using sim-to-real reinforcement learning (RL) to train such locomotion controllers, with approaches differing mainly in their reward functions. However, prior works lack a clear method to systematically test new reward functions and compare controller performance through repeatable experiments. This limits our understanding of the trade-offs between approaches and hinders progress. To address this, we propose a low-cost, quantitative benchmarking method to evaluate and compare the real-world performance of standing and walking (SaW) controllers on metrics like command following, disturbance recovery, and energy efficiency. We also revisit reward function design and construct a minimally constraining reward function to train SaW controllers. We experimentally verify that our benchmarking framework can identify areas for improvement, which can be systematically addressed to enhance the policies. We also compare our new controller to state-of-the-art controllers on the Digit humanoid robot. The results provide clear quantitative trade-offs among the controllers and suggest directions for future improvements to the reward functions and expansion of the benchmarks., Comment: 8 pages, 5 figs

We present the implementation and validation of an immersed boundary volume penalization method in the computational fluid dynamics solver CODA (from ONERA, DLR, and Airbus). Our goal is to model and simulate turbulent fluid flows in complex 3D aerodynamic configurations through the numerical solution of the Reynolds--averaged Navier--Stokes equations using the Spalart--Allmaras turbulent model. To do that, an immersed boundary method has been implemented in CODA and an efficient preprocessing tool for the construction of unstructured hexahedral meshes with adaptive mesh refinement around immersed geometries has been developed. We report several numerical examples, including subsonic and transonic flow past a NACA0012 airfoil, subsonic flow past an MDA30P30N multi-element airfoil, and subsonic flow around the NASA high-lift CRM aircraft. These simulations have been performed in the CODA solver with a second-order finite volume scheme as spatial discretization and an implicit backward Euler scheme based on the matrix-free GMRES block-Jacobi iterative method. The reported numerical simulations are in very good agreement with their corresponding experimental data. We conclude that the implemented immersed boundary method is efficient, flexible, and accurate and can therefore be used for aeronautical applications in industry., Comment: 22 pages, 15 figures

Terahertz (THz) communication systems hold immense potential for high-speed data transfer across various domains yet face challenges due to directionality constraints because of free space path loss. To address this, directional beams are commonly employed in THz technology. With the usage directional beams, it is important to track the transmitting device to keep the link connectivity. Leaky Parallel Plate Waveguides (LPPWs) were introduced to tackle this challenge. But traditional LPPW uses unique angle-frequency relationship which requires broad range of frequencies. This study proposes a novel approach to mitigate these constraints, balancing directionality with reduced directional gain to enhance LPPW adaptability. The proposed device can accurately determine the receiving angle of a beam by analyzing unique features extracted from the dual peak outputs. Experimentation and simulations reveal that the device allows for a broader angle of acceptance and calculation of the received angle.

The rapid advancement of machine learning techniques has led to their widespread application in various domains including water resources. However, snowmelt modeling remains an area that has not been extensively explored. In this study, we propose a state-of-the-art (SOTA) deep learning sequential model, leveraging the Temporal Convolutional Network (TCN), for snowmelt-driven discharge modeling in the Himalayan basin of the Hindu Kush Himalayan Region. To evaluate the performance of our proposed model, we conducted a comparative analysis with other popular models including Support Vector Regression (SVR), Long Short Term Memory (LSTM), and Transformer. Furthermore, Nested cross-validation (CV) is used with five outer folds and three inner folds, and hyper-parameter tuning is performed on the inner folds. To evaluate the performance of the model mean absolute error (MAE), root mean square error (RMSE), R square ($R^{2}$), Kling-Gupta Efficiency (KGE), and Nash-Sutcliffe Efficiency (NSE) are computed for each outer fold. The average metrics revealed that TCN outperformed the other models, with an average MAE of 0.011, RMSE of 0.023, $R^{2}$ of 0.991, KGE of 0.992, and NSE of 0.991. The findings of this study demonstrate the effectiveness of the deep learning model as compared to traditional machine learning approaches for snowmelt-driven streamflow forecasting. Moreover, the superior performance of TCN highlights its potential as a promising deep learning model for similar hydrological applications., Comment: 17 pages, 4 Tables, 7 figures

As generative AI progresses rapidly, new synthetic image generators continue to emerge at a swift pace. Traditional detection methods face two main challenges in adapting to these generators: the forensic traces of synthetic images from new techniques can vastly differ from those learned during training, and access to data for these new generators is often limited. To address these issues, we introduce the Ensemble of Expert Embedders (E3), a novel continual learning framework for updating synthetic image detectors. E3 enables the accurate detection of images from newly emerged generators using minimal training data. Our approach does this by first employing transfer learning to develop a suite of expert embedders, each specializing in the forensic traces of a specific generator. Then, all embeddings are jointly analyzed by an Expert Knowledge Fusion Network to produce accurate and reliable detection decisions. Our experiments demonstrate that E3 outperforms existing continual learning methods, including those developed specifically for synthetic image detection., Comment: 11 pages, 4 figures, To be published in CVPRWMF24

In the rapidly evolving field of artificial intelligence, the ability to harness and integrate knowledge across various domains stands as a paramount challenge and opportunity. This study introduces a novel approach to cross-domain knowledge discovery through the deployment of multi-AI agents, each specialized in distinct knowledge domains. These AI agents, designed to function as domain-specific experts, collaborate in a unified framework to synthesize and provide comprehensive insights that transcend the limitations of single-domain expertise. By facilitating seamless interaction among these agents, our platform aims to leverage the unique strengths and perspectives of each, thereby enhancing the process of knowledge discovery and decision-making. We present a comparative analysis of the different multi-agent workflow scenarios evaluating their performance in terms of efficiency, accuracy, and the breadth of knowledge integration. Through a series of experiments involving complex, interdisciplinary queries, our findings demonstrate the superior capability of domain specific multi-AI agent system in identifying and bridging knowledge gaps. This research not only underscores the significance of collaborative AI in driving innovation but also sets the stage for future advancements in AI-driven, cross-disciplinary research and application. Our methods were evaluated on a small pilot data and it showed a trend we expected, if we increase the amount of data we custom train the agents, the trend is expected to be more smooth.

Cavity quantum electrodynamics (cQED), the interaction of a two-level system with a high quality factor (Q) cavity, is a foundational building block in different architectures for quantum computation, communication, and metrology. The strong interaction between the atom and the cavity enables single photon operation which is required for quantum gates and sources. Cold atoms, quantum dots, and color centers in crystals are amongst the systems that have shown single photon operations, but they require significant physical infrastructure. Atomic vapors, on the other hand, require limited experimental infrastructure and are hence much easier to deploy outside a laboratory, but they produce an ensemble of moving atoms that results in short interaction times involving multiple atoms, which can hamper quantum operations. A solution to this issue can be found in nanophotonic cavities, where light-matter interaction is enhanced and the volume of operation is small, so that fast single-atom, single-photon operations are enabled. In this work, we study the interaction of an atomically-clad microring resonator (ACMRR) with different-sized ensembles of Rb atoms. We demonstrate strong coupling between an ensemble of ~50 atoms interacting with a high-quality factor (Q > 4 x 10^5) ACMRR, yielding a many-atom cooperativity C ~ 5.5. We continue to observe signatures of atom-photon interaction for a few (< 3) atoms, for which we observe saturation at the level of one intracavity photon. Further development of our platform, which includes integrated thermo-optic heaters to enable cavity tuning and stabilization, should enable the observation of interactions between single photons and single atoms.

Keywords play a crucial role in bridging the gap between human understanding and machine processing of textual data. They are essential to data enrichment because they form the basis for detailed annotations that provide a more insightful and in-depth view of the underlying data. Keyword/domain driven term extraction is a pivotal task in natural language processing, facilitating information retrieval, document summarization, and content categorization. This review focuses on keyword extraction methods, emphasizing the use of three major Large Language Models(LLMs): Llama2-7B, GPT-3.5, and Falcon-7B. We employed a custom Python package to interface with these LLMs, simplifying keyword extraction. Our study, utilizing the Inspec and PubMed datasets, evaluates the performance of these models. The Jaccard similarity index was used for assessment, yielding scores of 0.64 (Inspec) and 0.21 (PubMed) for GPT-3.5, 0.40 and 0.17 for Llama2-7B, and 0.23 and 0.12 for Falcon-7B. This paper underlines the role of prompt engineering in LLMs for better keyword extraction and discusses the impact of hallucination in LLMs on result evaluation. It also sheds light on the challenges in using LLMs for keyword extraction, including model complexity, resource demands, and optimization techniques.

Socially assistive robots (SARs) have shown great success in providing personalized cognitive-affective support for user populations with special needs such as older adults, children with autism spectrum disorder (ASD), and individuals with mental health challenges. The large body of work on SAR demonstrates its potential to provide at-home support that complements clinic-based interventions delivered by mental health professionals, making these interventions more effective and accessible. However, there are still several major technical challenges that hinder SAR-mediated interactions and interventions from reaching human-level social intelligence and efficacy. With the recent advances in large language models (LLMs), there is an increased potential for novel applications within the field of SAR that can significantly expand the current capabilities of SARs. However, incorporating LLMs introduces new risks and ethical concerns that have not yet been encountered, and must be carefully be addressed to safely deploy these more advanced systems. In this work, we aim to conduct a brief survey on the use of LLMs in SAR technologies, and discuss the potentials and risks of applying LLMs to the following three major technical challenges of SAR: 1) natural language dialog; 2) multimodal understanding; 3) LLMs as robot policies., Comment: 2 pages, accepted to the Proceedings of the AAAI Symposium Series, 2024

Existing text-to-image generative models reflect or even amplify societal biases ingrained in their training data. This is especially concerning for human image generation where models are biased against certain demographic groups. Existing attempts to rectify this issue are hindered by the inherent limitations of the pre-trained models and fail to substantially improve demographic diversity. In this work, we introduce Fair Retrieval Augmented Generation (FairRAG), a novel framework that conditions pre-trained generative models on reference images retrieved from an external image database to improve fairness in human generation. FairRAG enables conditioning through a lightweight linear module that projects reference images into the textual space. To enhance fairness, FairRAG applies simple-yet-effective debiasing strategies, providing images from diverse demographic groups during the generative process. Extensive experiments demonstrate that FairRAG outperforms existing methods in terms of demographic diversity, image-text alignment, and image fidelity while incurring minimal computational overhead during inference., Comment: CVPR 2024

This paper introduces the concept of a design tool for artistic performances based on attribute descriptions. To do so, we used a specific performance of falling actions. The platform integrates a novel machine-learning (ML) model with an interactive interface to generate and visualize artistic movements. Our approach's core is a cyclic Attribute-Conditioned Variational Autoencoder (AC-VAE) model developed to address the challenge of capturing and generating realistic 3D human body motions from motion capture (MoCap) data. We created a unique dataset focused on the dynamics of falling movements, characterized by a new ontology that divides motion into three distinct phases: Impact, Glitch, and Fall. The ML model's innovation lies in its ability to learn these phases separately. It is achieved by applying comprehensive data augmentation techniques and an initial pose loss function to generate natural and plausible motion. Our web-based interface provides an intuitive platform for artists to engage with this technology, offering fine-grained control over motion attributes and interactive visualization tools, including a 360-degree view and a dynamic timeline for playback manipulation. Our research paves the way for a future where technology amplifies the creative potential of human expression, making sophisticated motion generation accessible to a wider artistic community.

Poor generalization performance caused by distribution shifts in unseen domains often hinders the trustworthy deployment of deep neural networks. Many domain generalization techniques address this problem by adding a domain invariant regularization loss terms during training. However, there is a lack of modular software that allows users to combine the advantages of different methods with minimal effort for reproducibility. DomainLab is a modular Python package for training user specified neural networks with composable regularization loss terms. Its decoupled design allows the separation of neural networks from regularization loss construction. Hierarchical combinations of neural networks, different domain generalization methods, and associated hyperparameters, can all be specified together with other experimental setup in a single configuration file. Hierarchical combinations of neural networks, different domain generalization methods, and associated hyperparameters, can all be specified together with other experimental setup in a single configuration file. In addition, DomainLab offers powerful benchmarking functionality to evaluate the generalization performance of neural networks in out-of-distribution data. The package supports running the specified benchmark on an HPC cluster or on a standalone machine. The package is well tested with over 95 percent coverage and well documented. From the user perspective, it is closed to modification but open to extension. The package is under the MIT license, and its source code, tutorial and documentation can be found at https://github.com/marrlab/DomainLab.

Visual Inspection with Acetic Acid (VIA) remains the most feasible cervical cancer screening test in resource-constrained settings of low- and middle-income countries (LMICs), which are often performed screening camps or primary/community health centers by nurses instead of the preferred but unavailable expert Gynecologist. To address the highly subjective nature of the test, various handheld devices integrating cameras or smartphones have been recently explored to capture cervical images during VIA and aid decision-making via telemedicine or AI models. Most studies proposing AI models retrospectively use a relatively small number of already collected images from specific devices, digital cameras, or smartphones; the challenges and protocol for quality image acquisition during VIA in resource-constrained camp settings, challenges in getting gold standard, data imbalance, etc. are often overlooked. We present a novel approach and describe the end-to-end design process to build a robust smartphone-based AI-assisted system that does not require buying a separate integrated device: the proposed protocol for quality image acquisition in resource-constrained settings, dataset collected from 1,430 women during VIA performed by nurses in screening camps, preprocessing pipeline, and training and evaluation of a deep-learning-based classification model aimed to identify (pre)cancerous lesions. Our work shows that the readily available smartphones and a suitable protocol can capture the cervix images with the required details for the VIA test well; the deep-learning-based classification model provides promising results to assist nurses in VIA screening; and provides a direction for large-scale data collection and validation in resource-constrained settings.

With the limited availability of labeled data with various atmospheric conditions in remote sensing images, it seems useful to work with self-supervised algorithms. Few pretext-based algorithms, including from rotation, spatial context and jigsaw puzzles are not appropriate for satellite images. Often, satellite images have a higher temporal frequency. So, the temporal dimension of remote sensing data provides natural augmentation without requiring us to create artificial augmentation of images. Here, we propose S3-TSS, a novel method of self-supervised learning technique that leverages natural augmentation occurring in temporal dimension. We compare our results with current state-of-the-art methods and also perform various experiments. We observed that our method was able to perform better than baseline SeCo in four downstream datasets. Code for our work can be found here: https://github.com/hewanshrestha/Why-Self-Supervision-in-Time

Recent advancements in multimodal Human-Robot Interaction (HRI) datasets have highlighted the fusion of speech and gesture, expanding robots' capabilities to absorb explicit and implicit HRI insights. However, existing speech-gesture HRI datasets often focus on elementary tasks, like object pointing and pushing, revealing limitations in scaling to intricate domains and prioritizing human command data over robot behavior records. To bridge these gaps, we introduce NatSGD, a multimodal HRI dataset encompassing human commands through speech and gestures that are natural, synchronized with robot behavior demonstrations. NatSGD serves as a foundational resource at the intersection of machine learning and HRI research, and we demonstrate its effectiveness in training robots to understand tasks through multimodal human commands, emphasizing the significance of jointly considering speech and gestures. We have released our dataset, simulator, and code to facilitate future research in human-robot interaction system learning; access these resources at https://www.snehesh.com/natsgd/