Search

We present a method for teaching dexterous manipulation tasks to robots from human hand motion demonstrations. Unlike existing approaches that solely rely on kinematics information without taking into account the plausibility of robot and object interaction, our method directly infers plausible robot manipulation actions from human motion demonstrations. To address the embodiment gap between the human hand and the robot system, our approach learns a joint motion manifold that maps human hand movements, robot hand actions, and object movements in 3D, enabling us to infer one motion component from others. Our key idea is the generation of pseudo-supervision triplets, which pair human, object, and robot motion trajectories synthetically. Through real-world experiments with robot hand manipulation, we demonstrate that our data-driven retargeting method significantly outperforms conventional retargeting techniques, effectively bridging the embodiment gap between human and robotic hands. Website at https://rureadyo.github.io/MocapRobot/., Comment: Preprint. Under Review

Group fairness requires that different protected groups, characterized by a given sensitive attribute, receive equal outcomes overall. Typically, the level of group fairness is measured by the statistical gap between predictions from different protected groups. In this study, we reveal an implicit property of existing group fairness measures, which provides an insight into how the group-fair models behave. Then, we develop a new group-fair constraint based on this implicit property to learn group-fair models. To do so, we first introduce a notable theoretical observation: every group-fair model has an implicitly corresponding transport map between the input spaces of each protected group. Based on this observation, we introduce a new group fairness measure termed Matched Demographic Parity (MDP), which quantifies the averaged gap between predictions of two individuals (from different protected groups) matched by a given transport map. Then, we prove that any transport map can be used in MDP to learn group-fair models, and develop a novel algorithm called Fairness Through Matching (FTM), which learns a group-fair model using MDP constraint with an user-specified transport map. We specifically propose two favorable types of transport maps for MDP, based on the optimal transport theory, and discuss their advantages. Experiments reveal that FTM successfully trains group-fair models with certain desirable properties by choosing the transport map accordingly., Comment: Published in TMLR

Chirality, an intrinsic preference for a specific handedness, is a fundamental characteristic observed in nature. In magnetism, magnetic chirality arises from the anti-symmetric Dzyaloshinskii-Moriya interaction in competition with the symmetric Heisenberg exchange interaction. Traditionally, the anti-symmetric interaction has been considered minor relative to the symmetric interaction. In this study, we demonstrate an observation of giant magnetic chirality during the dimensionality crossover of magnetic materials from three-dimensional to two-dimensional. The ratio between the anti-symmetric and symmetric interactions exhibits a reversal in their dominance over this crossover, overturning the traditional consideration. This observation is validated theoretically using a non-local interaction model and tight-binding calculation with distinct pairing schemes for each exchange interaction throughout the crossover. Additional experiments investigating the asphericity of orbital moments corroborate the robustness of our findings. Our findings highlight the critical role of dimensionality in shaping magnetic chirality and offer strategies for engineering chiral magnet states with unprecedented strength, desired for the design of spintronic materials., Comment: 21 pages, 4 figures

Text augmentation (TA) is a critical technique for text classification, especially in few-shot settings. This paper introduces a novel LLM-based TA method, TARDiS, to address challenges inherent in the generation and alignment stages of two-stage TA methods. For the generation stage, we propose two generation processes, SEG and CEG, incorporating multiple class-specific prompts to enhance diversity and separability. For the alignment stage, we introduce a class adaptation (CA) method to ensure that generated examples align with their target classes through verification and modification. Experimental results demonstrate TARDiS's effectiveness, outperforming state-of-the-art LLM-based TA methods in various few-shot text classification tasks. An in-depth analysis confirms the detailed behaviors at each stage., Comment: 10 pages

Scientific figure captioning is a complex task that requires generating contextually appropriate descriptions of visual content. However, existing methods often fall short by utilizing incomplete information, treating the task solely as either an image-to-text or text summarization problem. This limitation hinders the generation of high-quality captions that fully capture the necessary details. Moreover, existing data sourced from arXiv papers contain low-quality captions, posing significant challenges for training large language models (LLMs). In this paper, we introduce a framework called Multi-LLM Collaborative Figure Caption Generation (MLBCAP) to address these challenges by leveraging specialized LLMs for distinct sub-tasks. Our approach unfolds in three key modules: (Quality Assessment) We utilize multimodal LLMs to assess the quality of training data, enabling the filtration of low-quality captions. (Diverse Caption Generation) We then employ a strategy of fine-tuning/prompting multiple LLMs on the captioning task to generate candidate captions. (Judgment) Lastly, we prompt a prominent LLM to select the highest quality caption from the candidates, followed by refining any remaining inaccuracies. Human evaluations demonstrate that informative captions produced by our approach rank better than human-written captions, highlighting its effectiveness. Our code is available at https://github.com/teamreboott/MLBCAP, Comment: Accepted to AAAI 2025 AI4Research Workshop

Although read disturbance has emerged as a major reliability concern, managing read disturbance in modern NAND flash memory has not been thoroughly investigated yet. From a device characterization study using real modern NAND flash memory, we observe that reading a page incurs heterogeneous reliability impacts on each WL, which makes the existing block-level read reclaim extremely inefficient. We propose a new WL-level read-reclaim technique, called STRAW, which keeps track of the accumulated read-disturbance effect on each WL and reclaims only heavily-disturbed WLs. By avoiding unnecessary read-reclaim operations, STRAW reduces read-reclaim-induced page writes by 83.6\% with negligible storage overhead., Comment: Accepted for publication at IEEE Computer Architecture Letters (IEEE CAL), 2024

The goal of video moment retrieval and highlight detection is to identify specific segments and highlights based on a given text query. With the rapid growth of video content and the overlap between these tasks, recent works have addressed both simultaneously. However, they still struggle to fully capture the overall video context, making it challenging to determine which words are most relevant. In this paper, we present a novel Video Context-aware Keyword Attention module that overcomes this limitation by capturing keyword variation within the context of the entire video. To achieve this, we introduce a video context clustering module that provides concise representations of the overall video context, thereby enhancing the understanding of keyword dynamics. Furthermore, we propose a keyword weight detection module with keyword-aware contrastive learning that incorporates keyword information to enhance fine-grained alignment between visual and textual features. Extensive experiments on the QVHighlights, TVSum, and Charades-STA benchmarks demonstrate that our proposed method significantly improves performance in moment retrieval and highlight detection tasks compared to existing approaches. Our code is available at: https://github.com/VisualAIKHU/Keyword-DETR, Comment: Accepted at AAAI 2025

We examined the anomalies in the light curves of the lensing events MOA-2022-BLG-033, KMT-2023-BLG-0119, and KMT-2023-BLG-1896. We conducted detailed modeling of the light curves to uncover the nature of the anomalies. This modeling revealed that all signals originated from planetary companions to the primary lens. The planet-to-host mass ratios are very low: $q\sim 7.5\times 10^{-5}$ for MOA-2022-BLG-033, $q\sim 3.6\times 10^{-4}$ for KMT-2023-BLG-0119, and $q\sim 6.9\times 10^{-5}$ for KMT-2023-BLG-1896. The anomalies occurred as the source passed through the negative deviation region behind the central caustic along the planet-host axis. The solutions are subject to a common inner-outer degeneracy, resulting in variations in estimating the projected planet-host separation. For KMT-2023-BLG-1896, although the planetary scenario provides the best explanation of the anomaly, the binary companion scenario is marginally possible. We estimate the physical parameters of the planetary systems through Bayesian analyses based on the lensing observables. The analysis identifies MOA-2022-BLG-033L as a planetary system with an ice giant, approximately 12 times the mass of Earth, orbiting an early M dwarf star. The companion of KMT-2023-BLG-1896L is also an ice giant, with a mass around 16 Earth masses, orbiting a mid-K-type main-sequence star. The companion of KMT-2023-BLG-0119L, which has a mass about the mass of Saturn, orbits a mid-K-type dwarf star. The lens for MOA-2022-BLG-033 is highly likely to be located in the disk, whereas for the other events, the probabilities of the lens being in the disk or the bulge are roughly comparable., Comment: 9 pages, 10 figures, 6 tables

Bacterial heterogeneity is pivotal for adaptation to diverse environments, posing significant challenges in microbial diagnostics and therapeutic interventions. Recent advancements in high-resolution optical microscopy have revolutionized our ability to observe and characterize individual bacteria, offering unprecedented insights into their metabolic states and behaviors at the single-cell level. This review discusses the transformative impact of various high-resolution imaging techniques, including fluorescence and label-free imaging, which have enhanced our understanding of bacterial pathophysiology. These methods provide detailed visualizations that are crucial for developing targeted treatments and improving clinical diagnostics. We highlight the integration of these imaging techniques with computational tools, which has facilitated rapid, accurate pathogen identification and real-time monitoring of bacterial responses to treatments. The ongoing development of these optical imaging technologies promises to significantly advance our understanding of microbiology and to catalyze the translation of these insights into practical healthcare solutions.

Continuous gravitational waves (CWs) emission from neutron stars carries information about their internal structure and equation of state, and it can provide tests of General Relativity. We present a search for CWs from a set of 45 known pulsars in the first part of the fourth LIGO--Virgo--KAGRA observing run, known as O4a. We conducted a targeted search for each pulsar using three independent analysis methods considering the single-harmonic and the dual-harmonic emission models. We find no evidence of a CW signal in O4a data for both models and set upper limits on the signal amplitude and on the ellipticity, which quantifies the asymmetry in the neutron star mass distribution. For the single-harmonic emission model, 29 targets have the upper limit on the amplitude below the theoretical spin-down limit. The lowest upper limit on the amplitude is $6.4\!\times\!10^{-27}$ for the young energetic pulsar J0537-6910, while the lowest constraint on the ellipticity is $8.8\!\times\!10^{-9}$ for the bright nearby millisecond pulsar J0437-4715. Additionally, for a subset of 16 targets we performed a narrowband search that is more robust regarding the emission model, with no evidence of a signal. We also found no evidence of non-standard polarizations as predicted by the Brans-Dicke theory., Comment: main paper: 12 pages, 6 figures, 4 tables

The goal of voice conversion is to transform the speech of a source speaker to sound like that of a reference speaker while preserving the original content. A key challenge is to extract disentangled linguistic content from the source and voice style from the reference. While existing approaches leverage various methods to isolate the two, a generalization still requires further attention, especially for robustness in zero-shot scenarios. In this paper, we achieve successful disentanglement of content and speaker features by tuning self-supervised speech features with adapters. The adapters are trained to dynamically encode nuanced features from rich self-supervised features, and the decoder fuses them to produce speech that accurately resembles the reference with minimal loss of content. Moreover, we leverage a conditional flow matching decoder with cross-attention speaker conditioning to further boost the synthesis quality and efficiency. Subjective and objective evaluations in a zero-shot scenario demonstrate that the proposed method outperforms existing models in speech quality and similarity to the reference speech., Comment: ICASSP 2025; demo available https://mm.kaist.ac.kr/projects/AdaptVC

Layers have become indispensable tools for professional artists, allowing them to build a hierarchical structure that enables independent control over individual visual elements. In this paper, we propose LayeringDiff, a novel pipeline for the synthesis of layered images, which begins by generating a composite image using an off-the-shelf image generative model, followed by disassembling the image into its constituent foreground and background layers. By extracting layers from a composite image, rather than generating them from scratch, LayeringDiff bypasses the need for large-scale training to develop generative capabilities for individual layers. Furthermore, by utilizing a pretrained off-the-shelf generative model, our method can produce diverse contents and object scales in synthesized layers. For effective layer decomposition, we adapt a large-scale pretrained generative prior to estimate foreground and background layers. We also propose high-frequency alignment modules to refine the fine-details of the estimated layers. Our comprehensive experiments demonstrate that our approach effectively synthesizes layered images and supports various practical applications.

We propose SoundBrush, a model that uses sound as a brush to edit and manipulate visual scenes. We extend the generative capabilities of the Latent Diffusion Model (LDM) to incorporate audio information for editing visual scenes. Inspired by existing image-editing works, we frame this task as a supervised learning problem and leverage various off-the-shelf models to construct a sound-paired visual scene dataset for training. This richly generated dataset enables SoundBrush to learn to map audio features into the textual space of the LDM, allowing for visual scene editing guided by diverse in-the-wild sound. Unlike existing methods, SoundBrush can accurately manipulate the overall scenery or even insert sounding objects to best match the audio inputs while preserving the original content. Furthermore, by integrating with novel view synthesis techniques, our framework can be extended to edit 3D scenes, facilitating sound-driven 3D scene manipulation. Demos are available at https://soundbrush.github.io/., Comment: AAAI 2025

Conventional mode switching mechanisms, which rely on dynamically encircling exceptional points (EPs) through non-adiabatic transitions (NATs), suffer from intrinsic nonlinear dynamics that hinder precise control and reproducibility in experimental settings. Additionally, these methods exhibit low transmission efficiencies due to path-dependent attenuation, limiting their effectiveness in optical switching and logic gate applications. To overcome these limitations, we propose a novel mode switching approach that leverages a pair of EPs configuration in an optical microcavity, characterized by superradiance and subradiance. This mechanism exploits the topological structure of the Riemann surface to enable robust mode switching control and tunable Q-factor through purely adiabatic encircling. Furthermore, topological protection validated via braid isotopy ensures robustness against noise and parametric perturbations, facilitating a compact, robust, and adaptive non-Hermitian system., Comment: 12 pages, 5 figures; This study fully improved previous version of arXiv:2208.06860v2

This paper proposes a Quantum-MUSIC, the first multiple signal classification (MUSIC) algorithm for quantum wireless sensing of multi-user. Since an atomic receiver for quantum wireless sensing can only measure the magnitude of a received signal, sensing performance degradation of traditional antenna-based signal processing algorithms is inevitable. To overcome this limitation, the proposed algorithm recovers the channel information and incorporates the traditional MUSIC algorithm, enabling the sensing of multi-user with magnitude-only measurement. Simulation results showed that the proposed algorithm outperforms the existing MUSIC algorithm, validating the superior potential of quantum wireless sensing., Comment: 5 pages, 6 figures

We report recent progress on $\varepsilon_K$ evaluated directly from the standard model (SM) with lattice QCD inputs such as $\hat{B}_K$, exclusive $|V_{cb}|$, $|V_{us}|$, $|V_{ud}|$, $\xi_0$, $\xi_2$, $\xi_\text{LD}$, $f_K$, and $m_c$. We find that the standard model with exclusive $|V_{cb}|$ and lattice QCD inputs describes only $2/3 \cong 65\%$ of the experimental value of $|\varepsilon_K|$ and does not explain its remaining 35\%, which represents a strong tension in $|\varepsilon_K|$ at the $5.1\sigma \sim 4.1\sigma$ level between the SM theory and experiment. We also find that this tension disappears when we use the inclusive value of $|V_{cb}|$ obtained using the heavy quark expansion based on the QCD sum rule approach. We also report results for $|\varepsilon_K|$ obtained using the Brod-Gorbahn-Stamou (BGS) method for $\eta_i$ of $u-t$ unitarity, which leads to even a stronger tension of $5.7\sigma \sim 4.2\sigma$ with lattice QCD inputs., Comment: 11 pages, 4 figures, Lattice 2024 proceeding. arXiv admin note: substantial text overlap with arXiv:2312.02986, arXiv:2301.12375

With the rise of AI, NVIDIA GPUs have become the de facto standard for AI system design. This paper presents a comprehensive evaluation of Intel Gaudi NPUs as an alternative to NVIDIA GPUs for AI model serving. First, we create a suite of microbenchmarks to compare Intel Gaudi-2 with NVIDIA A100, showing that Gaudi-2 achieves competitive performance not only in primitive AI compute, memory, and communication operations but also in executing several important AI workloads end-to-end. We then assess Gaudi NPU's programmability by discussing several software-level optimization strategies to employ for implementing critical FBGEMM operators and vLLM, evaluating their efficiency against GPU-optimized counterparts. Results indicate that Gaudi-2 achieves energy efficiency comparable to A100, though there are notable areas for improvement in terms of software maturity. Overall, we conclude that, with effective integration into high-level AI frameworks, Gaudi NPUs could challenge NVIDIA GPU's dominance in the AI server market, though further improvements are necessary to fully compete with NVIDIA's robust software ecosystem., Comment: Under Review

The objective of polarization rendering is to simulate the interaction of light with materials exhibiting polarization-dependent behavior. However, integrating polarization into rendering is challenging and increases computational costs significantly. The primary difficulty lies in efficiently modeling and computing the complex reflection phenomena associated with polarized light. Specifically, frequency-domain analysis, essential for efficient environment lighting and storage of complex light interactions, is lacking. To efficiently simulate and reproduce polarized light interactions using frequency-domain techniques, we address the challenge of maintaining continuity in polarized light transport represented by Stokes vectors within angular domains. The conventional spherical harmonics method cannot effectively handle continuity and rotation invariance for Stokes vectors. To overcome this, we develop a new method called polarized spherical harmonics (PSH) based on the spin-weighted spherical harmonics theory. Our method provides a rotation-invariant representation of Stokes vector fields. Furthermore, we introduce frequency domain formulations of polarized rendering equations and spherical convolution based on PSH. We first define spherical convolution on Stokes vector fields in the angular domain, and it also provides efficient computation of polarized light transport, nearly on an entry-wise product in the frequency domain. Our frequency domain formulation, including spherical convolution, led to the development of the first real-time polarization rendering technique under polarized environmental illumination, named precomputed polarized radiance transfer, using our polarized spherical harmonics. Results demonstrate that our method can effectively and accurately simulate and reproduce polarized light interactions in complex reflection phenomena.

Acquisition and modeling of polarized light reflection and scattering help reveal the shape, structure, and physical characteristics of an object, which is increasingly important in computer graphics. However, current polarimetric acquisition systems are limited to static and opaque objects. Human faces, on the other hand, present a particularly difficult challenge, given their complex structure and reflectance properties, the strong presence of spatially-varying subsurface scattering, and their dynamic nature. We present a new polarimetric acquisition method for dynamic human faces, which focuses on capturing spatially varying appearance and precise geometry, across a wide spectrum of skin tones and facial expressions. It includes both single and heterogeneous subsurface scattering, index of refraction, and specular roughness and intensity, among other parameters, while revealing biophysically-based components such as inner- and outer-layer hemoglobin, eumelanin and pheomelanin. Our method leverages such components' unique multispectral absorption profiles to quantify their concentrations, which in turn inform our model about the complex interactions occurring within the skin layers. To our knowledge, our work is the first to simultaneously acquire polarimetric and spectral reflectance information alongside biophysically-based skin parameters and geometry of dynamic human faces. Moreover, our polarimetric skin model integrates seamlessly into various rendering pipelines.

Using the Event Horizon Telescope (EHT), the gravitationally lensed rings around the supermassive black holes (SMBHs) in Messier 87 (M87) and Sagittarius A* (Sgr A*) have now been successfully imaged at a resolution under 10 gravitational radii (R$_{\rm g}$ $ = \rm{GM/c^2}$). To expand studies beyond M87 and Sgr A*, we have constructed the Event Horizon and Environs (ETHER) sample, a comprehensive database encompassing approximately 3.15 million SMBH mass estimates, $\sim$ 20,000 Very-Long Baseline Interferometry (VLBI) radio flux densities, and $\sim$ 36,000 hard X-ray flux densities. This database is designed to identify and optimize target selection for the EHT and its upgrades on the ground and in space. We have identified a Gold Sample (GS) of nearby low-luminosity Active Galactic Nuclei (AGNs) within it that are ideal for studying jet bases and potentially imaging black hole shadows. We observed 27 of these AGNs using the EHT from 2022 to 2024, providing an opportunity to resolve and image accretion flows and jets at resolutions of $\leq$ 100 R$_{\rm g}$. Only a few SMBHs have sufficiently high enough flux density to be imaged at scales of $\leq$ 50 R$_{\rm g}$ with the present EHT. Among these are M87, Sgr A*, NGC4594 (Sombrero/M104), NGC4261, and NGC4374 (Messier 84/M84). Of these, NGC4261, Sombrero, and M84 have been observed and/or are scheduled for deep imaging with EHT+ALMA from 2023 to 2025. Sombrero, NGC4261, M84, NGC4278, and NGC5232 are clearly detected in our EHT+ALMA observations in 2022, indicating that the 230 GHz flux density from the accretion flows is significantly high. Ongoing imaging of the ETHER GS will enable measurements of black hole mass and spin, help constrain General Relativity, and enrich our understanding of jet launching and accretion inflows across a broad multi-parameter space, including black hole mass, spin, accretion rate, and orientation., Comment: 9 pages, 6 figures, 1 table, published in Proceedings of the 16th EVN Symposium, Ed. E. Ros, P. Benke, S.A. Dzib, I. Rottmann, & J.A. Zensus, Bonn: Max-Planck-Institut f\"ur Radioastronomie, 2024, pages 75-84, https://cloud.mpifr-bonn.mpg.de/index.php/s/BkX2CC2Xjn2aKR4

The expansion of large language models (LLMs) with hundreds of billions of parameters presents significant challenges to computational resources, particularly data movement and memory bandwidth. Long-context LLMs, which process sequences of tens of thousands of tokens, further increase the demand on the memory system as the complexity in attention layers and key-value cache sizes is proportional to the context length. Processing-in-Memory (PIM) maximizes memory bandwidth by moving compute to the data and can address the memory bandwidth challenges; however, PIM is not necessarily scalable to accelerate long-context LLM because of limited per-module memory capacity and the inflexibility of fixed-functional unit PIM architecture and static memory management. In this work, we propose LoL-PIM which is a multi-node PIM architecture that accelerates long context LLM through hardware-software co-design. In particular, we propose how pipeline parallelism can be exploited across a multi-PIM module while a direct PIM access (DPA) controller (or DMA for PIM) is proposed that enables dynamic PIM memory management and results in efficient PIM utilization across a diverse range of context length. We developed an MLIR-based compiler for LoL-PIM extending a commercial PIM-based compiler where the software modifications were implemented and evaluated, while the hardware changes were modeled in the simulator. Our evaluations demonstrate that LoL-PIM significantly improves throughput and reduces latency for long-context LLM inference, outperforming both multi-GPU and GPU-PIM systems (up to 8.54x and 16.0x speedup, respectively), thereby enabling more efficient deployment of LLMs in real-world applications., Comment: 15 pages, 12 figures

The goal of this work is to generate natural speech in multiple languages while maintaining the same speaker identity, a task known as cross-lingual speech synthesis. A key challenge of cross-lingual speech synthesis is the language-speaker entanglement problem, which causes the quality of cross-lingual systems to lag behind that of intra-lingual systems. In this paper, we propose CrossSpeech++, which effectively disentangles language and speaker information and significantly improves the quality of cross-lingual speech synthesis. To this end, we break the complex speech generation pipeline into two simple components: language-dependent and speaker-dependent generators. The language-dependent generator produces linguistic variations that are not biased by specific speaker attributes. The speaker-dependent generator models acoustic variations that characterize speaker identity. By handling each type of information in separate modules, our method can effectively disentangle language and speaker representation. We conduct extensive experiments using various metrics, and demonstrate that CrossSpeech++ achieves significant improvements in cross-lingual speech synthesis, outperforming existing methods by a large margin.

We present a measurement of the branching fraction and fraction of longitudinal polarization of $B^0 \to \rho^+ \rho^-$ decays, which have two $\pi^0$'s in the final state. We also measure time-dependent $CP$ violation parameters for decays into longitudinally polarized $\rho^+ \rho^-$ pairs. This analysis is based on a data sample containing $(387\pm6) \times 10^6$ \BBbar pairs collected with the Belle~II detector at the SuperKEKB asymmetric-energy $e^+e^-$ collider in 2019-2022. We obtain ${B}(B^0\to\rho^+\rho^-) = (2.88 ^{+0.23}_{-0.22} {}^{+0.29}_{-0.27}) \times 10^{-5}, f_{L} = 0.921 ^{+0.024}_{-0.025} {}^{+0.017}_{-0.015}$, $S = -0.26\pm0.19\pm0.08$, and $C = -0.02\pm0.12^{+0.06}_{-0.05}$, where the first uncertainties are statistical and the second are systematic. We use these results to perform an isospin analysis to constrain the CKM angle $\phi_2$ and obtain two solutions; the result consistent with other Standard Model constraints is $\phi_2 = (92.6^{+4.5}_{-4.8})^\circ$.

Recent lightweight image captioning models using retrieved data mainly focus on text prompts. However, previous works only utilize the retrieved text as text prompts, and the visual information relies only on the CLIP visual embedding. Because of this issue, there is a limitation that the image descriptions inherent in the prompt are not sufficiently reflected in the visual embedding space. To tackle this issue, we propose ViPCap, a novel retrieval text-based visual prompt for lightweight image captioning. ViPCap leverages the retrieved text with image information as visual prompts to enhance the ability of the model to capture relevant visual information. By mapping text prompts into the CLIP space and generating multiple randomized Gaussian distributions, our method leverages sampling to explore randomly augmented distributions and effectively retrieves the semantic features that contain image information. These retrieved features are integrated into the image and designated as the visual prompt, leading to performance improvements on the datasets such as COCO, Flickr30k, and NoCaps. Experimental results demonstrate that ViPCap significantly outperforms prior lightweight captioning models in efficiency and effectiveness, demonstrating the potential for a plug-and-play solution. The source code is available at https://github.com/taewhankim/VIPCAP., Comment: Accepted to AAAI 2025

Understanding hydrogen-grain boundary (GB) interactions is critical to the analysis of hydrogen embrittlement in metals. This work presents a mesoscale fully kinetic model to investigate the effect of GB misorientation on hydrogen diffusion and trapping using phase-field based representative volume elements (RVEs). The flux equation consists of three terms: a diffusive term and two terms for high and low angle grain boundary (H/LAGB) trapping. Uptake simulations showed that decreasing the grain size resulted in higher hydrogen content due to increasing the GB density. Permeation simulations showed that GBs are high flux paths due to their higher enrichment with hydrogen. Since HAGBs have higher enrichment than LAGBs, due to their higher trap-binding energy, they generally have the highest hydrogen flux. Nevertheless, the flux shows a convoluted behavior as it depends on the local concentration, alignment of GB with external concentration gradient as well as the GB network connectivity. Finally, decreasing the grain size resulted in a larger break-through time and a larger steady-state exit flux.

Recent works on dynamic neural field reconstruction assume input from synchronized multi-view videos with known poses. These input constraints are often unmet in real-world setups, making the approach impractical. We demonstrate that unsynchronized videos with unknown poses can generate dynamic neural fields if the videos capture human motion. Humans are one of the most common dynamic subjects whose poses can be estimated using state-of-the-art methods. While noisy, the estimated human shape and pose parameters provide a decent initialization for the highly non-convex and under-constrained problem of training a consistent dynamic neural representation. Given the sequences of pose and shape of humans, we estimate the time offsets between videos, followed by camera pose estimations by analyzing 3D joint locations. Then, we train dynamic NeRF employing multiresolution rids while simultaneously refining both time offsets and camera poses. The setup still involves optimizing many parameters, therefore, we introduce a robust progressive learning strategy to stabilize the process. Experiments show that our approach achieves accurate spatiotemporal calibration and high-quality scene reconstruction in challenging conditions.

Hydrogen grain boundary (GB) trapping is widely accepted as the main cause for hydrogen induced intergranular failure. Several studies were conducted to unveil the role of GBs on hydrogen transport; however, a clear understanding is yet to be attained. This is due to the limitations of the state-of-the-art experimental procedures for such highly kinetic processes. In this study, we aim at providing a deeper understanding of hydrogen-GB interactions using full-field representative volume element (RVE). The phase-field method is chosen for generating RVEs, since it is the an appropriate numerical tool to represent GBs. A novel fully-kinetic formulation for hydrogen diffusion and GB trapping is presented, which is compatible with the phase-field based RVEs. GB diffusivity ($D_\mathrm{gb}$) and trap-binding energy ($E_\mathrm{gb}$) were used as parameters to understand the interactions between diffusion and GB trapping. Uptake and permeation simulations were performed with constant and gradient occupancy boundary conditions respectively. In both cases, increasing $E_\mathrm{gb}$, increased the hydrogen GB occupancy. The permeation simulations showed that the hydrogen flux along the GBs increased with increasing both, $D_\mathrm{gb}$ and, surprisingly, $E_\mathrm{gb}$. Since trapping increases the hydrogen occupancy along GBs, it also increases the occupancy gradients, resulting in a higher flux. This led to the conclusion that, in the case of an external occupancy gradient, GB trapping and diffusion cooperate, rather than compete, to increase the hydrogen flux. On the other hand, the decisive factor for the retention of hydrogen at the GBs in permeation simulations was $D_\mathrm{gb}$ rather than $E_\mathrm{gb}$.

We report an updated neutrino mixing angle of $\theta_{13}$ obtained from a complete data sample of the RENO experiment. The experiment has measured the amplitude and frequency of reactor anti-electron-neutrinos ($\bar{\nu}_{e}$) oscillations at the Hanbit nuclear power plant, Younggwang, Korea, since August 2011. As of March 2023, the data acquisition was completed after a total of 3800 live days of detector operation. The observed candidates via inverse beta decay (IBD) are 1,211,995 (144,667) in the near (far) detector. Based on an observed energy-dependent reactor neutrino disappearance, neutrino oscillation parameters of $\theta_{13}$ and $\lvert\Delta m_{ee}^2\rvert$ are precisely determined as $\sin^{2}2\theta_{13}=0.0920_{-0.0042}^{+0.0044}(\text{stat.})_{-0.0041}^{+0.0041}(\text{syst.})$ and $\lvert\Delta m_{ee}^2\rvert=\left[2.57_{-0.11}^{+0.10}(\text{stat.})_{-0.05}^{+0.05}(\text{syst.})\right]\times10^{-3}~\text{eV}^{2}$. Compared to the previous RENO results published in Ref.~\cite{PhysRevLett.121.201801}, the precision is improved from 7.5\% to 6.4\% for $\sin^{2}2\theta_{13}$ and from 5.2\% to 4.5\% for $\lvert\Delta m_{ee}^2\rvert$. The statistical error of the measurement has reached our goal and is hardly improved with additional data-taking., Comment: 13 pages, 11 figures

JSNS$^2$ (J-PARC Sterile Neutrino Search at J-PARC Spallation Neutron Source) is an experiment searching for sterile neutrinos through the observation of $\bar{\nu}_{\mu} \rightarrow \bar{\nu}_e$ appearance oscillations, using neutrinos produced by muon decay-at-rest. A key aspect of the experiment involves accurately understanding the neutrino flux and the quantities of pions and muons, which are progenitors of (anti-)neutrinos, given that their production rates have yet to be measured. We present the first electron-neutrino flux measurement using $^{12}\mathrm{C}(\nu_{e},e^{-}) ^{12}\mathrm{N}_{g.s.}$ reaction in JSNS$^2$, yielding a flux of (6.7 $\pm$ 1.6 (stat.) $\pm$ 1.7 (syst.)) $\times$ 10$^{-9}$ cm$^{-2}$ proton$^{-1}$ at the JSNS$^2$ detector location, located at 24 meters distance from the mercury target. This flux measurement is consistent with predictions from simulations based on hadron models.

Let $X$ be the Fano threefold of index one, degree $22$, and $\mathrm{Pic}(X)\cong\mathbb{Z}$. Such a threefold $X$ can be realized by a regular zero section $\mathbf{s}$ of $(\bigwedge^2\mathcal{F}^{*})^{\oplus 3}$ over Grassmannian variety $\mathrm{Gr}(3,V)$, $\dim V=7$ with the universal subbundle $\mathcal{F}$. When the section $\mathbf{s}$ is given by the net of the $\mathrm{SL}_2$-invariant skew forms, we call it by the Mukai-Umemura (MU) variety. In this paper, we prove that the Hilbert scheme of rational quartic curves in the MU-variety is smooth and compute its Poincar\'e polynomial by applying the Bia{\l}ynicki-Birula's theorem., Comment: 23 pages, 2 figures

As part of the BISTRO survey, we present JCMT 850 $\mu$m polarimetric observations towards the Orion Integral-Shaped Filament (ISF) that covers three portions known as OMC-1, OMC-2, and OMC-3. The magnetic field threading the ISF seen in the JCMT POL-2 map appears as a tale of three: pinched for OMC-1, twisted for OMC-2, and nearly uniform for OMC-3. A multi-scale analysis shows that the magnetic field structure in OMC-3 is very consistent at all the scales, whereas the field structure in OMC-2 shows no correlation across different scales. In OMC-1, the field retains its mean orientation from large to small scales, but shows some deviations at small scales. Histograms of relative orientations between the magnetic field and filaments reveal a bimodal distribution for OMC-1, a relatively random distribution for OMC-2, and a distribution with a predominant peak at 90$^\circ$ for OMC-3. Furthermore, the magnetic fields in OMC-1 and OMC-3 both appear to be aligned perpendicular to the fibers, which are denser structures within the filament, but the field in OMC-2 is aligned along with the fibers. All these suggest that gravity, turbulence, and magnetic field are each playing a leading role in OMC-1, 2, and 3, respectively. While OMC-2 and 3 have almost the same gas mass, density, and non-thermal velocity dispersion, there are on average younger and fewer young stellar objects in OMC-3, providing evidence that a stronger magnetic field will induce slower and less efficient star formation in molecular clouds., Comment: published in the ApJ Letters

Variational quantum algorithms (VQAs) have attracted remarkable interest over the past few years because of their potential computational advantages on near-term quantum devices. They leverage a hybrid approach that integrates classical and quantum computing resources to solve high-dimensional problems that are challenging for classical approaches alone. In the training process of variational circuits, constructing an accurate probability distribution for each epoch is not always necessary, creating opportunities to reduce computational costs through shot reduction. However, existing shot-allocation methods that capitalize on this potential often lack adaptive feedback or are tied to specific classical optimizers, which limits their applicability to common VQAs and broader optimization techniques. Our observations indicate that the information entropy of a quantum circuit's output distribution exhibits an approximately exponential relationship with the number of shots needed to achieve a target Hellinger distance. In this work, we propose a distribution-adaptive dynamic shot (DDS) framework that efficiently adjusts the number of shots per iteration in VQAs using the entropy distribution from the prior training epoch. Our results demonstrate that the DDS framework sustains inference accuracy while achieving a ~50% reduction in average shot count compared to fixed-shot training, and ~60% higher accuracy than recently proposed tiered shot allocation methods. Furthermore, in noisy simulations that reflect the error rates of actual IBM quantum systems, DDS achieves approximately a ~30% reduction in the total number of shots compared to the fixed-shot method with minimal degradation in accuracy, and offers about ~70% higher computational accuracy than tiered shot allocation methods., Comment: 18 pages, 24 figures

In this study, we develop an artificial neural network to estimate the infrared (IR) luminosity and star formation rates (SFR) of galaxies. Our network is trained using 'true' IR luminosity values derived from modeling the IR spectral energy distributions (SEDs) of FIR-detected galaxies. We explore five different sets of input features, each incorporating optical, mid-infrared (MIR), near-infrared (NIR), ultraviolet (UV), and emission line data, along with spectroscopic redshifts and uncertainties. All feature sets yield similar IR luminosity predictions, but including all photometric data leads to slightly improved performance. This suggests that comprehensive photometric information enhances the accuracy of our predictions. Our network is applied to a sample of SDSS galaxies defined as unseen data, and the results are compared with three published catalogs of SFRs. Overall, our network demonstrates excellent performance for star-forming galaxies while we observe discrepancies in composite and AGN samples. These inconsistencies may stem from uncertainties inherent in the compared catalogs or potential limitations in the performance of our network., Comment: 13 pages, 7 figures, and 2 tables. Accepted for publication in ApJ

Learning a perception and reasoning module for robotic assistants to plan steps to perform complex tasks based on natural language instructions often requires large free-form language annotations, especially for short high-level instructions. To reduce the cost of annotation, large language models (LLMs) are used as a planner with few data. However, when elaborating the steps, even the state-of-the-art planner that uses LLMs mostly relies on linguistic common sense, often neglecting the status of the environment at command reception, resulting in inappropriate plans. To generate plans grounded in the environment, we propose FLARE (Few-shot Language with environmental Adaptive Replanning Embodied agent), which improves task planning using both language command and environmental perception. As language instructions often contain ambiguities or incorrect expressions, we additionally propose to correct the mistakes using visual cues from the agent. The proposed scheme allows us to use a few language pairs thanks to the visual cues and outperforms state-of-the-art approaches. Our code is available at https://github.com/snumprlab/flare., Comment: AAAI 2025 (Project page: https://twoongg.github.io/projects/flare/)

We present the first search for the lepton flavor-violating decay modes $B^0 \rightarrow K_S^0 \tau^\pm \ell^\mp~(\ell=\mu, e)$ using the 711 fb$^{-1}$ and 365 fb$^{-1}$ data samples recorded by the Belle and Belle II detectors, respectively. We use a hadronic $B$-tagging technique, and search for the signal decay in the system recoiling against the fully reconstructed $B$ meson. We find no evidence for $B^0 \rightarrow K_S^0 \tau^\pm \ell^\mp$ decays and set 90\% confidence level upper limits on the branching fractions in the range of $[0.8,\,3.6]\times10^{-5}$.

The transmon, which has a short gate time and remarkable scalability, is the most commonly utilized superconducting qubit, based on the Cooper pair box as a qubit or coupler in superconducting quantum computers. Lattice and heavy-hexagon structures are well-known large-scale configurations for transmon-based quantum computers that classical computers cannot simulate. These structures share a common feature: a resonator coupler that connects two transmon qubits. Although significant progress has been made in implementing quantum error correction and quantum computing using quantum error mitigation, fault-tolerant quantum computing remains unachieved due to the inherent vulnerability of these structures. This raises the question of whether the transmon-resonator-transmon structure is the best option for constructing a transmon-based quantum computer. To address this, we demonstrate that the average fidelity of CNOT gates can exceed 0.98 in a structure where a resonator coupler mediates the coupling of three transmon qubits. This result suggests that our novel structure could be a key method for increasing the number of connections among qubits while preserving gate performance in a transmon-based quantum computer.

Previous Deepfake detection methods perform well within their training domains, but their effectiveness diminishes significantly with new synthesis techniques. Recent studies have revealed that detection models often create decision boundaries based on facial identity rather than synthetic artifacts, resulting in poor performance on cross-domain datasets. To address this limitation, we propose Facial Recognition Identity Attenuation (FRIDAY), a novel training method that mitigates facial identity influence using a face recognizer. Specifically, we first train a face recognizer using the same backbone as the Deepfake detector. The recognizer is then frozen and employed during the detector's training to reduce facial identity information. This is achieved by feeding input images into both the recognizer and the detector, and minimizing the similarity of their feature embeddings through our Facial Identity Attenuating loss. This process encourages the detector to generate embeddings distinct from the recognizer, effectively reducing the impact of facial identity. Extensive experiments demonstrate that our approach significantly enhances detection performance on both in-domain and cross-domain datasets., Comment: 5 pages, 4 figures. In 2024 IEEE International Conference on Visual Communications and Image Processing (VCIP) Oral

With the growing demand for solutions to real-world video challenges, interest in dense video captioning (DVC) has been on the rise. DVC involves the automatic captioning and localization of untrimmed videos. Several studies highlight the challenges of DVC and introduce improved methods utilizing prior knowledge, such as pre-training and external memory. In this research, we propose a model that leverages the prior knowledge of human-oriented hierarchical compact memory inspired by human memory hierarchy and cognition. To mimic human-like memory recall, we construct a hierarchical memory and a hierarchical memory reading module. We build an efficient hierarchical compact memory by employing clustering of memory events and summarization using large language models. Comparative experiments demonstrate that this hierarchical memory recall process improves the performance of DVC by achieving state-of-the-art performance on YouCook2 and ViTT datasets., Comment: AAAI2025

A remarkable characteristic of quantum computing is the potential for reliable computation despite faulty qubits. This can be achieved through quantum error correction, which is typically implemented by repeatedly applying static syndrome checks, permitting correction of logical information. Recently, the development of time-dynamic approaches to error correction has uncovered new codes and new code implementations. In this work, we experimentally demonstrate three time-dynamic implementations of the surface code, each offering a unique solution to hardware design challenges and introducing flexibility in surface code realization. First, we embed the surface code on a hexagonal lattice, reducing the necessary couplings per qubit from four to three. Second, we walk a surface code, swapping the role of data and measure qubits each round, achieving error correction with built-in removal of accumulated non-computational errors. Finally, we realize the surface code using iSWAP gates instead of the traditional CNOT, extending the set of viable gates for error correction without additional overhead. We measure the error suppression factor when scaling from distance-3 to distance-5 codes of $\Lambda_{35,\text{hex}} = 2.15(2)$, $\Lambda_{35,\text{walk}} = 1.69(6)$, and $\Lambda_{35,\text{iSWAP}} = 1.56(2)$, achieving state-of-the-art error suppression for each. With detailed error budgeting, we explore their performance trade-offs and implications for hardware design. This work demonstrates that dynamic circuit approaches satisfy the demands for fault-tolerance and opens new alternative avenues for scalable hardware design., Comment: 11 pages, 5 figures, Supplementary Information

We measure the branching fraction and $\it CP$-violating flavor-dependent rate asymmetry of $B^{0} \to \pi^{0} \pi^{0}$ decays reconstructed using the Belle II detector in an electron-positron collision sample containing $387 \times 10^{6}$ $B\overline{B}$ pairs. Using an optimized event selection, we find $126\pm 20$ signal decays in a fit to background-discriminating and flavor-sensitive distributions. The resulting branching fraction is $(1.25 \pm 0.23)\times 10^{-6}$ and the $\it CP$-violating asymmetry is $0.03 \pm 0.30$.

Quantum error correction is essential for bridging the gap between the error rates of physical devices and the extremely low logical error rates required for quantum algorithms. Recent error-correction demonstrations on superconducting processors have focused primarily on the surface code, which offers a high error threshold but poses limitations for logical operations. In contrast, the color code enables much more efficient logic, although it requires more complex stabilizer measurements and decoding techniques. Measuring these stabilizers in planar architectures such as superconducting qubits is challenging, and so far, realizations of color codes have not addressed performance scaling with code size on any platform. Here, we present a comprehensive demonstration of the color code on a superconducting processor, achieving logical error suppression and performing logical operations. Scaling the code distance from three to five suppresses logical errors by a factor of $\Lambda_{3/5}$ = 1.56(4). Simulations indicate this performance is below the threshold of the color code, and furthermore that the color code may be more efficient than the surface code with modest device improvements. Using logical randomized benchmarking, we find that transversal Clifford gates add an error of only 0.0027(3), which is substantially less than the error of an idling error correction cycle. We inject magic states, a key resource for universal computation, achieving fidelities exceeding 99% with post-selection (retaining about 75% of the data). Finally, we successfully teleport logical states between distance-three color codes using lattice surgery, with teleported state fidelities between 86.5(1)% and 90.7(1)%. This work establishes the color code as a compelling research direction to realize fault-tolerant quantum computation on superconducting processors in the near future.

Visual re-ranking using Nearest Neighbor graph~(NN graph) has been adapted to yield high retrieval accuracy, since it is beneficial to exploring an high-dimensional manifold and applicable without additional fine-tuning. The quality of visual re-ranking using NN graph, however, is limited to that of connectivity, i.e., edges of the NN graph. Some edges can be misconnected with negative images. This is known as a noisy edge problem, resulting in a degradation of the retrieval quality. To address this, we propose a complementary denoising method based on Continuous Conditional Random Field (C-CRF) that uses a statistical distance of our similarity-based distribution. This method employs the concept of cliques to make the process computationally feasible. We demonstrate the complementarity of our method through its application to three visual re-ranking methods, observing quality boosts in landmark retrieval and person re-identification (re-ID).

Large language models (LLMs) have exhibited outstanding performance in natural language processing tasks. However, these models remain susceptible to adversarial attacks in which slight input perturbations can lead to harmful or misleading outputs. A gradient-based defensive suffix generation algorithm is designed to bolster the robustness of LLMs. By appending carefully optimized defensive suffixes to input prompts, the algorithm mitigates adversarial influences while preserving the models' utility. To enhance adversarial understanding, a novel total loss function ($L_{\text{total}}$) combining defensive loss ($L_{\text{def}}$) and adversarial loss ($L_{\text{adv}}$) generates defensive suffixes more effectively. Experimental evaluations conducted on open-source LLMs such as Gemma-7B, mistral-7B, Llama2-7B, and Llama2-13B show that the proposed method reduces attack success rates (ASR) by an average of 11\% compared to models without defensive suffixes. Additionally, the perplexity score of Gemma-7B decreased from 6.57 to 3.93 when applying the defensive suffix generated by openELM-270M. Furthermore, TruthfulQA evaluations demonstrate consistent improvements with Truthfulness scores increasing by up to 10\% across tested configurations. This approach significantly enhances the security of LLMs in critical applications without requiring extensive retraining., Comment: 9 pages, 2 figures

The Multimodal Learning Workshop (PBVS 2024) aims to improve the performance of automatic target recognition (ATR) systems by leveraging both Synthetic Aperture Radar (SAR) data, which is difficult to interpret but remains unaffected by weather conditions and visible light, and Electro-Optical (EO) data for simultaneous learning. The subtask, known as the Multi-modal Aerial View Imagery Challenge - Classification, focuses on predicting the class label of a low-resolution aerial image based on a set of SAR-EO image pairs and their respective class labels. The provided dataset consists of SAR-EO pairs, characterized by a severe long-tail distribution with over a 1000-fold difference between the largest and smallest classes, making typical long-tail methods difficult to apply. Additionally, the domain disparity between the SAR and EO datasets complicates the effectiveness of standard multimodal methods. To address these significant challenges, we propose a two-stage learning approach that utilizes self-supervised techniques, combined with multimodal learning and inference through SAR-to-EO translation for effective EO utilization. In the final testing phase of the PBVS 2024 Multi-modal Aerial View Image Challenge - Classification (SAR Classification) task, our model achieved an accuracy of 21.45%, an AUC of 0.56, and a total score of 0.30, placing us 9th in the competition., Comment: 4 pages, 3 figures, 1 Table

Real-world decision-making often requires integrating and reasoning over information from multiple modalities. While recent multimodal large language models (MLLMs) have shown promise in such tasks, their ability to perform multi-hop reasoning across diverse sources remains insufficiently evaluated. Existing benchmarks, such as MMQA, face challenges due to (1) data contamination and (2) a lack of complex queries that necessitate operations across more than two modalities, hindering accurate performance assessment. To address this, we present Financial Cross-Modal Multi-Hop Reasoning (FCMR), a benchmark created to analyze the reasoning capabilities of MLLMs by urging them to combine information from textual reports, tables, and charts within the financial domain. FCMR is categorized into three difficulty levels-Easy, Medium, and Hard-facilitating a step-by-step evaluation. In particular, problems at the Hard level require precise cross-modal three-hop reasoning and are designed to prevent the disregard of any modality. Experiments on this new benchmark reveal that even state-of-the-art MLLMs struggle, with the best-performing model (Claude 3.5 Sonnet) achieving only 30.4% accuracy on the most challenging tier. We also conduct analysis to provide insights into the inner workings of the models, including the discovery of a critical bottleneck in the information retrieval phase.

Recently, generalizable feed-forward methods based on 3D Gaussian Splatting have gained significant attention for their potential to reconstruct 3D scenes using finite resources. These approaches create a 3D radiance field, parameterized by per-pixel 3D Gaussian primitives, from just a few images in a single forward pass. However, unlike multi-view methods that benefit from cross-view correspondences, 3D scene reconstruction with a single-view image remains an underexplored area. In this work, we introduce CATSplat, a novel generalizable transformer-based framework designed to break through the inherent constraints in monocular settings. First, we propose leveraging textual guidance from a visual-language model to complement insufficient information from a single image. By incorporating scene-specific contextual details from text embeddings through cross-attention, we pave the way for context-aware 3D scene reconstruction beyond relying solely on visual cues. Moreover, we advocate utilizing spatial guidance from 3D point features toward comprehensive geometric understanding under single-view settings. With 3D priors, image features can capture rich structural insights for predicting 3D Gaussians without multi-view techniques. Extensive experiments on large-scale datasets demonstrate the state-of-the-art performance of CATSplat in single-view 3D scene reconstruction with high-quality novel view synthesis.

Recent AI-based video editing has enabled users to edit videos through simple text prompts, significantly simplifying the editing process. However, recent zero-shot video editing techniques primarily focus on global or single-object edits, which can lead to unintended changes in other parts of the video. When multiple objects require localized edits, existing methods face challenges, such as unfaithful editing, editing leakage, and lack of suitable evaluation datasets and metrics. To overcome these limitations, we propose a zero-shot $\textbf{M}$ulti-$\textbf{I}$nstance $\textbf{V}$ideo $\textbf{E}$diting framework, called MIVE. MIVE is a general-purpose mask-based framework, not dedicated to specific objects (e.g., people). MIVE introduces two key modules: (i) Disentangled Multi-instance Sampling (DMS) to prevent editing leakage and (ii) Instance-centric Probability Redistribution (IPR) to ensure precise localization and faithful editing. Additionally, we present our new MIVE Dataset featuring diverse video scenarios and introduce the Cross-Instance Accuracy (CIA) Score to evaluate editing leakage in multi-instance video editing tasks. Our extensive qualitative, quantitative, and user study evaluations demonstrate that MIVE significantly outperforms recent state-of-the-art methods in terms of editing faithfulness, accuracy, and leakage prevention, setting a new benchmark for multi-instance video editing. The project page is available at https://kaist-viclab.github.io/mive-site/, Comment: The first two authors contributed equally to this work. The last two authors are co-corresponding authors. Please visit our project page at https://kaist-viclab.github.io/mive-site/

Autonomous exploration is a crucial aspect of robotics, enabling robots to explore unknown environments and generate maps without prior knowledge. This paper proposes a method to enhance exploration efficiency by integrating neural network-based occupancy grid map prediction with uncertainty-aware Bayesian neural network. Uncertainty from neural network-based occupancy grid map prediction is probabilistically integrated into mutual information for exploration. To demonstrate the effectiveness of the proposed method, we conducted comparative simulations within a frontier exploration framework in a realistic simulator environment against various information metrics. The proposed method showed superior performance in terms of exploration efficiency., Comment: 7pages

Scene Graph Generation (SGG) research has suffered from two fundamental challenges: the long-tailed predicate distribution and semantic ambiguity between predicates. These challenges lead to a bias towards head predicates in SGG models, favoring dominant general predicates while overlooking fine-grained predicates. In this paper, we address the challenges of SGG by framing it as multi-label classification problem with partial annotation, where relevant labels of fine-grained predicates are missing. Under the new frame, we propose Retrieval-Augmented Scene Graph Generation (RA-SGG), which identifies potential instances to be multi-labeled and enriches the single-label with multi-labels that are semantically similar to the original label by retrieving relevant samples from our established memory bank. Based on augmented relations (i.e., discovered multi-labels), we apply multi-prototype learning to train our SGG model. Several comprehensive experiments have demonstrated that RA-SGG outperforms state-of-the-art baselines by up to 3.6% on VG and 5.9% on GQA, particularly in terms of F@K, showing that RA-SGG effectively alleviates the issue of biased prediction caused by the long-tailed distribution and semantic ambiguity of predicates., Comment: 7 pages

This paper studies a hybrid language model (HLM) architecture that integrates a small language model (SLM) operating on a mobile device with a large language model (LLM) hosted at the base station (BS) of a wireless network. The HLM token generation process follows the speculative inference principle: the SLM's vocabulary distribution is uploaded to the LLM, which either accepts or rejects it, with rejected tokens being resampled by the LLM. While this approach ensures alignment between the vocabulary distributions of the SLM and LLM, it suffers from low token throughput due to uplink transmission and the computation costs of running both language models. To address this, we propose a novel HLM structure coined Uncertainty-aware opportunistic HLM (U-HLM), wherein the SLM locally measures its output uncertainty and skips both uplink transmissions and LLM operations for tokens that are likely to be accepted. This opportunistic skipping is enabled by our empirical finding of a linear correlation between the SLM's uncertainty and the LLM's rejection probability. We analytically derive the uncertainty threshold and evaluate its expected risk of rejection. Simulations show that U-HLM reduces uplink transmissions and LLM computations by 45.93%, while achieving up to 97.54% of the LLM's inference accuracy and 2.54$\times$ faster token throughput than HLM without skipping., Comment: 6 pages, 6 figures; This work has been submitted to the IEEE for possible publication