Search

Recent advances on Multi-modal Large Language Models have demonstrated that high-resolution image input is crucial for model capabilities, especially for fine-grained tasks. However, high-resolution images lead to a quadratic increase in the number of visual tokens input into LLMs, resulting in significant computational costs. Current work develop visual token compression methods to achieve efficiency improvements, often at the expense of performance. We argue that removing visual redundancy can simultaneously improve both efficiency and performance. We build a coarse-to-fine visual token compression method, with a vision-guided sampler for compressing redundant regions with low information density, and a text-guided sampler for selecting visual tokens that are strongly correlated with the user instructions.With these two modules, the proposed FocusLLaVA achieves improvements in both efficiency and performance. We validate the effectiveness of our approach on a wide range of evaluation datasets.

Shadow, as a natural consequence of light interacting with objects, plays a crucial role in shaping the aesthetics of an image, which however also impairs the content visibility and overall visual quality. Recent shadow removal approaches employ the mechanism of attention, due to its effectiveness, as a key component. However, they often suffer from two issues including large model size and high computational complexity for practical use. To address these shortcomings, this work devises a lightweight yet accurate shadow removal framework. First, we analyze the characteristics of the shadow removal task to seek the key information required for reconstructing shadow regions and designing a novel regional attention mechanism to effectively capture such information. Then, we customize a Regional Attention Shadow Removal Model (RASM, in short), which leverages non-shadow areas to assist in restoring shadow ones. Unlike existing attention-based models, our regional attention strategy allows each shadow region to interact more rationally with its surrounding non-shadow areas, for seeking the regional contextual correlation between shadow and non-shadow areas. Extensive experiments are conducted to demonstrate that our proposed method delivers superior performance over other state-of-the-art models in terms of accuracy and efficiency, making it appealing for practical applications.

$H_0$ tension in the spatially flat $\Lambda$CDM model is reevaluated by employing three sets of non-Planck CMB data, namely WMAP, WMAP+ACT, and WMAP+SPT, in conjunction with DESI BAO data and non-DESI BAO datasets including 6dFGS, SDSS DR7, and SDSS DR16. Our analysis yields $H_0 = 68.86\pm 0.68~\mathrm{km\ s^{-1} Mpc^{-1}}$ with WMAP+DESI BAO, $H_0 = 68.72\pm 0.51~\mathrm{km\ s^{-1} Mpc^{-1}}$ with WMAP+ACT+DESI BAO, and $H_0 = 68.62\pm 0.52~\mathrm{km\ s^{-1} Mpc^{-1}}$ with WMAP+SPT+DESI BAO. The results of non-Planck CMB+DESI BAO exhibit a $3.4\sigma$, $3.7\sigma$, and $3.8\sigma$ tension with the SH0ES local measurement respectively which are around $1 \sigma$ lower in significance for the Hubble tension compared to Planck CMB+DESI BAO. Moreover, by combining DESI BAO data+non-Planck CMB measurements, we obtain a more stringent constraint on the Hubble constant compared to non-DESI BAO data+non-Planck CMB data, as well as reducing the significance of the Hubble tension., Comment: 5 pages, 1 figure, 2 tables

Massive black holes (MBHs) exist in the Galactic center (GC) and other nearby galactic nuclei. As natural outcome of galaxy mergers, some MBHs may have a black hole (BH) companion. In this paper, assuming that the MBHs in the GC and some nearby galaxies are in binaries with orbital periods ranging from months to years (gravitational-wave frequency $\sim1-100$\,nHz), we investigate the detectability of gravitational-waves from these binary MBHs (BBHs) and constraints on the parameter space for the existence of BBHs in the GC, LMC, M31, M32, and M87, that may be obtained by current/future pulsar timing array (PTA) observations. We find that a BBH in the GC, if any, can be revealed by the Square Kilometer Array PTA (SKA-PTA) if its mass ratio $q\gtrsim10^{-4}-10^{-3}$ and semimajor axis $a\sim20-10^3$\,AU. The existence of a BH companion of the MBH can be revealed by SKA-PTA with $\sim20$-year observations in M31 if $q\gtrsim10^{-4}$ and $a\sim10^2-10^4$\,AU or in M87 if $q\gtrsim10^{-5}$ and $a\sim10^3-2\times10^4$\,AU, but not in LMC and M32 if $q\ll1$. If a number of milli-second stable pulsars with distances $\lesssim0.1-1$\,pc away from the central MBH in the GC, LMC, M32, or M31, can be detected in future and applied to PTAs, the BH companion with mass even down to $\sim100M_\odot$, close to stellar masses, can be revealed by such PTAs. Future PTAs are expected to provide an independent way to reveal BBHs and low-mass MBH companions in the GC and nearby galaxies, improving our understandings of the formation and evolution of MBHs and galaxies., Comment: 20 pages, 13 figures, accepted for publication in ApJ

A high-efficient one-step synthesis of cubic gauche polymeric nitrogen was developed just by thermal treatment of KN3 powders. The Raman and infrared spectra confirm the formation of polymeric nitrogen networks. Thermogravimetric differential scanning calorimeter measurements show that the content of cubic gauche polymeric nitrogen is as high as 1.5 wt% with high thermal stability, which is the highest content value so far., Comment: 7 pages, 4 figures

Stereo matching has been a pivotal component in 3D vision, aiming to find corresponding points between pairs of stereo images to recover depth information. In this work, we introduce StereoAnything, a highly practical solution for robust stereo matching. Rather than focusing on a specialized model, our goal is to develop a versatile foundational model capable of handling stereo images across diverse environments. To this end, we scale up the dataset by collecting labeled stereo images and generating synthetic stereo pairs from unlabeled monocular images. To further enrich the model's ability to generalize across different conditions, we introduce a novel synthetic dataset that complements existing data by adding variability in baselines, camera angles, and scene types. We extensively evaluate the zero-shot capabilities of our model on five public datasets, showcasing its impressive ability to generalize to new, unseen data. Code will be available at \url{https://github.com/XiandaGuo/OpenStereo}., Comment: Code will be available at \url{https://github.com/XiandaGuo/OpenStereo}

Peptides are biomolecules comprised of amino acids that play an important role in our body. In recent years, peptides have received extensive attention in drug design and synthesis, and peptide prediction tasks help us better search for functional peptides. Typically, we use the primary sequence and structural information of peptides for model encoding. However, recent studies have focused more on single-modal information (structure or sequence) for prediction without multi-modal approaches. We found that single-modal models are not good at handling datasets with less information in that particular modality. Therefore, this paper proposes the M2oE multi-modal collaborative expert peptide model. Based on previous work, by integrating sequence and spatial structural information, employing expert model and Cross-Attention Mechanism, the model's capabilities are balanced and improved. Experimental results indicate that the M2oE model performs excellently in complex task predictions., Comment: accepted by bibm 2024

Hidden spin polarization (HSP) with zero net spin polarization in total but non-zero local spin polarization has been proposed in certain nonmagnetic centrosymmetric compounds, where the individual sectors forming the inversion partners are all inversion asymmetry. Here, we extend this idea to antiferromagnetic materials with $PT$ symmetry (the joint symmetry of space inversion symmetry ($P$) and time-reversal symmetry ($T$)), producing zero net spin polarization in total, but either of the two inversion-partner sectors possesses altermagnetism, giving rise to non-zero local spin polarization in the real space, dubbed "hidden altermagnetism". By first-principle calculations, we predict that $PT$-symmetric bilayer $\mathrm{Cr_2SO}$ can serve as a possible candidate showing altermagnetic HSP. By applying an external electric field to break the global $P$ symmetry, the hidden altermagnetism can be separated and observed experimentally. Our works extend the hidden physics, and will also advance the theoretical and experimental search for new type of spin-polarized materials., Comment: 7 pages, 5 figures

This paper is concerned with the asymptotic spreading behavior of solutions of the Lotka-Volterra system with strong competition in $\mathbb{R}^{N}$. Two types of initial conditions are proposed: (C1) two species initially occupy bounded domains; (C2) two species initially occupy the whole space separately. The spreading dynamics for (C1) (C2) is strongly depending on the speeds of traveling fronts of the scalar equations with no competition and the system. We give the asymptotic speeds of spreading for both (C1) (C2).

While text-to-image models offer numerous benefits, they also pose significant societal risks. Detecting AI-generated images is crucial for mitigating these risks. Detection methods can be broadly categorized into passive and watermark-based approaches: passive detectors rely on artifacts present in AI-generated images, whereas watermark-based detectors proactively embed watermarks into such images. A key question is which type of detector performs better in terms of effectiveness, robustness, and efficiency. However, the current literature lacks a comprehensive understanding of this issue. In this work, we aim to bridge that gap by developing ImageDetectBench, the first comprehensive benchmark to compare the effectiveness, robustness, and efficiency of passive and watermark-based detectors. Our benchmark includes four datasets, each containing a mix of AI-generated and non-AI-generated images. We evaluate five passive detectors and four watermark-based detectors against eight types of common perturbations and three types of adversarial perturbations. Our benchmark results reveal several interesting findings. For instance, watermark-based detectors consistently outperform passive detectors, both in the presence and absence of perturbations. Based on these insights, we provide recommendations for detecting AI-generated images, e.g., when both types of detectors are applicable, watermark-based detectors should be the preferred choice.

Telesurgery is an effective way to deliver service from expert surgeons to areas without immediate access to specialized resources. However, many of these areas, such as rural districts or battlefields, might be subject to different problems in communication, especially latency and intermittent periods of communication outage. This challenge motivates the use of a digital twin for the surgical system, where a simulation would mirror the robot hardware and surgical environment in the real world. The surgeon would then be able to interact with the digital twin during communication outage, followed by a recovery strategy on the real robot upon reestablishing communication. This paper builds the digital twin for the da Vinci surgical robot, with a buffering and replay strategy that reduces the mean task completion time by 23% when compared to the baseline, for a peg transfer task subject to intermittent communication outage.

Being closely connected to the origin of the nucleon mass, the gravitational form factors of the nucleon have attracted significant attention in recent years. We present the first model-independent precise determinations of the gravitational form factors of the pion and nucleon at the physical pion mass, using a data-driven dispersive approach. The so-called ``last unknown global property'' of the nucleon, the $D$-term, is determined to be $-\left(3.38^{+0.26}_{-0.32}\right)$. The root mean square radius of the mass distribution inside the nucleon is determined to be $0.97^{+0.02}_{-0.03}~\text{fm}$. Notably, this value is larger than the proton charge radius, suggesting a modern structural view of the nucleon where gluons, responsible for most of the nucleon mass, are distributed over a larger spatial region than quarks, which dominate the charge distribution. We also predict the nucleon angular momentum and mechanical radii, providing further insights into the intricate internal structure of the nucleon., Comment: 8 pages, 6 figures

As large language models (LLMs) rapidly advance and integrate into daily life, the privacy risks they pose are attracting increasing attention. We focus on a specific privacy risk where LLMs may help identify the authorship of anonymous texts, which challenges the effectiveness of anonymity in real-world systems such as anonymous peer review systems. To investigate these risks, we present AIDBench, a new benchmark that incorporates several author identification datasets, including emails, blogs, reviews, articles, and research papers. AIDBench utilizes two evaluation methods: one-to-one authorship identification, which determines whether two texts are from the same author; and one-to-many authorship identification, which, given a query text and a list of candidate texts, identifies the candidate most likely written by the same author as the query text. We also introduce a Retrieval-Augmented Generation (RAG)-based method to enhance the large-scale authorship identification capabilities of LLMs, particularly when input lengths exceed the models' context windows, thereby establishing a new baseline for authorship identification using LLMs. Our experiments with AIDBench demonstrate that LLMs can correctly guess authorship at rates well above random chance, revealing new privacy risks posed by these powerful models. The source code and data will be made publicly available after acceptance., Comment: 21 pages, 7 figures

The 1D Schr\"odinger equation closed with the transparent boundary conditions(TBCs) is known as a successful model for describing quantum effects, and is usually considered with a self-consistent Poisson equation in simulating quantum devices. We introduce discrete fourth order transparent boundary conditions(D4TBCs), which have been proven to be essentially non-oscillating when the potential vanishes, and to share the same accuracy order with the finite difference scheme used to discretize the 1D Schr\"odinger equation. Furthermore, a framework of analytic discretization of TBCs(aDTBCs) is proposed, which does not introduce any discretization error, thus is accurate. With the accurate discretizations, one is able to improve the accuracy of the discretization for the 1D Schr\"odinger problem to arbitrarily high levels. As numerical tools, two globally fourth order compact finite difference schemes are proposed for the 1D Schr\"odinger-Poisson problem, involving either of the D4TBCs or the aDTBCs, respectively, and the uniqueness of solutions of both discrete Schr\"odinger problems are rigorously proved. Numerical experiments, including simulations of a resistor and two nanoscale resonant tunneling diodes, verify the accuracy order of the discretization schemes and show potential of the numerical algorithm introduced for the 1D Schr\"odinger-Poisson problem in simulating various quantum devices.

The autobidding system generates huge revenue for advertising platforms, garnering substantial research attention. Existing studies in autobidding systems focus on designing Autobidding Incentive Compatible (AIC) mechanisms, where the mechanism is Incentive Compatible (IC) under ex ante expectations. However, upon deploying AIC mechanisms in advertising platforms, we observe a notable deviation between the actual auction outcomes and these expectations during runtime, particularly in the scene with few clicks (sparse-click). This discrepancy undermines truthful bidding among advertisers in AIC mechanisms, especially for risk-averse advertisers who are averse to outcomes that do not align with the expectations. To address this issue, we propose a mechanism, Decoupled First-Price Auction (DFP), that retains its IC property even during runtime. DFP dynamically adjusts the payment based on real-time user conversion outcomes, ensuring that advertisers' realized utilities closely approximate their expected utilities during runtime. To realize the payment mechanism of DFP, we propose a PPO-based RL algorithm, with a meticulously crafted reward function. This algorithm dynamically adjusts the payment to fit DFP mechanism. We conduct extensive experiments leveraging real-world data to validate our findings.

Quantum frequency converters that enable the interface between the itinerant photons and qubits are indispensable for realizing long-distance quantum network. However, the cascaded connection between converters and qubits usually brings additional insertion loss and intermediate noises. Here, we propose a cooperative quantum interface (CQI) that integrates the converter and qubit coupling into a single device for efficient long-distance entanglement generation. Compared to traditional cascaded systems, our scheme offers several advantages, including compactness, reduced insertion loss, and suppression of noise from intermediate modes. We prove the excellent performance over the separated devices by about two orders of magnitude for the entangled infidelity of two remote nodes. Moreover, we discuss an extended scheme for multiple remote nodes, revealing an exponential advantage in performance as the number of nodes increases. The cooperative effect is universal that can be further applied to multifunctional integrated quantum devices. This work opens up novel prospects for quantum networks, distributed quantum computing, and sensing., Comment: 7 pages, 3 figures

Efficient microwave-to-optical frequency conversion (MOC) is crucial for applications such as radiometry, electrometry, quantum microwave illumination and quantum networks. Rydberg atoms provide a unique platform for realizing free-space MOC, promising wide-bandwidth, scalable, and flexible quantum interfaces. Here, we develop a theoretical framework to evaluate the system conversion efficiency, accounting for the mismatch between microwave and optical wavelengths comparing with the atomic ensemble size. Our analysis reveals that the conversion efficiency is fundamentally limited by the focusing of the free-space microwave field, with an upper bound of about 3/16 for diffraction-limited focusing. We propose using a microwave near-field antenna to overcome this limit. Our work provides a foundation for assessing and optimizing free-space MOC, paving the way for a variety of applications based on free-space MOC., Comment: 6 pages, 4 figures

Autonomous driving requires a comprehensive understanding of 3D environments to facilitate high-level tasks such as motion prediction, planning, and mapping. In this paper, we introduce DriveMLLM, a benchmark specifically designed to evaluate the spatial understanding capabilities of multimodal large language models (MLLMs) in autonomous driving. DriveMLLM includes 880 front-facing camera images and introduces both absolute and relative spatial reasoning tasks, accompanied by linguistically diverse natural language questions. To measure MLLMs' performance, we propose novel evaluation metrics focusing on spatial understanding. We evaluate several state-of-the-art MLLMs on DriveMLLM, and our results reveal the limitations of current models in understanding complex spatial relationships in driving contexts. We believe these findings underscore the need for more advanced MLLM-based spatial reasoning methods and highlight the potential for DriveMLLM to drive further research in autonomous driving. Code will be available at \url{https://github.com/XiandaGuo/Drive-MLLM}., Comment: Code will be available at \url{https://github.com/XiandaGuo/Drive-MLLM}

Generalist vision language models (VLMs) have made significant strides in computer vision, but they fall short in specialized fields like healthcare, where expert knowledge is essential. In traditional computer vision tasks, creative or approximate answers may be acceptable, but in healthcare, precision is paramount.Current large multimodal models like Gemini and GPT-4o are insufficient for medical tasks due to their reliance on memorized internet knowledge rather than the nuanced expertise required in healthcare. VLMs are usually trained in three stages: vision pre-training, vision-language pre-training, and instruction fine-tuning (IFT). IFT has been typically applied using a mixture of generic and healthcare data. In contrast, we propose that for medical VLMs, a fourth stage of specialized IFT is necessary, which focuses on medical data and includes information from domain expert models. Domain expert models developed for medical use are crucial because they are specifically trained for certain clinical tasks, e.g. to detect tumors and classify abnormalities through segmentation and classification, which learn fine-grained features of medical data$-$features that are often too intricate for a VLM to capture effectively especially in radiology. This paper introduces a new framework, VILA-M3, for medical VLMs that utilizes domain knowledge via expert models. Through our experiments, we show an improved state-of-the-art (SOTA) performance with an average improvement of ~9% over the prior SOTA model Med-Gemini and ~6% over models trained on the specific tasks. Our approach emphasizes the importance of domain expertise in creating precise, reliable VLMs for medical applications.

The following paper provides a multi-band channel measurement analysis on the frequency range (FR)3. This study focuses on the FR3 low frequencies 6.5 GHz and 8.75 GHz with a setup tailored to the context of integrated sensing and communication (ISAC), where the data are collected with and without the presence of a target. A method based on multiple signal classification (MUSIC) is used to refine the delays of the channel impulse response estimates. The results reveal that the channel at the lower frequency 6.5 GHz has additional distinguishable multipath components in the presence of the target, while the one associated with the higher frequency 8.75 GHz has more blockage. The set of results reported in this paper serves as a benchmark for future multi-band studies in the FR3 spectrum.

Recent advances in code-specific large language models (LLMs) have greatly enhanced code generation and refinement capabilities. However, the safety of code LLMs remains under-explored, posing potential risks as insecure code generated by these models may introduce vulnerabilities into real-world systems. Previous work proposes to collect security-focused instruction-tuning dataset from real-world vulnerabilities. It is constrained by the data sparsity of vulnerable code, and has limited applicability in the iterative post-training workflows of modern LLMs. In this paper, we propose ProSec, a novel proactive security alignment approach designed to align code LLMs with secure coding practices. ProSec systematically exposes the vulnerabilities in a code LLM by synthesizing error-inducing coding scenarios from Common Weakness Enumerations (CWEs), and generates fixes to vulnerable code snippets, allowing the model to learn secure practices through advanced preference learning objectives. The scenarios synthesized by ProSec triggers 25 times more vulnerable code than a normal instruction-tuning dataset, resulting in a security-focused alignment dataset 7 times larger than the previous work. Experiments show that models trained with ProSec is 29.2% to 35.5% more secure compared to previous work, with a marginal negative effect of less than 2 percentage points on model's utility., Comment: The first two authors contributed equally to this work

Traditional error detection and correction codes focus on bit-level fidelity, which is insufficient for emerging technologies like eXtended Reality (XR) and holographic communications requiring high-data-rate, low-latency systems. Bit-level metrics cannot comprehensively evaluate Quality-of-Service (QoS) in these scenarios. This letter proposes TopoCode which leverages Topological Data Analysis (TDA) and persistent homology to encode topological information for message-level error detection and correction. It introduces minimal redundancy while enabling effective data reconstruction, especially in low Signal-to-Noise Ratio (SNR) conditions. TopoCode offers a promising approach to meet the demands of next-generation communication systems prioritizing semantic accuracy and message-level integrity.

We investigate the evolution of supernova remnants (SNRs) in a two-phase cloudy medium by performing a series of high-resolution (up to $\Delta x\approx0.01\,\mathrm{pc}$), 3D hydrodynamical simulations including radiative cooling and thermal conduction. We aim to reach a resolution that directly captures the shock-cloud interactions for the majority of the clouds initialized by the saturation of thermal instability. In comparison to the SNR in a uniform medium with the volume filling warm medium, the SNR expands similarly (following $\propto t^{2/5}$) but sweeps up more mass as the cold clouds contribute before shocks in the warm medium become radiative. However, the SNR in a cloudy medium continuously loses energy after shocks toward the cold clouds cool, resulting in less hot gas mass, thermal energy, and terminal momentum. Thermal conduction has little effect on the dynamics of the SNR but smooths the morphology and modifies the internal structure by increasing the density of hot gas by a factor of $\sim 3-5$. The simulation results are not fully consistent with many previous 1D models describing the SNR in a cloudy medium including a mass loading term. By direct measurement in the simulations, we find that, apart from the mass source, the energy sink is also important with a spatially flat cooling rate $\dot{e}\propto t^{-11/5}$. As an illustration, we show an example 1D model including both mass source and energy sink terms (in addition to the radiative cooling in the volume filling component) that better describes the structure of the simulated SNR., Comment: 24 pages, 14 figures, submitted to ApJ

Vision-Language (V-L) pre-trained models such as CLIP show prominent capabilities in various downstream tasks. Despite this promise, V-L models are notoriously limited by their inherent social biases. A typical demonstration is that V-L models often produce biased predictions against specific groups of people, significantly undermining their real-world applicability. Existing approaches endeavor to mitigate the social bias problem in V-L models by removing biased attribute information from model embeddings. However, after our revisiting of these methods, we find that their bias removal is frequently accompanied by greatly compromised V-L alignment capabilities. We then reveal that this performance degradation stems from the unbalanced debiasing in image and text embeddings. To address this issue, we propose a novel V-L debiasing framework to align image and text biases followed by removing them from both modalities. By doing so, our method achieves multi-modal bias mitigation while maintaining the V-L alignment in the debiased embeddings. Additionally, we advocate a new evaluation protocol that can 1) holistically quantify the model debiasing and V-L alignment ability, and 2) evaluate the generalization of social bias removal models. We believe this work will offer new insights and guidance for future studies addressing the social bias problem in CLIP.

The analysis of 3D medical images is crucial for modern healthcare, yet traditional task-specific models are becoming increasingly inadequate due to limited generalizability across diverse clinical scenarios. Multimodal large language models (MLLMs) offer a promising solution to these challenges. However, existing MLLMs have limitations in fully leveraging the rich, hierarchical information embedded in 3D medical images. Inspired by clinical practice, where radiologists focus on both 3D spatial structure and 2D planar content, we propose Med-2E3, a novel MLLM for 3D medical image analysis that integrates 3D and 2D encoders. To aggregate 2D features more effectively, we design a Text-Guided Inter-Slice (TG-IS) scoring module, which scores the attention of each 2D slice based on slice contents and task instructions. To the best of our knowledge, Med-2E3 is the first MLLM to integrate both 3D and 2D features for 3D medical image analysis. Experiments on a large-scale, open-source 3D medical multimodal benchmark demonstrate that Med-2E3 exhibits task-specific attention distribution and significantly outperforms current state-of-the-art models, with a 14% improvement in report generation and a 5% gain in medical visual question answering (VQA), highlighting the model's potential in addressing complex multimodal clinical tasks. The code will be released upon acceptance.

Architects adopt visual scripting and parametric design tools to explore more expansive design spaces (Coates, 2010), refine their thinking about the geometric logic of their design (Woodbury, 2010), and overcome conventional software limitations (Burry, 2011). Despite two decades of effort to make design scripting more accessible, a disconnect between a designer's free ways of thinking and the rigidity of algorithms remains (Burry, 2011). Recent developments in Large Language Models (LLMs) suggest this might soon change, as LLMs encode a general understanding of human context and exhibit the capacity to produce geometric logic. This project speculates that if LLMs can effectively mediate between user intent and algorithms, they become a powerful tool to make scripting in design more widespread and fun. We explore if such systems can interpret natural language prompts to assemble geometric operations relevant to computational design scripting. In the system, multiple layers of LLM agents are configured with specific context to infer the user intent and construct a sequential logic. Given a user's high-level text prompt, a geometric description is created, distilled into a sequence of logic operations, and mapped to software-specific commands. The completed script is constructed in the user's visual programming interface. The system succeeds in generating complete visual scripts up to a certain complexity but fails beyond this complexity threshold. It shows how LLMs can make design scripting much more aligned with human creativity and thought. Future research should explore conversational interactions, expand to multimodal inputs and outputs, and assess the performance of these tools., Comment: Published at ACADIA 2024

Transitions between solid-like and fluid-like states in living tissues have been found in steps of embryonic development and in stages of disease progression. Our current understanding of these transitions has been guided by experimental and theoretical investigations focused on how motion becomes arrested with increased mechanical coupling between cells, typically as a function of packing density or cell cohesiveness. However, cells actively respond to externally applied forces by contracting after a time delay, so it is possible that at some packing densities or levels of cell cohesiveness, mechanical coupling stimulates cell motion instead of suppressing it. Here we report our findings that at low densities and within multiple ranges of cell cohesiveness, cell migration speeds increase with these measures of mechanical coupling. Our observations run counter to our intuition that cell motion will be suppressed by increasingly packing or sticking cells together and may provide new insight into biological processes involving motion in dense cell populations., Comment: 5 figures; supplementary information

Snapshot Compressive Imaging (SCI) offers a possibility for capturing information in high-speed dynamic scenes, requiring efficient reconstruction method to recover scene information. Despite promising results, current deep learning-based and NeRF-based reconstruction methods face challenges: 1) deep learning-based reconstruction methods struggle to maintain 3D structural consistency within scenes, and 2) NeRF-based reconstruction methods still face limitations in handling dynamic scenes. To address these challenges, we propose SCIGS, a variant of 3DGS, and develop a primitive-level transformation network that utilizes camera pose stamps and Gaussian primitive coordinates as embedding vectors. This approach resolves the necessity of camera pose in vanilla 3DGS and enhances multi-view 3D structural consistency in dynamic scenes by utilizing transformed primitives. Additionally, a high-frequency filter is introduced to eliminate the artifacts generated during the transformation. The proposed SCIGS is the first to reconstruct a 3D explicit scene from a single compressed image, extending its application to dynamic 3D scenes. Experiments on both static and dynamic scenes demonstrate that SCIGS not only enhances SCI decoding but also outperforms current state-of-the-art methods in reconstructing dynamic 3D scenes from a single compressed image. The code will be made available upon publication.

Integrated photonic circuits play a crucial role in implementing quantum information processing in the noisy intermediate-scale quantum (NISQ) era. Variational learning is a promising avenue that leverages classical optimization techniques to enhance quantum advantages on NISQ devices. However, most variational algorithms are circuit-model-based and encounter challenges when implemented on integrated photonic circuits, because they involve explicit decomposition of large quantum circuits into sequences of basic entangled gates, leading to an exponential decay of success probability due to the non-deterministic nature of photonic entangling gates. Here, we present a variational learning approach for designing quantum photonic circuits, which directly incorporates post-selection and elementary photonic elements into the training process. The complicated circuit is treated as a single nonlinear logical operator, and a unified design is discovered for it through variational learning. Engineering an integrated photonic chip with automated control, we adjust and optimize the internal parameters of the chip in real time for task-specific cost functions. We utilize a simple case of designing photonic circuits for a single ancilla CNOT gate with improved success rate to illustrate how our proposed approach works, and then apply the approach in the first demonstration of quantum stochastic simulation using integrated photonics.

Neutral atoms have emerged as a promising technology for implementing quantum computers due to their scalability and long coherence times. However, the execution frequency of neutral atom quantum computers is constrained by image processing procedures, particularly the assembly of defect-free atom arrays, which is a crucial step in preparing qubits (atoms) for execution. To optimize this assembly process, we propose a novel quadrant-based rearrangement algorithm that employs a divide-and-conquer strategy and also enables the simultaneous movement of multiple atoms, even across different columns and rows. We implement the algorithm on FPGA to handle each quadrant independently (hardware-level optimization) while maximizing parallelization. To the best of our knowledge, this is the first hardware acceleration work for atom rearrangement, and it significantly reduces the processing time. This achievement also contributes to the ongoing efforts of tightly integrating quantum accelerators into High-Performance Computing (HPC) systems. Tested on a Zynq RFSoC FPGA at 250 MHz, our hardware implementation is able to complete the rearrangement process of a 30$\times$30 compact target array, derived from a 50$\times$50 initial loaded array, in approximately 1.0 $\mu s$. Compared to a comparable CPU implementation and to state-of-the-art FPGA work, we achieved about 54$\times$ and 300$\times$ speedups in the rearrangement analysis time, respectively. Additionally, the FPGA-based acceleration demonstrates good scalability, allowing for seamless adaptation to varying sizes of the atom array, which makes this algorithm a promising solution for large-scale quantum systems., Comment: The paper is accepted by Design, Automation and Test in Europe Conference (DATE), 2025

The fast evolution of SARS-CoV-2 and other infectious viruses poses a grand challenge to the rapid response in terms of viral tracking, diagnostics, and design and manufacture of monoclonal antibodies (mAbs) and vaccines, which are both time-consuming and costly. This underscores the need for efficient computational approaches. Recent advancements, like topological deep learning (TDL), have introduced powerful tools for forecasting emerging dominant variants, yet they require deep mutational scanning (DMS) of viral surface proteins and associated three-dimensional (3D) protein-protein interaction (PPI) complex structures. We propose an AlphaFold 3 (AF3)-assisted multi-task topological Laplacian (MT-TopLap) strategy to address this need. MT-TopLap combines deep learning with topological data analysis (TDA) models, such as persistent Laplacians (PL) to extract detailed topological and geometric characteristics of PPIs, thereby enhancing the prediction of DMS and binding free energy (BFE) changes upon virus mutations. Validation with four experimental DMS datasets of SARS-CoV-2 spike receptor-binding domain (RBD) and the human angiotensin-converting enzyme-2 (ACE2) complexes indicates that our AF3 assisted MT-TopLap strategy maintains robust performance, with only an average 1.1% decrease in Pearson correlation coefficients (PCC) and an average 9.3% increase in root mean square errors (RMSE), compared with the use of experimental structures. Additionally, AF3-assisted MT-TopLap achieved a PCC of 0.81 when tested with a SARS-CoV-2 HK.3 variant DMS dataset, confirming its capability to accurately predict BFE changes and adapt to new experimental data, thereby showcasing its potential for rapid and effective response to fast viral evolution., Comment: 21 pages, 4 figures

It is widely acknowledged that the performance of Transformer models is exponentially related to their number of parameters and computational complexity. While approaches like Mixture of Experts (MoE) decouple parameter count from computational complexity, they still face challenges in inference due to high memory access costs. This work introduces UltraMem, incorporating large-scale, ultra-sparse memory layer to address these limitations. Our approach significantly reduces inference latency while maintaining model performance. We also investigate the scaling laws of this new architecture, demonstrating that it not only exhibits favorable scaling properties but outperforms traditional models. In our experiments, we train networks with up to 20 million memory slots. The results show that our method achieves state-of-the-art inference speed and model performance within a given computational budget., Comment: 10 pages, 6 figures

The challenge in LLM-based video understanding lies in preserving visual and semantic information in long videos while maintaining a memory-affordable token count. However, redundancy and correspondence in videos have hindered the performance potential of existing methods. Through statistical learning on current datasets, we observe that redundancy occurs in both repeated and answer-irrelevant frames, and the corresponding frames vary with different questions. This suggests the possibility of adopting dynamic encoding to balance detailed video information preservation with token budget reduction. To this end, we propose a dynamic cooperative network, DynFocus, for memory-efficient video encoding in this paper. Specifically, i) a Dynamic Event Prototype Estimation (DPE) module to dynamically select meaningful frames for question answering; (ii) a Compact Cooperative Encoding (CCE) module that encodes meaningful frames with detailed visual appearance and the remaining frames with sketchy perception separately. We evaluate our method on five publicly available benchmarks, and experimental results consistently demonstrate that our method achieves competitive performance., Comment: 8 pages, 6 figures

Quantum computing solutions are increasingly deployed in commercial environments through delegated computing, especially one of the most critical issues is to guarantee the confidentiality and proprietary of quantum implementations. Since the proposal of general-purpose indistinguishability obfuscation (iO) and functional encryption schemes, iO has emerged as a seemingly versatile cryptography primitive. Existing research on quantum indistinguishable obfuscation (QiO) primarily focuses on task-oriented, lacking solutions to general quantum computing. In this paper, we propose a scheme for constructing QiO via the equivalence of quantum circuits. It introduces the concept of quantum subpath sum equivalence, demonstrating that indistinguishability between two quantum circuits can be achieved by incremental changes in quantum subpaths. The restriction of security loss is solved by reducing the distinguisher to polynomial probability test. The scheme obfuscates the quantum implementation of classical functions in a path-sum specification, ensuring the indistinguishability between different quantum implementations. The results demonstrate the feasibility of indistinguishability obfuscation for general circuits and provide novel insights on intellectual property protection and secure delegated quantum computing.

Here, we propose an isospectral reduction (IR) approach for the mapping of a trimer Su-Schrieffer-Heeger (SSH3) lattice into a simplified two-site model, whose coupling dynamics ingeniously results in a precise bulk-edge correspondence of the original lattice. The isospectrally-reduced model has inter-cell couplings with dynamic response to the eigenstate energy, allowing for the control of topological phase transition by energy. We relate the bulk property of the reduced model to the band topology of the SSH3 lattice, allowing for there distinct topological phases with different number of topological edge state pair. An acoustic SSH3 chain is fabricated for experimental demonstration. The said topological edge state pairs are measured. Our study takes a pivotal step toward the exploration of topology in multiple wave systems, opening up possibilities for advanced control of topological waves.

The aim of this paper is to study the product of $n$ linear forms over function fields. We calculate the maximum value of the minima of the forms with determinant one when $n$ is small. The value is equal to the natural bound given by algebraic number theory. Our proof is based on a reduction theory of diagonal group orbits on homogeneous spaces. We also show that the forms defined algebraically correspond to periodic orbits with respect to the diagonal group actions.

Human's perception of the visual world is shaped by the stereo processing of 3D information. Understanding how the brain perceives and processes 3D visual stimuli in the real world has been a longstanding endeavor in neuroscience. Towards this goal, we introduce a new neuroscience task: decoding 3D visual perception from EEG signals, a neuroimaging technique that enables real-time monitoring of neural dynamics enriched with complex visual cues. To provide the essential benchmark, we first present EEG-3D, a pioneering dataset featuring multimodal analysis data and extensive EEG recordings from 12 subjects viewing 72 categories of 3D objects rendered in both videos and images. Furthermore, we propose Neuro-3D, a 3D visual decoding framework based on EEG signals. This framework adaptively integrates EEG features derived from static and dynamic stimuli to learn complementary and robust neural representations, which are subsequently utilized to recover both the shape and color of 3D objects through the proposed diffusion-based colored point cloud decoder. To the best of our knowledge, we are the first to explore EEG-based 3D visual decoding. Experiments indicate that Neuro-3D not only reconstructs colored 3D objects with high fidelity, but also learns effective neural representations that enable insightful brain region analysis. The dataset and associated code will be made publicly available.

Dynamic scene reconstruction is a long-term challenge in 3D vision. Recent methods extend 3D Gaussian Splatting to dynamic scenes via additional deformation fields and apply explicit constraints like motion flow to guide the deformation. However, they learn motion changes from individual timestamps independently, making it challenging to reconstruct complex scenes, particularly when dealing with violent movement, extreme-shaped geometries, or reflective surfaces. To address the above issue, we design a plug-and-play module called TimeFormer to enable existing deformable 3D Gaussians reconstruction methods with the ability to implicitly model motion patterns from a learning perspective. Specifically, TimeFormer includes a Cross-Temporal Transformer Encoder, which adaptively learns the temporal relationships of deformable 3D Gaussians. Furthermore, we propose a two-stream optimization strategy that transfers the motion knowledge learned from TimeFormer to the base stream during the training phase. This allows us to remove TimeFormer during inference, thereby preserving the original rendering speed. Extensive experiments in the multi-view and monocular dynamic scenes validate qualitative and quantitative improvement brought by TimeFormer. Project Page: https://patrickddj.github.io/TimeFormer/

Robotic manipulation refers to the autonomous handling and interaction of robots with objects using advanced techniques in robotics and artificial intelligence. The advent of powerful tools such as large language models (LLMs) and large vision-language models (LVLMs) has significantly enhanced the capabilities of these robots in environmental perception and decision-making. However, the introduction of these intelligent agents has led to security threats such as jailbreak attacks and adversarial attacks. In this research, we take a further step by proposing a backdoor attack specifically targeting robotic manipulation and, for the first time, implementing backdoor attack in the physical world. By embedding a backdoor visual language model into the visual perception module within the robotic system, we successfully mislead the robotic arm's operation in the physical world, given the presence of common items as triggers. Experimental evaluations in the physical world demonstrate the effectiveness of the proposed backdoor attack., Comment: Initial version with preliminary results. We welcome any feedback or suggestions

By employing ray-tracing techniques, we investigate the shadow images of rotating Bardeen black holes surrounded by perfect fluid dark matter. In this work, two models are considered for the background light source, namely the celestial light source model and the thin accretion disk model. Regarding the celestial light source, the investigation focuses on the impact of variations in relevant parameters and observed inclination on the contour and size of the shadow. For the thin accretion disk model, the optical appearance of a black hole is evidently contingent upon the radiative properties exhibited by the accretion disk, as well as factors such as observed inclination and relevant parameters governing spacetime. With an increasing observation inclination, the observed flux of direct and lensed images of the accretion disk gradually converge towards the lower region of the image, while an increase in the dark matter parameter $a$ significantly expands the region encompassing both direct and lensed images. Furthermore, the predominant effect is redshift at lower observation angles, whereas the blueshift effect only becomes apparent at higher observation angles. Simultaneously, the increase in the observation inclination will amplify the redshift effect, whereas an increase in the magnetic charge $\mathcal{G}$, rotation parameter $a$ and the absolute value of dark matter parameter $\alpha$ will attenuate the redshift effect observed in the image. These observations of a rotating Bardeen black hole surrounded by perfect fluid dark matter could provide a convenient way to distinguish it from other black hole models., Comment: 32 pages, 12 figures

The Kirkwood-Dirac distribution, serving as an informationally complete representation of a quantum state, has recently garnered { increasing} attention. We investigate the Kirkwood-Dirac classicality with respect to mutually unbiased bases. For prime dimensional Hilbert spaces, we demonstrate that quantum states which exhibit Kirkwood-Dirac classicality for two distinct sets of mutually unbiased bases $A$, $B$ and $A$, $B'$ must necessarily be incoherent with respect to $A$. We subsequently introduce a coherence monotone based on Kirkwood-Dirac nonclassicality with respect to mutually unbiased bases. Additionally, we establish that this coherence monotone can be expressed through weak values, suggesting that quantum coherence can be utilized to detect anomalous weak values., Comment: 9 pages, 3 figures

The Genebass dataset, released by Karczewski et al. (2022), provides a comprehensive resource elucidating associations between genes and 4,529 phenotypes based on nearly 400,000 exomes from the UK Biobank. This extensive dataset enables the evaluation of gene set enrichment across a wide range of phenotypes, facilitating the inference of associations between specified gene sets and phenotypic traits. Despite its potential, no established method for applying gene set enrichment analysis (GSEA) to Genebass data exists. To address this gap, we propose utilizing fast pre-ranked gene set enrichment analysis (FGSEA) as a novel approach to determine whether a specified set of genes is significantly enriched in phenotypes within the UK Biobank. We developed an R package, ukbFGSEA, to implement this analysis, completed with a hands-on tutorial. Our approach has been validated by analyzing gene sets associated with autism spectrum disorder, developmental disorder, and neurodevelopmental disorders, demonstrating its capability to reveal established and novel associations.

Decades ago, Sondheimer discovered that the electric conductivity of metallic crystals hosting ballistic electrons oscillates with magnetic field. These oscillations, periodic in magnetic field and the period proportional to the sample thickness, have been understood in a semi-classical framework. Here, we present a study of longitudinal and transverse conductivity in cadmium single crystals with thickness varying between 12.6 to 475 $\mu$m. When the magnetic field is sufficiently large or the sample sufficiently thick, the amplitude of oscillation falls off as $B^{-4}$ as previously reported. In contrast, the ten first oscillations follow a $B^{-2.5}e^{-B/B_0}$ field dependence and their amplitude is set by the quantum of conductance, the sample thickness, the magnetic length and the Fermi surface geometry. We demonstrate that they are beyond the semi-classical picture, as the exponential prefactor indicates quantum tunneling between distinct quantum states. We draw a picture of these quantum oscillations, in which the linear dispersion of the semi-Dirac band in the cadmium plays a crucial role. The oscillations arise by the intersection between the lowest Landau tube and flat toroids on the Fermi surface induced by confinement. Positive and negative corrections to semi-classical magneto-conductance can occur by alternation between destructive and constructive interference in phase-coherent helical states. The quantum limit of Sondheimer oscillations emerges as another manifestation of Aharanov-Bohm flux quantization., Comment: 12 pages, 10 figures

2D van der Waals (vdW) magnetic semiconductors offer an ideal platform to achieve exciton-polaritons correlated with magnetic orders for developing solid-state quantum, spintronic, and photonic devices. However, in the as-demonstrated systems, the coupling strength, light velocity, and effective mass of exciton-polaritons are almost inert to the external magnetic field. Here, we report the experimental observation of giant magnetic tuning of B exciton-polaritons in vdW magnet CrSBr crystals. The Rabi splitting energy is tuned up to 100 meV within a moderate 0.45 T in-plane magnetic field due to changes in excitonic states during the spin transitions. Besides, the light velocity and effective mass of exciton-polaritons are tuned by 10.3% and 26.1% below 100 meV of exciton energy. Further, the strong coupling persists at room temperature, controlled by temperature-tailored exciton-magnon and exciton-phonon coupling. These results advance the development of on-demand strong coupling systems for spin-correlated quantum optoelectronic applications.

We establish the profound equivalence between measures of genuine multipartite entanglement(GME) and their corresponding coherence measures. Initially we construct two distinct classes of measures for genuine multipartite entanglement utilizing real symmetric concave functions and the convex roof technique. We then demonstrate that all coherence measures for any qudit states, defined through the convex roof approach, are identical to our two classes of GME measures of the states combined with an incoherent ancilla under a unitary incoherent operation. This relationship implies that genuine multipartite entanglement can be generated from the coherence inherent in an initial state through the unitary incoherent operations. Furthermore, we explore the interplay between coherence and other forms of genuine quantum correlations, specifically genuine multipartite steering and genuine multipartite nonlocality. In the instance of special three-qubit X-states (only nonzero elements of X-state are diagonal or antidiagonal when written in an orthonormal basis), we find that genuine multipartite steering and nonlocality are present if and only if the coherence exists in the corresponding qubit states.

Research on two-dimensional van der Waals materials has demonstrated that the layer degree of freedom can significantly alter the physical properties of materials due to the substantial modification of bulk bands. Inspired by this concept, layered photonic systems have been proposed and realized, revealing novel phenomena absent in their monolayer counterparts. In this work, we experimentally investigate the band structures of bilayer valley photonic crystals. Two typical structures with different stacking configurations are experimentally imaged via the near-field scanning technology, exhibiting distinct bulk band structures. Furthermore, different topological edge modes induced by distinct topology are observed, revealing that the layer degree of freedom can be regarded as a pseudospin and offer further capabilities for controlling the flow of light. Our work not only elucidates the evolution of band structures from monolayer to bilayer topological systems but also provides an experimental platform for the further exploration of bilayer topological insulators., Comment: 20 pages, 5 figures

Transmission line defect recognition models have traditionally used general pre-trained weights as the initial basis for their training. These models often suffer weak generalization capability due to the lack of domain knowledge in the pre-training dataset. To address this issue, we propose a two-stage transmission-line-oriented contrastive language-image pre-training (TL-CLIP) framework, which lays a more effective foundation for transmission line defect recognition. The pre-training process employs a novel power-specific multimodal algorithm assisted with two power-specific pre-training tasks for better modeling the power-related semantic knowledge contained in the inspection data. To fine-tune the pre-trained model, we develop a transfer learning strategy, namely fine-tuning with pre-training objective (FTP), to alleviate the overfitting problem caused by limited inspection data. Experimental results demonstrate that the proposed method significantly improves the performance of transmission line defect recognition in both classification and detection tasks, indicating clear advantages over traditional pre-trained models in the scene of transmission line inspection.

In recent years, few-shot action recognition has achieved remarkable performance through spatio-temporal relation modeling. Although a wide range of spatial and temporal alignment modules have been proposed, they primarily address spatial or temporal misalignments at the video level, while the spatio-temporal relationships across different videos at the task level remain underexplored. Recent studies utilize class prototypes to learn task-specific features but overlook the spatio-temporal relationships across different videos at the task level, especially in the spatial dimension, where these relationships provide rich information. In this paper, we propose a novel Dual Motion-Guided Attention Learning method (called DMGAL) for few-shot action recognition, aiming to learn the spatio-temporal relationships from the video-specific to the task-specific level. To achieve this, we propose a carefully designed Motion-Guided Attention (MGA) method to identify and correlate motion-related region features from the video level to the task level. Specifically, the Self Motion-Guided Attention module (S-MGA) achieves spatio-temporal relation modeling at the video level by identifying and correlating motion-related region features between different frames within a video. The Cross Motion-Guided Attention module (C-MGA) identifies and correlates motion-related region features between frames of different videos within a specific task to achieve spatio-temporal relationships at the task level. This approach enables the model to construct class prototypes that fully incorporate spatio-temporal relationships from the video-specific level to the task-specific level. We validate the effectiveness of our DMGAL method by employing both fully fine-tuning and adapter-tuning paradigms. The models developed using these paradigms are termed DMGAL-FT and DMGAL-Adapter, respectively.

We consider the following stochastic space-time fractional diffusion equation with vanishing initial condition:$$ \partial^{\beta} u(t, x)=- \left(-\Delta\right)^{\alpha / 2} u(t, x)+ I_{0+}^{\gamma}\left[\dot{W}(t, x)\right],\quad t\in[0,T],\: x \in \mathbb{R}^d,$$ where $\alpha>0$, $\beta\in(0,2)$, $\gamma\in[0,1)$, $\left(-\Delta\right)^{\alpha/2}$ is the fractional/power of Laplacian and $\dot{W}$ is a fractional space-time Gaussian noise. We prove the existence and uniqueness of the solution and then focus on various sample path regularity properties of the solution. More specifically, we establish the exact uniform and local moduli of continuity and Chung-type laws of the iterated logarithm. The small ball probability is also studied., Comment: 42 pages

We address the issue of the testability of instrumental variables derived from observational data. Most existing testable implications are centered on scenarios where the treatment is a discrete variable, e.g., instrumental inequality (Pearl, 1995), or where the effect is assumed to be constant, e.g., instrumental variables condition based on the principle of independent mechanisms (Burauel, 2023). However, treatments can often be continuous variables, such as drug dosages or nutritional content levels, and non-constant effects may occur in many real-world scenarios. In this paper, we consider an additive nonlinear, non-constant effects model with unmeasured confounders, in which treatments can be either discrete or continuous, and propose an Auxiliary-based Independence Test (AIT) condition to test whether a variable is a valid instrument. We first show that if the candidate instrument is valid, then the AIT condition holds. Moreover, we illustrate the implications of the AIT condition and demonstrate that, in certain conditions, AIT conditions are necessary and sufficient to detect all invalid IVs. We also extend the AIT condition to include covariates and introduce a practical testing algorithm. Experimental results on both synthetic and three different real-world datasets show the effectiveness of our proposed condition.