Search

This study investigates the application of machine/deep learning to classify snoring sounds excited at different levels of the upper airway in patients with obstructive sleep apnea (OSA) using data from drug-induced sleep endoscopy (DISE). The snoring sounds of 39 subjects were analyzed and labeled according to the Velum, Oropharynx, Tongue Base, and Epiglottis (VOTE) classification system. The dataset, comprising 5,173 one-second segments, was used to train and test models, including Support Vector Machine (SVM), Bidirectional Long Short-Term Memory (BiLSTM), and ResNet-50. The ResNet-50, a convolutional neural network (CNN), showed the best overall performance in classifying snoring acoustics, particularly in identifying multi-level obstructions. The study emphasizes the potential of integrating snoring acoustics with deep learning to improve the diagnosis and treatment of OSA. However, challenges such as limited sample size, data imbalance, and differences between pharmacologically induced and natural snoring sounds were noted, suggesting further research to enhance model accuracy and generalizability.

Transparent objects are common in daily life, while their unique optical properties pose challenges for RGB-D cameras, which struggle to capture accurate depth information. For assistant robots, accurately perceiving transparent objects held by humans is essential for effective human-robot interaction. This paper presents a Hand-Aware Depth Restoration (HADR) method for hand-held transparent objects based on creating an implicit neural representation function from a single RGB-D image. The proposed method introduces the hand posture as an important guidance to leverage semantic and geometric information. To train and evaluate the proposed method, we create a high-fidelity synthetic dataset called TransHand-14K with a real-to-sim data generation scheme. Experiments show that our method has a better performance and generalization ability compared with existing methods. We further develop a real-world human-to-robot handover system based on the proposed depth restoration method, demonstrating its application value in human-robot interaction., Comment: 7 pages, 7 figures, conference

GitHub issue resolving is a critical task in software engineering, recently gaining significant attention in both industry and academia. Within this task, SWE-bench has been released to evaluate issue resolving capabilities of large language models (LLMs), but has so far only focused on Python version. However, supporting more programming languages is also important, as there is a strong demand in industry. As a first step toward multilingual support, we have developed a Java version of SWE-bench, called SWE-bench-java. We have publicly released the dataset, along with the corresponding Docker-based evaluation environment and leaderboard, which will be continuously maintained and updated in the coming months. To verify the reliability of SWE-bench-java, we implement a classic method SWE-agent and test several powerful LLMs on it. As is well known, developing a high-quality multi-lingual benchmark is time-consuming and labor-intensive, so we welcome contributions through pull requests or collaboration to accelerate its iteration and refinement, paving the way for fully automated programming., Comment: This work is in progress

Low-dose computed tomography (LDCT) offers significant advantages in reducing the potential harm to human bodies. However, reducing the X-ray dose in CT scanning often leads to severe noise and artifacts in the reconstructed images, which might adversely affect diagnosis. By utilizing the expectation maximization (EM) algorithm, statistical priors could be combined with artificial priors to improve LDCT reconstruction quality. However, conventional EM-based regularization methods adopt an alternating solving strategy, i.e. full reconstruction followed by image-regularization, resulting in over-smoothing and slow convergence. In this paper, we propose to integrate TV regularization into the ``M''-step of the EM algorithm, thus achieving effective and efficient regularization. Besides, by employing the Chambolle-Pock (CP) algorithm and the ordered subset (OS) strategy, we propose the OSEM-CP algorithm for LDCT reconstruction, in which both reconstruction and regularization are conducted view-by-view. Furthermore, by unrolling OSEM-CP, we propose an end-to-end reconstruction neural network (NN), named OSEM-CPNN, with remarkable performance and efficiency that achieves high-quality reconstructions in just one full-view iteration. Experiments on different models and datasets demonstrate our methods' outstanding performance compared to traditional and state-of-the-art deep-learning methods., Comment: 11 pages, 8 figures

The drift instabilities driven by the slab ion temperature gradient (ITG) in Kappa-distributed plasmas are investigated by the kinetic method. The linear dispersion relation is given in an integral representation involving only the standard plasma dispersion function. The wave frequency and growth rate are derived without the density inhomogeneity. Numerical solutions of the dispersion equation are conducted to show the different effects of the suprathermal ions and electrons. We find that the suprathermal ions can enhance the instability in large wavenumbers but suppress it in small wavenumbers. Thus, the suprathermalization of ions could be one of the factors leading to a lower limit of wavenumbers for the ITG instabilities. Besides, the numerical calculations also imply that the thermal speed ratio affects the intensities of the suprathermal effects. Finally, in the presence of density inhomogeneity, the ITG instability boundary is numerically analyzed., Comment: 17 pages, 6 figures. Accepted by Plasma Physics and Controlled Fusion

Objective: To explore and compare the performance of ChatGPT and other state-of-the-art LLMs on domain-specific NER tasks covering different entity types and domains in TCM against COVID-19 literature. Methods: We established a dataset of 389 articles on TCM against COVID-19, and manually annotated 48 of them with 6 types of entities belonging to 3 domains as the ground truth, against which the NER performance of LLMs can be assessed. We then performed NER tasks for the 6 entity types using ChatGPT (GPT-3.5 and GPT-4) and 4 state-of-the-art BERT-based question-answering (QA) models (RoBERTa, MiniLM, PubMedBERT and SciBERT) without prior training on the specific task. A domain fine-tuned model (GSAP-NER) was also applied for a comprehensive comparison. Results: The overall performance of LLMs varied significantly in exact match and fuzzy match. In the fuzzy match, ChatGPT surpassed BERT-based QA models in 5 out of 6 tasks, while in exact match, BERT-based QA models outperformed ChatGPT in 5 out of 6 tasks but with a smaller F-1 difference. GPT-4 showed a significant advantage over other models in fuzzy match, especially on the entity type of TCM formula and the Chinese patent drug (TFD) and ingredient (IG). Although GPT-4 outperformed BERT-based models on entity type of herb, target, and research method, none of the F-1 scores exceeded 0.5. GSAP-NER, outperformed GPT-4 in terms of F-1 by a slight margin on RM. ChatGPT achieved considerably higher recalls than precisions, particularly in the fuzzy match. Conclusions: The NER performance of LLMs is highly dependent on the entity type, and their performance varies across application scenarios. ChatGPT could be a good choice for scenarios where high recall is favored. However, for knowledge acquisition in rigorous scenarios, neither ChatGPT nor BERT-based QA models are off-the-shelf tools for professional practitioners., Comment: 22 pages with 2 figures

Many bacteria live in natural and clinical environments with abundant macromolecular polymers. Macromolecular fluids commonly display viscoelasticity and non-Newtonian rheological behavior; it is unclear how these complex-fluid properties affect bacterial transport in flows. Here we combine high-resolution microscopy and numerical simulations to study bacterial response to shear flows of various macromolecular fluids. In stark contrast to the case in Newtonian shear flows, we found that flagellated bacteria in macromolecular flows display a giant capacity of upstream swimming (a behavior resembling fish swimming against current) near solid surfaces: The cells can counteract flow washing at shear rates up to ~65 $s^{-1}$, one order of magnitude higher than the limit for cells swimming in Newtonian flows. The significant enhancement of upstream swimming depends on two characteristic complex-fluid properties, namely viscoelasticity and shear-thinning viscosity; meanwhile, increasing the viscosity with a Newtonian polymer can prevent upstream motion. By visualizing flagellar bundles and modeling bacterial swimming in complex fluids, we explain the phenomenon as primarily arising from the augmentation of a "weathervane effect" in macromolecular flows due to the presence of a viscoelastic lift force and a shear-thinning induced azimuthal torque promoting the alignment of bacteria against the flow direction. Our findings shed light on bacterial transport and surface colonization in macromolecular environments, and may inform the design of artificial helical microswimmers for biomedical applications in physiological conditions.

Bacteria often exhibit upstream swimming, which can cause the contamination of biomedical devices and the infection of organs including the urethra or lungs. This process, called rheotaxis, has been studied extensively in Newtonian fluids. However, most microorganisms thrive in non-Newtonian fluids that contain suspended polymers such as mucus and biofilms. Here, we investigate the rheotatic behavior of E. coli near walls in non-Newtonian fluids. Our experiments demonstrate that bacterial upstream swimming is enhanced by an order of magnitude in shear-thinning polymeric fluids relative to Newtonian fluids. This result is explained by direct numerical simulations, revealing a torque that promotes the alignment of bacteria against the flow. From this analysis, we develop a theoretical model that accurately describes experimental rheotatic data in both Newtonian and shear-thinning fluids.

We present Bidirectional Gaussian Primitives, an image-based novel view synthesis technique designed to represent and render 3D objects with surface and volumetric materials under dynamic illumination. Our approach integrates light intrinsic decomposition into the Gaussian splatting framework, enabling real-time relighting of 3D objects. To unify surface and volumetric material within a cohesive appearance model, we adopt a light- and view-dependent scattering representation via bidirectional spherical harmonics. Our model does not use a specific surface normal-related reflectance function, making it more compatible with volumetric representations like Gaussian splatting, where the normals are undefined. We demonstrate our method by reconstructing and rendering objects with complex materials. Using One-Light-At-a-Time (OLAT) data as input, we can reproduce photorealistic appearances under novel lighting conditions in real time.

In this study, we conducted simulations to find the geometric aberrations expected for images taken by the Main Survey Camera (MSC) of the Chinese Space Station Telescope (CSST) due to its motion. As anticipated by previous work, our findings indicate that the geometric distortion of light impacts the focal plane's apparent scale, with a more pronounced influence as the size of the focal plane increases. Our models suggest that the effect consistently influences the pixel scale in both the vertical and parallel directions. The apparent scale variation follows a sinusoidal distribution throughout one orbit period. Simulations reveal that the effect is particularly pronounced in the center of the Galaxy and gradually diminishes along the direction of ecliptic latitude. At low ecliptic latitudes, the total aberration leads to about 0.94 pixels offset (a 20-minute exposure) and 0.26 pixels offset (a 300-second exposure) at the edge of the field of view, respectively. Appropriate processings for the geometric effect during the CSST pre- and post-observation phases are presented., Comment: 13 pages, 8 figures; accepted by RAA

We present xGen-VideoSyn-1, a text-to-video (T2V) generation model capable of producing realistic scenes from textual descriptions. Building on recent advancements, such as OpenAI's Sora, we explore the latent diffusion model (LDM) architecture and introduce a video variational autoencoder (VidVAE). VidVAE compresses video data both spatially and temporally, significantly reducing the length of visual tokens and the computational demands associated with generating long-sequence videos. To further address the computational costs, we propose a divide-and-merge strategy that maintains temporal consistency across video segments. Our Diffusion Transformer (DiT) model incorporates spatial and temporal self-attention layers, enabling robust generalization across different timeframes and aspect ratios. We have devised a data processing pipeline from the very beginning and collected over 13M high-quality video-text pairs. The pipeline includes multiple steps such as clipping, text detection, motion estimation, aesthetics scoring, and dense captioning based on our in-house video-LLM model. Training the VidVAE and DiT models required approximately 40 and 642 H100 days, respectively. Our model supports over 14-second 720p video generation in an end-to-end way and demonstrates competitive performance against state-of-the-art T2V models., Comment: Accepted by ECCV24 AI4VA

Recent advancements in reinforcement learning have made significant impacts across various domains, yet they often struggle in complex multi-agent environments due to issues like algorithm instability, low sampling efficiency, and the challenges of exploration and dimensionality explosion. Hierarchical reinforcement learning (HRL) offers a structured approach to decompose complex tasks into simpler sub-tasks, which is promising for multi-agent settings. This paper advances the field by introducing a hierarchical architecture that autonomously generates effective subgoals without explicit constraints, enhancing both flexibility and stability in training. We propose a dynamic goal generation strategy that adapts based on environmental changes. This method significantly improves the adaptability and sample efficiency of the learning process. Furthermore, we address the critical issue of credit assignment in multi-agent systems by synergizing our hierarchical architecture with a modified QMIX network, thus improving overall strategy coordination and efficiency. Comparative experiments with mainstream reinforcement learning algorithms demonstrate the superior convergence speed and performance of our approach in both single-agent and multi-agent environments, confirming its effectiveness and flexibility in complex scenarios. Our code is open-sourced at: \url{https://github.com/SICC-Group/GMAH}.

To complete a complex task where a robot navigates to a goal object and fetches it, the robot needs to have a good understanding of the instructions and the surrounding environment. Large pre-trained models have shown capabilities to interpret tasks defined via language descriptions. However, previous methods attempting to integrate large pre-trained models with daily tasks are not competent in many robotic goal navigation tasks due to poor understanding of the environment. In this work, we present a visual scene representation built with large-scale visual language models to form a feature representation of the environment capable of handling natural language queries. Combined with large language models, this method can parse language instructions into action sequences for a robot to follow, and accomplish goal navigation with querying the scene representation. Experiments demonstrate that our method enables the robot to follow a wide range of instructions and complete complex goal navigation tasks.

This paper focuses on target-oriented grasping in occluded scenes, where the target object is specified by a binary mask and the goal is to grasp the target object with as few robotic manipulations as possible. Most existing methods rely on a push-grasping synergy to complete this task. To deliver a more powerful target-oriented grasping pipeline, we present MPGNet, a three-branch network for learning a synergy between moving, pushing, and grasping actions. We also propose a multi-stage training strategy to train the MPGNet which contains three policy networks corresponding to the three actions. The effectiveness of our method is demonstrated via both simulated and real-world experiments., Comment: Accepted to IROS 2024

Efficiently detecting target weld seams while ensuring sub-millimeter accuracy has always been an important challenge in autonomous welding, which has significant application in industrial practice. Previous works mostly focused on recognizing and localizing welding seams one by one, leading to inferior efficiency in modeling the workpiece. This paper proposes a novel framework capable of multiple weld seams extraction using both RGB images and 3D point clouds. The RGB image is used to obtain the region of interest by approximately localizing the weld seams, and the point cloud is used to achieve the fine-edge extraction of the weld seams within the region of interest using region growth. Our method is further accelerated by using a pre-trained deep learning model to ensure both efficiency and generalization ability. The performance of the proposed method has been comprehensively tested on various workpieces featuring both linear and curved weld seams and in physical experiment systems. The results showcase considerable potential for real-world industrial applications, emphasizing the method's efficiency and effectiveness. Videos of the real-world experiments can be found at https://youtu.be/pq162HSP2D4.

Structure and dynamics of an active polymer on a smooth cylindrical surface are studied by Brownian dynamics simulations. The effect of active force on the polymer adsorption behavior and the combined effect of chain mobility, length N, rigidity \k{appa}, and cylinder radius, R, on phase diagrams are systemically investigated. We find that complete adsorption is replaced by irregular alternative adsorption/desorption process at a large driving force. Three typical (spiral, helix-like, rod-like) conformations of the active polymer are observed, dependent on N, \k{appa}, and R. Dynamically, the polymer shows rotational motion in spiral state, snake-like motion in the intermediate state, and straight translational motion without turning back in the rod-like state. In the spiral state, we find that rotation velocity {\omega} and chain length follows a power-law relation {\omega}~N^(-0.42), consistent with the torque-balance theory of general Archimedean spirals. And the polymer shows super-diffusive behavior along the cylinder at long time in the helix-like and rod-like states. Our results highlight the mobility, rigidity, as well as curvature of surface can be used to regulate the polymer behavior.

The formation of the massive stars, and in particular, the role that the magnetic fields play in their early evolutionary phase is still far from being completely understood. Here, we present Atacama Large Millimeter/Submillimeter Array (ALMA) 1.2 mm full polarized continuum, and H$^{13}$CO$^+$(3$-$2), CS(5$-$4), and HN$^{13}$C(3$-$2) line observations with a high angular resolution ($\sim$0.4$''$ or 1100 au). In the 1.2 mm continuum emission, we reveal a dusty envelope surrounding the massive protostars, IRAS16547-E and IRAS16547-W, with dimensions of $\sim$10,000 au. This envelope has a bi-conical structure likely carved by the powerful thermal radio jet present in region. The magnetic fields vectors follow very-well the bi-conical envelope. The polarization fraction is $\sim$2.0\% in this region. Some of these vectors seem to converge to IRAS 16547-E, and IRAS 16547-W, the most massive protostars. Moreover, the velocity fields revealed from the spectral lines H$^{13}$CO$^+$(3$-$2), and HN$^{13}$C(3$-$2) show velocity gradients with a good correspondence with the magnetic fields, that maybe are tracing the cavities of molecular outflows or maybe in some parts infall. We derived a magnetic field strength in some filamentary regions that goes from 2 to 6.1\,mG. We also find that the CS(5$-$4) molecular line emission reveals multiple outflow cavities or bow-shocks with different orientations, some of which seem to follow the NW-SE radio thermal jet., Comment: Accepted by the Astrophysical Journal, 13 pages

We investigate the ground-state properties and quantum phase transitions of an ensemble consisting of $N$ four-level atoms within an optical cavity coupled to the single cavity mode and external laser fields. The system is described by an extended imbalanced Dicke model, in which the co- and counterrotating coupling terms are allowed to have different coupling strengths. Some novel polarized phases characterized by the phase differences between the cavity field or the atomic spin excitation and the Raman laser are found analytically. Meanwhile, the full phase diagram and quantum phase transitions are also revealed. Finally, the breaking or restoration of the intrinsic symmetry in this system is addressed. It is found that besides the continuous $U(1)$ and discrete $\mathbb{Z}_2$ symmetries, the system also exhibits two reflection symmetries $\sigma_v$s, a central symmetry $C_2$ in the abstract position-momentum representation, and a discrete reflection parity-time ($\mathcal{PT}$) symmetry, a parameter exchange symmetry $\mathcal{T}_\mathrm{ex}$ in the parameters space. These additional symmetries are governed by two Coxeter groups., Comment: 13 pages, 9 figures

Pre-trained language models are increasingly being used in multi-document summarization tasks. However, these models need large-scale corpora for pre-training and are domain-dependent. Other non-neural unsupervised summarization approaches mostly rely on key sentence extraction, which can lead to information loss. To address these challenges, we propose a lightweight yet effective unsupervised approach called GLIMMER: a Graph and LexIcal features based unsupervised Multi-docuMEnt summaRization approach. It first constructs a sentence graph from the source documents, then automatically identifies semantic clusters by mining low-level features from raw texts, thereby improving intra-cluster correlation and the fluency of generated sentences. Finally, it summarizes clusters into natural sentences. Experiments conducted on Multi-News, Multi-XScience and DUC-2004 demonstrate that our approach outperforms existing unsupervised approaches. Furthermore, it surpasses state-of-the-art pre-trained multi-document summarization models (e.g. PEGASUS and PRIMERA) under zero-shot settings in terms of ROUGE scores. Additionally, human evaluations indicate that summaries generated by GLIMMER achieve high readability and informativeness scores. Our code is available at https://github.com/Oswald1997/GLIMMER., Comment: 19 pages, 7 figures. Accepted by ECAI 2024

Layered two-dimensional (2D) semiconductors have shown enhanced ion migration capabilities along their van der Waals (vdW) gaps and on their surfaces. This effect can be employed for resistive switching (RS) in devices for emerging memories, selectors, and neuromorphic computing. To date, all lateral molybdenum disulfide (MoS${_2}$)-based volatile RS devices with silver (Ag) ion migration have been demonstrated using exfoliated, single-crystal MoS${_2}$ flakes requiring a forming step to enable RS. Here, we present volatile RS with multilayer MoS${_2}$ grown by metal-organic chemical vapor deposition (MOCVD) with repeatable forming-free operation. The devices show highly reproducible volatile RS with low operating voltages of approximately 2 V and fast switching times down to 130 ns considering their micrometer scale dimensions. We investigate the switching mechanism based on Ag ion surface migration through transmission electron microscopy, electronic transport modeling, and density functional theory. Finally, we develop a physics-based compact model and explore the implementation of our volatile memristors as artificial neurons in neuromorphic systems., Comment: 43 pages

Many molecular alterations serve as clinically prognostic or therapy-predictive biomarkers, typically detected using single or multi-gene molecular assays. However, these assays are expensive, tissue destructive and often take weeks to complete. Using AI on routine H&E WSIs offers a fast and economical approach to screen for multiple molecular biomarkers. We present a high-throughput AI-based system leveraging Virchow2, a foundation model pre-trained on 3 million slides, to interrogate genomic features previously determined by an next-generation sequencing (NGS) assay, using 47,960 scanned hematoxylin and eosin (H&E) whole slide images (WSIs) from 38,984 cancer patients. Unlike traditional methods that train individual models for each biomarker or cancer type, our system employs a unified model to simultaneously predict a wide range of clinically relevant molecular biomarkers across cancer types. By training the network to replicate the MSK-IMPACT targeted biomarker panel of 505 genes, it identified 80 high performing biomarkers with a mean AU-ROC of 0.89 in 15 most common cancer types. In addition, 40 biomarkers demonstrated strong associations with specific cancer histologic subtypes. Furthermore, 58 biomarkers were associated with targets frequently assayed clinically for therapy selection and response prediction. The model can also predict the activity of five canonical signaling pathways, identify defects in DNA repair mechanisms, and predict genomic instability measured by tumor mutation burden, microsatellite instability (MSI), and chromosomal instability (CIN). The proposed model can offer potential to guide therapy selection, improve treatment efficacy, accelerate patient screening for clinical trials and provoke the interrogation of new therapeutic targets.

Link prediction has become a critical problem in network science and has thus attracted increasing research interest. Popularity and similarity are two primary mechanisms in the formation of real networks. However, the roles of popularity and similarity mechanisms in link prediction across various domain networks remain poorly understood. Accordingly, this study used orbit degrees of graphlets to construct multi-order popularity- and similarity-based network link predictors, demonstrating that traditional popularity- and similarity-based indices can be efficiently represented in terms of orbit degrees. Moreover, we designed a supervised learning model that fuses multiple orbit-degree-based features and validated its link prediction performance. We also evaluated the mean absolute Shapley additive explanations of each feature within this model across 550 real-world networks from six domains. We observed that the homophily mechanism, which is a similarity-based feature, dominated social networks, with its win rate being 91\%. Moreover, a different similarity-based feature was prominent in economic, technological, and information networks. Finally, no single feature dominated the biological and transportation networks. The proposed approach improves the accuracy and interpretability of link prediction, thus facilitating the analysis of complex networks., Comment: 40 pages, 9 figures

Advancements in autonomous driving have increasingly focused on end-to-end (E2E) systems that manage the full spectrum of driving tasks, from environmental perception to vehicle navigation and control. This paper introduces V2X-VLM, an innovative E2E vehicle-infrastructure cooperative autonomous driving (VICAD) framework with large vision-language models (VLMs). V2X-VLM is designed to enhance situational awareness, decision-making, and ultimate trajectory planning by integrating data from vehicle-mounted cameras, infrastructure sensors, and textual information. The strength of the comprehensive multimodel data fusion of the VLM enables precise and safe E2E trajectory planning in complex and dynamic driving scenarios. Validation on the DAIR-V2X dataset demonstrates that V2X-VLM outperforms existing state-of-the-art methods in cooperative autonomous driving.

This report introduces xGen-MM (also known as BLIP-3), a framework for developing Large Multimodal Models (LMMs). The framework comprises meticulously curated datasets, a training recipe, model architectures, and a resulting suite of LMMs. xGen-MM, short for xGen-MultiModal, expands the Salesforce xGen initiative on foundation AI models. Our models undergo rigorous evaluation across a range of tasks, including both single and multi-image benchmarks. Our pre-trained base model exhibits strong in-context learning capabilities and the instruction-tuned model demonstrates competitive performance among open-source LMMs with similar model sizes. In addition, we introduce a safety-tuned model with DPO, aiming to mitigate harmful behaviors such as hallucinations and improve safety. We open-source our models, curated large-scale datasets, and our fine-tuning codebase to facilitate further advancements in LMM research. Associated resources will be available on our project page above.

Despite the impressive capabilities of Large Language Models (LLMs) in general medical domains, questions remain about their performance in diagnosing rare diseases. To answer this question, we aim to assess the diagnostic performance of LLMs in rare diseases, and explore methods to enhance their effectiveness in this area. In this work, we introduce a rare disease question-answering (ReDis-QA) dataset to evaluate the performance of LLMs in diagnosing rare diseases. Specifically, we collected 1360 high-quality question-answer pairs within the ReDis-QA dataset, covering 205 rare diseases. Additionally, we annotated meta-data for each question, facilitating the extraction of subsets specific to any given disease and its property. Based on the ReDis-QA dataset, we benchmarked several open-source LLMs, revealing that diagnosing rare diseases remains a significant challenge for these models. To facilitate retrieval augmentation generation for rare disease diagnosis, we collect the first rare diseases corpus (ReCOP), sourced from the National Organization for Rare Disorders (NORD) database. Specifically, we split the report of each rare disease into multiple chunks, each representing a different property of the disease, including their overview, symptoms, causes, effects, related disorders, diagnosis, and standard therapies. This structure ensures that the information within each chunk aligns consistently with a question. Experiment results demonstrate that ReCOP can effectively improve the accuracy of LLMs on the ReDis-QA dataset by an average of 8%. Moreover, it significantly guides LLMs to generate trustworthy answers and explanations that can be traced back to existing literature.

In this paper, we construct a large-scale benchmark dataset for Ground-to-Aerial Video-based person Re-Identification, named G2A-VReID, which comprises 185,907 images and 5,576 tracklets, featuring 2,788 distinct identities. To our knowledge, this is the first dataset for video ReID under Ground-to-Aerial scenarios. G2A-VReID dataset has the following characteristics: 1) Drastic view changes; 2) Large number of annotated identities; 3) Rich outdoor scenarios; 4) Huge difference in resolution. Additionally, we propose a new benchmark approach for cross-platform ReID by transforming the cross-platform visual alignment problem into visual-semantic alignment through vision-language model (i.e., CLIP) and applying a parameter-efficient Video Set-Level-Adapter module to adapt image-based foundation model to video ReID tasks, termed VSLA-CLIP. Besides, to further reduce the great discrepancy across the platforms, we also devise the platform-bridge prompts for efficient visual feature alignment. Extensive experiments demonstrate the superiority of the proposed method on all existing video ReID datasets and our proposed G2A-VReID dataset.

Aiming to generalize the label knowledge from a source domain with continuous outputs to an unlabeled target domain, Domain Adaptation Regression (DAR) is developed for complex practical learning problems. However, due to the continuity problem in regression, existing conditional distribution alignment theory and methods with discrete prior, which are proven to be effective in classification settings, are no longer applicable. In this work, focusing on the feasibility problems in DAR, we establish the sufficiency theory for the regression model, which shows the generalization error can be sufficiently dominated by the cross-domain conditional discrepancy. Further, to characterize conditional discrepancy with continuous conditioning variable, a novel Conditional Operator Discrepancy (COD) is proposed, which admits the metric property on conditional distributions via the kernel embedding theory. Finally, to minimize the discrepancy, a COD-based conditional invariant representation learning model is proposed, and the reformulation is derived to show that reasonable modifications on moment statistics can further improve the discriminability of the adaptation model. Extensive experiments on standard DAR datasets verify the validity of theoretical results and the superiority over SOTA DAR methods., Comment: Accepted to ECCV 2024 (oral)

Two-dimensional materials (2DMs) have been widely investigated because of their potential for heterogeneous integration with modern electronics. However, several major challenges remain, such as the deposition of high-quality dielectrics on 2DMs and the tuning of the 2DM doping levels. Here, we report a scalable plasma-enhanced atomic layer deposition (PEALD) process for direct deposition of a nonstoichiometric aluminum oxide (AlOX) dielectric, overcoming the damage issues associated with conventional methods. Furthermore, we control the thickness of the dielectric layer to systematically tune the doping level of 2DMs. The experimental results demonstrate successful deposition without detectable damage, as confirmed by Raman spectroscopy and electrical measurements. Our method enables tuning of the Dirac and threshold voltages of back-gated graphene and MoS${_2}$ field-effect transistors (FETs), respectively, while also increasing the charge carrier mobility in both device types. We further demonstrate the method in top-gated MoS${_2}$ FETs with double-stack dielectric layers (AlOX+Al${_2}$O${_3}$), achieving critical breakdown field strengths of 7 MV/cm and improved mobility compared with the back gate configuration. In summary, we present a PEALD process that offers a scalable and low-damage solution for dielectric deposition on 2DMs, opening new possibilities for precise tuning of device characteristics in heterogeneous electronic circuits., Comment: 28 pages

Expected free energy (EFE) is a central quantity in active inference which has recently gained popularity due to its intuitive decomposition of the expected value of control into a pragmatic and an epistemic component. While numerous conjectures have been made to justify EFE as a decision making objective function, the most widely accepted is still its intuitiveness and resemblance to variational free energy in approximate Bayesian inference. In this work, we take a bottom up approach and ask: taking EFE as given, what's the resulting agent's optimality gap compared with a reward-driven reinforcement learning (RL) agent, which is well understood? By casting EFE under a particular class of belief MDP and using analysis tools from RL theory, we show that EFE approximates the Bayes optimal RL policy via information value. We discuss the implications for objective specification of active inference agents.

Quantum vortices are fundamentally important for properties of superconductors. In conventional type-II superconductor they determine the magnetic response of the system and tend to form regular lattices. UTe$_2$ is a recently discovered heavy fermion superconductor exhibiting many anomalous macroscopic behaviors. However, the question whether it has a multicomponent order parameter remains open. Here, we study magnetic properties of UTe$_2$ by employing scanning superconducting quantum interference device microscopy. We find vortex behavior which is very different from that in ordinary superconductors. We imaged vortices generated by cooling in magnetic field applied along different crystalline directions. While a small out-of-plane magnetic field produces typical isolated vortices, higher field generates vortex stripe patterns which evolve with vortex density. The stripes form at different locations and along different directions in the surface plane when the vortices are crystalized along the crystalline b or c axes. The behavior is reproduced by our simulation based on an anisotropic two-component order parameter. This study shows that UTe$_2$ has a nontrivial disparity of multiple length scales, placing constraints on multicomponent superconductivity. The tendency of vortex stripe formation and their control by external field may be useful in fluxonics applications.

Diffusion-based text-to-image generation models have significantly advanced the field of art content synthesis. However, current portrait stylization methods generally require either model fine-tuning based on examples or the employment of DDIM Inversion to revert images to noise space, both of which substantially decelerate the image generation process. To overcome these limitations, this paper presents an inversion-free portrait stylization framework based on diffusion models that accomplishes content and style feature fusion in merely four sampling steps. We observed that Latent Consistency Models employing consistency distillation can effectively extract representative Consistency Features from noisy images. To blend the Consistency Features extracted from both content and style images, we introduce a Style Enhancement Attention Control technique that meticulously merges content and style features within the attention space of the target image. Moreover, we propose a feature merging strategy to amalgamate redundant features in Consistency Features, thereby reducing the computational load of attention control. Extensive experiments have validated the effectiveness of our proposed framework in enhancing stylization efficiency and fidelity. The code is available at \url{https://github.com/liujin112/ZePo}., Comment: Accepted by ACM MM 2024

A muon collider represents a promising candidate for the next generation of particle physics experiments after the expected end of LHC operations in the early 2040s. Rare or hard-to-detect processes at the LHC, such as the production of multiple gauge bosons, become accessible at a TeV muon collider. We present here the prospects of detecting quantum entanglement and the violation of Bell inequalities in H to ZZ to 4l events at a potential future muon collider. We show that the spin density matrix of the Z boson pairs can be reconstructed using the kinematics of the charged leptons from the Z boson decays. Once the density matrix is determined, it is straightforward to obtain the expectation values of various Bell operators and test the quantum entanglement between the Z boson pair. Through a detailed study based on Monte-Carlo simulation, we show that the generalized CGLMP inequality can be maximally violated, and testing Bell inequalities could be established with high significance., Comment: 11 pages, 3 figures

The unconstrained binary quadratic programming (UBQP) problem is a class of problems of significant importance in many practical applications, such as in combinatorial optimization, circuit design, and other fields. The positive semidefinite penalty (PSDP) method originated from research on semidefinite relaxation, where the introduction of an exact penalty function improves the efficiency and accuracy of problem solving. In this paper, we propose a vectorized PSDP method for solving the UBQP problem, which optimizes computational efficiency by vectorizing matrix variables within a PSDP framework. Algorithmic enhancements in penalty updating and initialization are implemented, along with the introduction of two algorithms that integrate the proximal point algorithm and the projection alternating BB method for subproblem resolution. Properties of the penalty function and algorithm convergence are analyzed. Numerical experiments show the superior performance of the method in providing high-quality solutions and satisfactory solution times compared to the semidefinite relaxation method and other established methods., Comment: 19 pages, 2 figures

The remarkable multimodal capabilities and interactive experience of GPT-4o underscore their necessity in practical applications, yet open-source models rarely excel in both areas. In this paper, we introduce VITA, the first-ever open-source Multimodal Large Language Model (MLLM) adept at simultaneous processing and analysis of Video, Image, Text, and Audio modalities, and meanwhile has an advanced multimodal interactive experience. Starting from Mixtral 8x7B as a language foundation, we expand its Chinese vocabulary followed by bilingual instruction tuning. We further endow the language model with visual and audio capabilities through two-stage multi-task learning of multimodal alignment and instruction tuning. VITA demonstrates robust foundational capabilities of multilingual, vision, and audio understanding, as evidenced by its strong performance across a range of both unimodal and multimodal benchmarks. Beyond foundational capabilities, we have made considerable progress in enhancing the natural multimodal human-computer interaction experience. To the best of our knowledge, we are the first to exploit non-awakening interaction and audio interrupt in MLLM. VITA is the first step for the open-source community to explore the seamless integration of multimodal understanding and interaction. While there is still lots of work to be done on VITA to get close to close-source counterparts, we hope that its role as a pioneer can serve as a cornerstone for subsequent research. Project Page: https://vita-home.github.io., Comment: Project Page: https://vita-home.github.io

Multi-objective evolutionary algorithms (MOEAs) have emerged as powerful tools for solving complex optimization problems characterized by multiple, often conflicting, objectives. While advancements have been made in computational efficiency as well as diversity and convergence of solutions, a critical challenge persists: the internal evolutionary mechanisms are opaque to human users. Drawing upon the successes of explainable AI in explaining complex algorithms and models, we argue that the need to understand the underlying evolutionary operators and population dynamics within MOEAs aligns well with a visual analytics paradigm. This paper introduces ParetoTracker, a visual analytics framework designed to support the comprehension and inspection of population dynamics in the evolutionary processes of MOEAs. Informed by preliminary literature review and expert interviews, the framework establishes a multi-level analysis scheme, which caters to user engagement and exploration ranging from examining overall trends in performance metrics to conducting fine-grained inspections of evolutionary operations. In contrast to conventional practices that require manual plotting of solutions for each generation, ParetoTracker facilitates the examination of temporal trends and dynamics across consecutive generations in an integrated visual interface. The effectiveness of the framework is demonstrated through case studies and expert interviews focused on widely adopted benchmark optimization problems., Comment: Accepted by IEEE VIS 2024 (will appear in IEEE TVCG)

Large language models (LLMs) can be abused at scale to create non-factual content and spread disinformation. Detecting LLM-generated content is essential to mitigate these risks, but current classifiers often fail to generalize in open-world contexts. Prior work shows that LLMs tend to rewrite LLM-generated content less frequently, which can be used for detection and naturally generalizes to unforeseen data. However, we find that the rewriting edit distance between human and LLM content can be indistinguishable across domains, leading to detection failures. We propose training an LLM to rewrite input text, producing minimal edits for LLM-generated content and more edits for human-written text, deriving a distinguishable and generalizable edit distance difference across different domains. Experiments on text from 21 independent domains and three popular LLMs (e.g., GPT-4o, Gemini, and Llama-3) show that our classifier outperforms the state-of-the-art zero-shot classifier by up to 20.6% on AUROC score and the rewriting classifier by 9.2% on F1 score. Our work suggests that LLM can effectively detect machine-generated text if they are trained properly.

The expanding applications of legged robots require their mastery of versatile motion skills. Correspondingly, researchers must address the challenge of integrating multiple diverse motion skills into controllers. While existing reinforcement learning (RL)-based approaches have achieved notable success in multi-skill integration for legged robots, these methods often require intricate reward engineering or are restricted to integrating a predefined set of motion skills constrained by specific task objectives, resulting in limited flexibility. In this work, we introduce a flexible multi-skill integration framework named Controllable Skills Integration (CSI). CSI enables the integration of a diverse set of motion skills with varying styles into a single policy without the need for complex reward tuning. Furthermore, in a hierarchical control manner, the trained low-level policy can be coupled with a high-level Natural Language Inference (NLI) module to enable preliminary language-directed skill control. Our experiments demonstrate that CSI can flexibly integrate a diverse array of motion skills more comprehensively and facilitate the transitions between different skills. Additionally, CSI exhibits good scalability as the number of motion skills to be integrated increases significantly.

High-redshift radio(-loud) galaxies (H$z$RGs) are massive galaxies with powerful radio-loud active galactic nuclei (AGNs) and serve as beacons for protocluster identification. However, the interplay between H$z$RGs and the large-scale environment remains unclear. To understand the connection between H$z$RGs and the surrounding obscured star formation, we investigated the overdensity and spatial distribution of submillimeter-bright galaxies (SMGs) in the field of 4C\,23.56, a well-known H$z$RG at $z=2.48$. We used SCUBA-2 data ($\sigma\,{\sim}\,0.6$\,mJy) to estimate the $850\,{\rm \mu m}$ source number counts and examine the radial and azimuthal overdensities of the $850\,{\rm \mu m}$ sources in the vicinity of the H$z$RG. The angular distribution of SMGs is inhomogeneous around the H$z$RG 4C\,23.56, with fewer sources oriented along the radio jet. We also find a significant overdensity of bright SMGs (${\rm S}_{850\rm\,\mu m}\geq5\,$mJy). Faint and bright SMGs exhibit different spatial distributions. The former are concentrated in the core region, while the latter prefer the outskirts of the H$z$RG field. High-resolution observations show that the seven brightest SMGs in our sample are intrinsically bright, suggesting that the overdensity of bright SMGs is less likely due to the source multiplicity., Comment: 19 pages, 17 figures, 5 tables, accepted to A&A

Online job advertisements on various job portals or websites have become the most popular way for people to find potential career opportunities nowadays. However, the majority of these job sites are limited to offering fundamental filters such as job titles, keywords, and compensation ranges. This often poses a challenge for job seekers in efficiently identifying relevant job advertisements that align with their unique skill sets amidst a vast sea of listings. Thus, we propose well-coordinated visualizations to provide job seekers with three levels of details of job information: a skill-job overview visualizes skill sets, employment posts as well as relationships between them with a hierarchical visualization design; a post exploration view leverages an augmented radar-chart glyph to represent job posts and further facilitates users' swift comprehension of the pertinent skills necessitated by respective positions; a post detail view lists the specifics of selected job posts for profound analysis and comparison. By using a real-world recruitment advertisement dataset collected from 51Job, one of the largest job websites in China, we conducted two case studies and user interviews to evaluate JobViz. The results demonstrated the usefulness and effectiveness of our approach.

The functionality of an entity frequently necessitates the support of a group situated in another layer of the system. To unravel the profound impact of such group support on a system's resilience against cascading failures, we devise a framework comprising a double-layer interdependent hypergraph system, wherein nodes are capable of receiving support via hyperedges. Our central hypothesis posits that the failure may transcend to another layer when all support groups of each dependent node fail, thereby initiating a potentially iterative cascade across layers. Through rigorous analytical methods, we derive the critical threshold for the initial node survival probability that marks the second-order phase transition point. A salient discovery is that as the prevalence of dependent nodes escalates, the system dynamics shift from a second-order to a first-order phase transition. Notably, irrespective of the collapse pattern, systems characterized by scale-free hyperdegree distributions within both hypergraph layers consistently demonstrate superior robustness compared to those adhering to Poisson hyperdegree distributions. In summary, our research underscores the paramount significance of group support mechanisms and intricate network topologies in determining the resilience of interconnected systems against the propagation of cascading failures. By exploring the interplay between these factors, we have gained insights into how systems can be designed or optimized to mitigate the risk of widespread disruptions, ensuring their continued functionality and stability in the face of adverse events., Comment: 18 pages, 5 figures

Drawing on the intricate structures of the brain, Spiking Neural Networks (SNNs) emerge as a transformative development in artificial intelligence, closely emulating the complex dynamics of biological neural networks. While SNNs show promising efficiency on specialized sparse-computational hardware, their practical training often relies on conventional GPUs. This reliance frequently leads to extended computation times when contrasted with traditional Artificial Neural Networks (ANNs), presenting significant hurdles for advancing SNN research. To navigate this challenge, we present a novel temporal fusion method, specifically designed to expedite the propagation dynamics of SNNs on GPU platforms, which serves as an enhancement to the current significant approaches for handling deep learning tasks with SNNs. This method underwent thorough validation through extensive experiments in both authentic training scenarios and idealized conditions, confirming its efficacy and adaptability for single and multi-GPU systems. Benchmarked against various existing SNN libraries/implementations, our method achieved accelerations ranging from $5\times$ to $40\times$ on NVIDIA A100 GPUs. Publicly available experimental codes can be found at https://github.com/EMI-Group/snn-temporal-fusion., Comment: International Conference on Artificial Neural Networks (ICANN) 2024

Accurate diagnosis of depression is crucial for timely implementation of optimal treatments, preventing complications and reducing the risk of suicide. Traditional methods rely on self-report questionnaires and clinical assessment, lacking objective biomarkers. Combining fMRI with artificial intelligence can enhance depression diagnosis by integrating neuroimaging indicators. However, the specificity of fMRI acquisition for depression often results in unbalanced and small datasets, challenging the sensitivity and accuracy of classification models. In this study, we propose the Spatio-Temporal Aggregation Network (STANet) for diagnosing depression by integrating CNN and RNN to capture both temporal and spatial features of brain activity. STANet comprises the following steps:(1) Aggregate spatio-temporal information via ICA. (2) Utilize multi-scale deep convolution to capture detailed features. (3) Balance data using the SMOTE to generate new samples for minority classes. (4) Employ the AFGRU classifier, which combines Fourier transformation with GRU, to capture long-term dependencies, with an adaptive weight assignment mechanism to enhance model generalization. The experimental results demonstrate that STANet achieves superior depression diagnostic performance with 82.38% accuracy and a 90.72% AUC. The STFA module enhances classification by capturing deeper features at multiple scales. The AFGRU classifier, with adaptive weights and stacked GRU, attains higher accuracy and AUC. SMOTE outperforms other oversampling methods. Additionally, spatio-temporal aggregated features achieve better performance compared to using only temporal or spatial features. STANet outperforms traditional or deep learning classifiers, and functional connectivity-based classifiers, as demonstrated by ten-fold cross-validation.

Literature research, vital for scientific advancement, is overwhelmed by the vast ocean of available information. Addressing this, we propose an automated review generation method based on Large Language Models (LLMs) to streamline literature processing and reduce cognitive load. In case study on propane dehydrogenation (PDH) catalysts, our method swiftly generated comprehensive reviews from 343 articles, averaging seconds per article per LLM account. Extended analysis of 1041 articles provided deep insights into catalysts' composition, structure, and performance. Recognizing LLMs' hallucinations, we employed a multi-layered quality control strategy, ensuring our method's reliability and effective hallucination mitigation. Expert verification confirms the accuracy and citation integrity of generated reviews, demonstrating LLM hallucination risks reduced to below 0.5% with over 95% confidence. Released Windows application enables one-click review generation, aiding researchers in tracking advancements and recommending literature. This approach showcases LLMs' role in enhancing scientific research productivity and sets the stage for further exploration., Comment: 16 pages, 3 figures, 3 tables

It has been supposed that the interplay of elasticity and activity plays a key role in triggering the non-equilibrium behaviors in biological systems. However, the experimental model system is missing to investigate the spatiotemporally dynamical phenomena. Here, a model system of an active chain, where active eccentric-disks are linked by a spring, is designed to study the interplay of activity, elasticity, and friction. Individual active chain exhibits longitudinal and transverse motion, however, it starts to self-rotate when pinning one end, and self-beats when clamping one end. Additionally, our eccentric-disk model can qualitatively reproduce such behaviors and explain the unusual self-rotation of the first disk around its geometric center. Further, the structure and dynamics of long chains were studied via simulations without steric interactions. It was found that hairpin conformation emerges in free motion, while in the constrained motions, the rotational and beating frequencies scale with the flexure number (the ratio of self-propelling force to bending rigidity), ~4/3. Scaling analysis suggests that it results from the balance between activity and energy dissipation. Our findings show that topological constraints play a vital role in non-equilibrium synergy behavior.

The fidelity of entangling operations is a key figure of merit in quantum information processing, especially in the context of quantum error correction. High-fidelity entangling gates in neutral atoms have seen remarkable advancement recently. A full understanding of error sources and their respective contributions to gate infidelity will enable the prediction of fundamental limits on quantum gates in neutral atom platforms with realistic experimental constraints. In this work, we implement the time-optimal Rydberg CZ gate, design a circuit to benchmark its fidelity, and achieve a fidelity, averaged over symmetric input states, of 0.9971(5), setting a new state-of-the-art for neutral atoms. The remaining infidelity is explained by an ab initio error model, consistent with our experimental results over a range of gate speeds, with varying contributions from different error sources. Further, we develop a linear response formalism to efficiently predict infidelity from laser noise with non-trivial power spectral densities and derive scaling laws of infidelity with gate speed. Besides its capability of predicting gate fidelity, we also utilize the linear response formalism to compare and optimize gate protocols, to learn laser frequency noise, and to study the noise response for quantum simulation tasks. Finally, we predict that a CZ gate fidelity of ${\gtrsim} 0.999$ is feasible with realistic experimental upgrades., Comment: RBST and XS contributed equally to this work

On meshes with the maximum angle condition violated, the standard conforming, nonconforming, and discontinuous Galerkin finite elements do not converge to the true solution when the mesh size goes to zero. It is shown that one type of weak Galerkin finite element method converges on triangular and tetrahedral meshes violating the maximum angle condition, i.e., on arbitrary meshes. Numerical tests confirm the theory.

In underwater image enhancement (UIE), convolutional neural networks (CNN) have inherent limitations in modeling long-range dependencies and are less effective in recovering global features. While Transformers excel at modeling long-range dependencies, their quadratic computational complexity with increasing image resolution presents significant efficiency challenges. Additionally, most supervised learning methods lack effective physical model constraint, which can lead to insufficient realism and overfitting in generated images. To address these issues, we propose a physical model constraint-based underwater image enhancement framework, Mamba-UIE. Specifically, we decompose the input image into four components: underwater scene radiance, direct transmission map, backscatter transmission map, and global background light. These components are reassembled according to the revised underwater image formation model, and the reconstruction consistency constraint is applied between the reconstructed image and the original image, thereby achieving effective physical constraint on the underwater image enhancement process. To tackle the quadratic computational complexity of Transformers when handling long sequences, we introduce the Mamba-UIE network based on linear complexity state space models. By incorporating the Mamba in Convolution block, long-range dependencies are modeled at both the channel and spatial levels, while the CNN backbone is retained to recover local features and details. Extensive experiments on three public datasets demonstrate that our proposed Mamba-UIE outperforms existing state-of-the-art methods, achieving a PSNR of 27.13 and an SSIM of 0.93 on the UIEB dataset. Our method is available at https://github.com/zhangsong1213/Mamba-UIE.

We use $P_t$ and $C_t$ to denote a path and a cycle on $t$ vertices, respectively. A {\em bull} is a graph consisting of a triangle with two disjoint pendant edges, a {\em hammer} is a graph obtained by identifying an endvertex of a $P_3$ with a vertex of a triangle. A class ${\cal F}$ is $\chi$-bounded if there is a function $f$ such that $\chi(G)\leq f(\omega(G))$ for all induced subgraphs $G$ of a graph in ${\cal F}$. A class ${\cal C}$ of graphs is {\em Pollyanna} (resp. {\em linear-Pollyanna}) if ${\cal C}\cap {\cal F}$ is polynomially (resp. linear-polynomially) $\chi$-bounded for every $\chi$-bounded class ${\cal F}$ of graphs. Chudnovsky {\em et al} \cite{CCDO2023} showed that both the classes of bull-free graphs and hammer-free graphs are Pollyannas. Let $G$ be a connected graph with no clique cutsets and no universal cliques. In this paper, we show that $G$ is $(C_4$, hammer)-free if and only if it has girth at least 5, and $G$ is $(C_4$, bull)-free if and only if it is a clique blowup of some graph of girth at least 5. As a consequence, we show that both the classes of $(C_4$, bull)-free graphs and $(C_4$, hammer)-free graphs are linear-Pollyannas., Comment: 11