Search

Emergent Large Language Models (LLMs) use their extraordinary performance and powerful deduction capacity to discern from traditional language models. However, the expenses of computational resources and storage for these LLMs are stunning, quantization then arises as a trending conversation. To address accuracy decay caused by quantization, two streams of works in post-training quantization methods stand out. One uses other weights to compensate existing quantization error, while the other transfers the quantization difficulty to other parts in the model. Combining both merits, we introduce Learnable Singular value Increment (LSI) as an advanced solution. LSI uses Singular Value Decomposition to extract singular values of the weights and make them learnable to help weights compensate each other conditioned on activation. Incorporating LSI with existing techniques, we achieve state-of-the-art performance in diverse quantization settings, no matter in weight-only, weight-activation or extremely low bit scenarios. By unleashing the potential of LSI, efficient finetuning on quantized model is no longer a prohibitive problem., Comment: Efficient quantization method

Echocardiography (ECHO) video is widely used for cardiac examination. In clinical, this procedure heavily relies on operator experience, which needs years of training and maybe the assistance of deep learning-based systems for enhanced accuracy and efficiency. However, it is challenging since acquiring sufficient customized data (e.g., abnormal cases) for novice training and deep model development is clinically unrealistic. Hence, controllable ECHO video synthesis is highly desirable. In this paper, we propose a novel diffusion-based framework named HeartBeat towards controllable and high-fidelity ECHO video synthesis. Our highlight is three-fold. First, HeartBeat serves as a unified framework that enables perceiving multimodal conditions simultaneously to guide controllable generation. Second, we factorize the multimodal conditions into local and global ones, with two insertion strategies separately provided fine- and coarse-grained controls in a composable and flexible manner. In this way, users can synthesize ECHO videos that conform to their mental imagery by combining multimodal control signals. Third, we propose to decouple the visual concepts and temporal dynamics learning using a two-stage training scheme for simplifying the model training. One more interesting thing is that HeartBeat can easily generalize to mask-guided cardiac MRI synthesis in a few shots, showcasing its scalability to broader applications. Extensive experiments on two public datasets show the efficacy of the proposed HeartBeat., Comment: Accepted by MICCAI 2024

Deep learning-based brain tumor segmentation (BTS) models for multi-modal MRI images have seen significant advancements in recent years. However, a common problem in practice is the unavailability of some modalities due to varying scanning protocols and patient conditions, making segmentation from incomplete MRI modalities a challenging issue. Previous methods have attempted to address this by fusing accessible multi-modal features, leveraging attention mechanisms, and synthesizing missing modalities using generative models. However, these methods ignore the intrinsic problems of medical image segmentation, such as the limited availability of training samples, particularly for cases with tumors. Furthermore, these methods require training and deploying a specific model for each subset of missing modalities. To address these issues, we propose a novel approach that enhances the BTS model from two perspectives. Firstly, we introduce a pre-training stage that generates a diverse pre-training dataset covering a wide range of different combinations of tumor shapes and brain anatomy. Secondly, we propose a post-training stage that enables the model to reconstruct missing modalities in the prediction results when only partial modalities are available. To achieve the pre-training stage, we conceptually decouple the MRI image into two parts: `anatomy' and `tumor'. We pre-train the BTS model using synthesized data generated from the anatomy and tumor parts across different training samples. ... Extensive experiments demonstrate that our proposed method significantly improves the performance over the baseline and achieves new state-of-the-art results on three brain tumor segmentation datasets: BRATS2020, BRATS2018, and BRATS2015.

Surgical scene perception via videos are critical for advancing robotic surgery, telesurgery, and AI-assisted surgery, particularly in ophthalmology. However, the scarcity of diverse and richly annotated video datasets has hindered the development of intelligent systems for surgical workflow analysis. Existing datasets for surgical workflow analysis, which typically face challenges such as small scale, a lack of diversity in surgery and phase categories, and the absence of time-localized annotations, limit the requirements for action understanding and model generalization validation in complex and diverse real-world surgical scenarios. To address this gap, we introduce OphNet, a large-scale, expert-annotated video benchmark for ophthalmic surgical workflow understanding. OphNet features: 1) A diverse collection of 2,278 surgical videos spanning 66 types of cataract, glaucoma, and corneal surgeries, with detailed annotations for 102 unique surgical phases and 150 granular operations; 2) It offers sequential and hierarchical annotations for each surgery, phase, and operation, enabling comprehensive understanding and improved interpretability; 3) Moreover, OphNet provides time-localized annotations, facilitating temporal localization and prediction tasks within surgical workflows. With approximately 205 hours of surgical videos, OphNet is about 20 times larger than the largest existing surgical workflow analysis benchmark. Our dataset and code have been made available at: \url{https://github.com/minghu0830/OphNet-benchmark}., Comment: Version 1

Utilizing an equivparticle model with both linear confinement and leading-order perturbative interactions, we obtain systematically the properties of strangelets and nonstrange quark matter ($ud$QM) nuggets at various baryon ($A$) and charge ($Z$) numbers, where the detailed single-quark-energy levels are fixed by solving Dirac equations in mean-field approximation (MFA). We then examine the structures of compact dwarfs made of light strangelets or $ud$QM nuggets forming body-centered cubic lattices in a uniform electron background. Despite the strangelets and $ud$QM nuggets generally become more stable at larger $A$, the compact dwarfs are still stable since the fusion reactions between those objects do not take place in the presence of a Coulomb barrier, which is similar to the cases of light nuclei in normal white dwarfs. If $ud$QM dwarfs or strangelet dwarfs are covered with normal matter, their masses and radii become larger but do not exceed those of ordinary white dwarfs. Finally, we investigate the radial oscillation frequencies of $ud$QM dwarfs and strangelet dwarfs, and find that their frequencies are typically higher than traditional white dwarfs. The stability of compact dwarfs are then analysised by examining radial oscillation frequencies of the fundamental mode, where compact dwarfs covered by normal matter are still stable.

3D environment recognition is essential for autonomous driving systems, as autonomous vehicles require a comprehensive understanding of surrounding scenes. Recently, the predominant approach to define this real-life problem is through 3D occupancy prediction. It attempts to predict the occupancy states and semantic labels for all voxels in 3D space, which enhances the perception capability. Birds-Eye-View(BEV)-based perception has achieved the SOTA performance for this task. Nonetheless, this architecture fails to represent various scales of BEV features. In this paper, inspired by the success of UNet in semantic segmentation tasks, we introduce a novel UNet-like Multi-scale Occupancy Head module to relieve this issue. Furthermore, we propose the class-balancing loss to compensate for rare classes in the dataset. The experimental results on nuScenes 3D occupancy challenge dataset show the superiority of our proposed approach over baseline and SOTA methods., Comment: 5 pages, 3 figures, accepted by IEEE CAI 2024

We study the properties of strangelets at finite temperature $T$, employing an equivparticle model that incorporates both linear confinement and leading-order perturbative interactions with density-dependent quark masses. The shell effects are analyzed by solving the Dirac equations for quarks within the mean-field approximation. As temperature increases, these effects weaken due to the occupation probability of single-particle levels being governed by the Fermi-Dirac statistics, a phenomenon known as shell dampening. Surprisingly, the surface tension, derived from a liquid-drop formula, does not decrease with temperature but instead rises until it peaks at $T \approx 20-40$ MeV. At this temperature, shell corrections become negligible, and the formula provides a reasonable approximation for the free energy per baryon of strangelets. However, the curvature term decreases with $T$ despite the presence of shell effects. The neutron and proton emission rates are determined microscopically by the external nucleon gas densities that are in equilibrium with strangelets. These emission rate generally increases with $T$ for stable strangelets, but decrease for those that are unstable to nucleon emission at $T$ = 0. The other properties of $\beta$-stable strangelets obtained with various parameter sets are presented as well. The results indicated in this work are useful for understanding the products of binary compact star mergers and heavy-ion collisions., Comment: Contributions to the conference proceedings of QCS2023

In recent years, numerous real-time stereo matching methods have been introduced, but they often lack accuracy. These methods attempt to improve accuracy by introducing new modules or integrating traditional methods. However, the improvements are only modest. In this paper, we propose a novel strategy by incorporating knowledge distillation and model pruning to overcome the inherent trade-off between speed and accuracy. As a result, we obtained a model that maintains real-time performance while delivering high accuracy on edge devices. Our proposed method involves three key steps. Firstly, we review state-of-the-art methods and design our lightweight model by removing redundant modules from those efficient models through a comparison of their contributions. Next, we leverage the efficient model as the teacher to distill knowledge into the lightweight model. Finally, we systematically prune the lightweight model to obtain the final model. Through extensive experiments conducted on two widely-used benchmarks, Sceneflow and KITTI, we perform ablation studies to analyze the effectiveness of each module and present our state-of-the-art results., Comment: International Conference on Robotics and Automation (ICRA) 2024

Most existing methods often rely on complex models to predict scene depth with high accuracy, resulting in slow inference that is not conducive to deployment. To better balance precision and speed, we first designed SmallDepth based on sparsity. Second, to enhance the feature representation ability of SmallDepth during training under the condition of equal complexity during inference, we propose an equivalent transformation module(ETM). Third, to improve the ability of each layer in the case of a fixed SmallDepth to perceive different context information and improve the robustness of SmallDepth to the left-right direction and illumination changes, we propose pyramid loss. Fourth, to further improve the accuracy of SmallDepth, we utilized the proposed function approximation loss (APX) to transfer knowledge in the pretrained HQDecv2, obtained by optimizing the previous HQDec to address grid artifacts in some regions, to SmallDepth. Extensive experiments demonstrate that each proposed component improves the precision of SmallDepth without changing the complexity of SmallDepth during inference, and the developed approach achieves state-of-the-art results on KITTI at an inference speed of more than 500 frames per second and with approximately 2 M parameters. The code and models will be publicly available at https://github.com/fwucas/FA-Depth.

Molecular adsorption is pivotal in device fabrication and material synthesis for quantum technology. However, elucidating the behavior of physisorption poses technical challenges. Here graphene with ultrahigh sensitivity was utilized to detect ozone adsorption at cryogenic temperatures. Significant hole doping observed in graphene indicates a strong interaction between ozone and graphene. Interestingly, the adsorption exhibits asymmetry with positive and negative gate voltages. The strong affinity of ozone provides a tool to modulate materials and devices, while the gate tunability of adsorption offers new insights into construction and manipulation of oxide quantum materials.

By considering baryons as clusters of three quarks, we extend the Nambu-Jona-Lasinio (NJL) model to describe baryonic matter, quark matter, and their transitions in a unified manner, where the Dirac sea and spontaneous chiral symmetry breaking are considered. In particular, a density-dependent structural function $\alpha_S$ is introduced to modulate the four(six)-point interaction strengths to reproduce the baryon masses in vacuum and medium. Vector interactions are considered with the exchange of $\omega$ and $\rho$ mesons, where the density-dependent coupling constants are fixed by reproducing nuclear matter properties as well as the $\Lambda$-hyperon potential depth in nuclear medium. As density increases, quarks will emerge as quasi-free particles and coexist with baryons. This phase is interpreted as quarkyonic matter, where quarks are restricted to the lowest energy states in the presence of baryons, i.e., quarks are still confined. Similar to the treatment of $\alpha$ clustering in nuclear medium, a Pauli blocking term is added to baryon masses so that baryons eventually become unbound in the presence of a quark Fermi Sea. Then at large enough densities baryons vanish and a Mott transition takes place, we consider such a transition as deconfinement phase transition. Depending on the strengths of Pauli blocking term and quark-vector meson couplings, both first-order and continues phase transitions are observed for quarkyonic, chiral, and deconfinement phase transitions. The corresponding compact star structures are then investigated and confronted with various astrophysical constraints.

Recent developments in neural rendering techniques have greatly enhanced the rendering of photo-realistic 3D scenes across both academic and commercial fields. The latest method, known as 3D Gaussian Splatting (3D-GS), has set new benchmarks for rendering quality and speed. Nevertheless, the limitations of 3D-GS become pronounced in synthesizing new viewpoints, especially for views that greatly deviate from those seen during training. Additionally, issues such as dilation and aliasing arise when zooming in or out. These challenges can all be traced back to a single underlying issue: insufficient sampling. In our paper, we present a bootstrapping method that significantly addresses this problem. This approach employs a diffusion model to enhance the rendering of novel views using trained 3D-GS, thereby streamlining the training process. Our results indicate that bootstrapping effectively reduces artifacts, as well as clear enhancements on the evaluation metrics. Furthermore, we show that our method is versatile and can be easily integrated, allowing various 3D reconstruction projects to benefit from our approach.

In the AI-for-science era, scientific computing scenarios such as concurrent learning and high-throughput computing demand a new generation of infrastructure that supports scalable computing resources and automated workflow management on both cloud and high-performance supercomputers. Here we introduce Dflow, an open-source Python toolkit designed for scientists to construct workflows with simple programming interfaces. It enables complex process control and task scheduling across a distributed, heterogeneous infrastructure, leveraging containers and Kubernetes for flexibility. Dflow is highly observable and can scale to thousands of concurrent nodes per workflow, enhancing the efficiency of complex scientific computing tasks. The basic unit in Dflow, known as an Operation (OP), is reusable and independent of the underlying infrastructure or context. Dozens of workflow projects have been developed based on Dflow, spanning a wide range of projects. We anticipate that the reusability of Dflow and its components will encourage more scientists to publish their workflows and OP components. These components, in turn, can be adapted and reused in various contexts, fostering greater collaboration and innovation in the scientific community.

Image denoising is a fundamental task in computer vision. While prevailing deep learning-based supervised and self-supervised methods have excelled in eliminating in-distribution noise, their susceptibility to out-of-distribution (OOD) noise remains a significant challenge. The recent emergence of contrastive language-image pre-training (CLIP) model has showcased exceptional capabilities in open-world image recognition and segmentation. Yet, the potential for leveraging CLIP to enhance the robustness of low-level tasks remains largely unexplored. This paper uncovers that certain dense features extracted from the frozen ResNet image encoder of CLIP exhibit distortion-invariant and content-related properties, which are highly desirable for generalizable denoising. Leveraging these properties, we devise an asymmetrical encoder-decoder denoising network, which incorporates dense features including the noisy image and its multi-scale features from the frozen ResNet encoder of CLIP into a learnable image decoder to achieve generalizable denoising. The progressive feature augmentation strategy is further proposed to mitigate feature overfitting and improve the robustness of the learnable decoder. Extensive experiments and comparisons conducted across diverse OOD noises, including synthetic noise, real-world sRGB noise, and low-dose CT image noise, demonstrate the superior generalization ability of our method., Comment: Accepted by CVPR2024

We calculate the tensor bispectrum mediated by an excited scalar field during inflation and find that the bispectrum peaks in the squeezed configuration, which is different from that of gravitational waves induced by enhanced curvature perturbations re-entering the horizon in the radiation-dominated era. Measuring the bispectrum provides a promising way to distinguish the stochastic gravitational-wave background generated during inflation from that generated after inflation., Comment: 6 pages, 4 figures

Geometric knowledge has been shown to be beneficial for the stereo matching task. However, prior attempts to integrate geometric insights into stereo matching algorithms have largely focused on geometric knowledge from single images while crucial cross-view factors such as occlusion and matching uniqueness have been overlooked. To address this gap, we propose a novel Intra-view and Cross-view Geometric knowledge learning Network (ICGNet), specifically crafted to assimilate both intra-view and cross-view geometric knowledge. ICGNet harnesses the power of interest points to serve as a channel for intra-view geometric understanding. Simultaneously, it employs the correspondences among these points to capture cross-view geometric relationships. This dual incorporation empowers the proposed ICGNet to leverage both intra-view and cross-view geometric knowledge in its learning process, substantially improving its ability to estimate disparities. Our extensive experiments demonstrate the superiority of the ICGNet over contemporary leading models., Comment: Accepted to CVPR2024

We study the magnetized strangelets in the baryon density-dependent quark mass model, including the effects of both confinement and lead-order perturbation interactions. The properties of magnetized strangelets are investigated under the the field strength 2*10^17 G, where the anisotropy caused by the strong magnetic field is insignificant can be treated approximately as an isotropic system. The consideration of anomalous magnetic moments in the energy spectrum naturally solves the difficulty of infrared divergence encountered in integrating the density of states. The Coulomb interaction is accounted for a self-consistent treatment. The energy per baryon, mechanically stable radius, strangeness and electric charge of magnetized strangelets are presented, where their dependence on the field strength and parameter of confinement and perturbation are investigated.

Motivated by recent advancements in theoretical and experimental studies on the high-energy excitations, we theoretically explore the quantum phase transition and composite dynamics of the antiferromagnetic trimer chains in a magnetic field using the exact diagonalization, density matrix renormalization group, time-dependent variational principle and cluster perturbation theory. We measure the entanglement entropy to uncover the phase diagram, encompassing the XY-I, $1/3$ magnetization plateau, XY-II and ferromagnetic phases. Both critical XY-I and XY-II phases are both described by the conformal field theory with the central charge $c \simeq 1$. By analyzing the dynamical structure factor, we elucidate the distinct features of spin dynamics across different phases. In the regime of weak intertrimer interaction, we identify the intermediate-energy and high-energy modes in the XY-I and $1/3$ magnetization plateau phases as the internal trimer excitations, corresponding to the propagation of doublon and quarton, respectively. Notably, the application of a magnetic field splits the high-energy spectra into two branches labeled as the upper quarton and lower quarton. Furthermore, we also explore the spin dynamics of a trimerized model closely related to the quantum magnet \ce{Na_2Cu_3Ge_4O_12}, and discuss the possibility of the quarton Bose-Einstein condensation. Our results can be verified in the inelastic neutron scattering experiments and provide deep insights for exploring the high-energy exotic excitations., Comment: 14+7 pages, 16 figures