Search

Radiology report generation aims to automatically generate detailed and coherent descriptive reports alongside radiology images. Previous work mainly focused on refining fine-grained image features or leveraging external knowledge. However, the precise alignment of fine-grained image features with corresponding text descriptions has not been considered. This paper presents a novel method called Fine-grained Image-Text Aligner (FITA) to construct fine-grained alignment for image and text features. It has three novel designs: Image Feature Refiner (IFR), Text Feature Refiner (TFR) and Contrastive Aligner (CA). IFR and TFR aim to learn fine-grained image and text features, respectively. We achieve this by leveraging saliency maps to effectively fuse symptoms with corresponding abnormal visual regions, and by utilizing a meticulously constructed triplet set for training. Finally, CA module aligns fine-grained image and text features using contrastive loss for precise alignment. Results show that our method surpasses existing methods on the widely used benchmark, Comment: 11 pages, 3 figures

This paper introduces a novel benchmark as part of the AIS 2024 Real-Time Image Super-Resolution (RTSR) Challenge, which aims to upscale compressed images from 540p to 4K resolution (4x factor) in real-time on commercial GPUs. For this, we use a diverse test set containing a variety of 4K images ranging from digital art to gaming and photography. The images are compressed using the modern AVIF codec, instead of JPEG. All the proposed methods improve PSNR fidelity over Lanczos interpolation, and process images under 10ms. Out of the 160 participants, 25 teams submitted their code and models. The solutions present novel designs tailored for memory-efficiency and runtime on edge devices. This survey describes the best solutions for real-time SR of compressed high-resolution images., Comment: CVPR 2024, AI for Streaming (AIS) Workshop

This paper reviews the NTIRE 2024 low light image enhancement challenge, highlighting the proposed solutions and results. The aim of this challenge is to discover an effective network design or solution capable of generating brighter, clearer, and visually appealing results when dealing with a variety of conditions, including ultra-high resolution (4K and beyond), non-uniform illumination, backlighting, extreme darkness, and night scenes. A notable total of 428 participants registered for the challenge, with 22 teams ultimately making valid submissions. This paper meticulously evaluates the state-of-the-art advancements in enhancing low-light images, reflecting the significant progress and creativity in this field., Comment: NTIRE 2024 Challenge Report

Ice crystal icing (ICI) in aircraft engines is a major threat to flight safety. Due to the complex thermodynamic and phase-change conditions involved in ICI, rigorous modelling of the accretion process remains limited. The present study proposes a novel modelling approach based on the physically-observed mixed-phase nature of the accretion layers. The mathematical model, which is derived from the enthalpy change after accretion (the enthalpy model), is compared to an existing pure-phase layer model (the three-layer model). Scaling laws and asymptotic solutions are developed for both models. The onset of ice accretion, the icing layer thickness, and solid ice fraction within the layer are determined by a set of non-dimensional parameters including the Peclet number, the Stefan number, the Biot number, the Melt Ratio, and the evaporative rate. Thresholds for freezing and non-freezing conditions are developed. The asymptotic solutions presents good agreement with numerical solutions at low Peclet numbers. Both the asymptotic and numerical solutions show that, when compared to the three-layer model, the enthalpy model presents a thicker icing layer and a thicker water layer above the substrate due to mixed-phased features and modified Stefan conditions. Modelling in terms of the enthalpy poses significant advantages in the development of numerical methods to complex three-dimensional geometrical and flow configurations. These results improve understanding of the accretion process and provide a novel, rigorous mathematical framework for accurate modelling of ICI.

Because of the complicated mechanism of ankle injury, it is very difficult to diagnose ankle fracture in clinic. In order to simplify the process of fracture diagnosis, an automatic diagnosis model of ankle fracture was proposed. Firstly, a tibia-fibula segmentation network is proposed for the joint tibiofibular region of the ankle joint, and the corresponding segmentation dataset is established on the basis of fracture data. Secondly, the image registration method is used to register the bone segmentation mask with the normal bone mask. Finally, a semi-supervised classifier is constructed to make full use of a large number of unlabeled data to classify ankle fractures. Experiments show that the proposed method can segment fractures with fracture lines accurately and has better performance than the general method. At the same time, this method is superior to classification network in several indexes.

With the emergence of pre-trained vision-language models like CLIP, how to adapt them to various downstream classification tasks has garnered significant attention in recent research. The adaptation strategies can be typically categorized into three paradigms: zero-shot adaptation, few-shot adaptation, and the recently-proposed training-free few-shot adaptation. Most existing approaches are tailored for a specific setting and can only cater to one or two of these paradigms. In this paper, we introduce a versatile adaptation approach that can effectively work under all three settings. Specifically, we propose the dual memory networks that comprise dynamic and static memory components. The static memory caches training data knowledge, enabling training-free few-shot adaptation, while the dynamic memory preserves historical test features online during the testing process, allowing for the exploration of additional data insights beyond the training set. This novel capability enhances model performance in the few-shot setting and enables model usability in the absence of training data. The two memory networks employ the same flexible memory interactive strategy, which can operate in a training-free mode and can be further enhanced by incorporating learnable projection layers. Our approach is tested across 11 datasets under the three task settings. Remarkably, in the zero-shot scenario, it outperforms existing methods by over 3\% and even shows superior results against methods utilizing external training data. Additionally, our method exhibits robust performance against natural distribution shifts. Codes are available at \url{https://github.com/YBZh/DMN}., Comment: CVPR2024; Codes are available at \url{https://github.com/YBZh/DMN}

We provide a systematic study on the electronic structure of a series of Ta$_2$O$_5$ polymorphs using standard density functional theory (DFT) calculations as well as the more accurate many-body perturbation theory within the GW approximation. For the defect-free polymorphs, the variation trend of band gap can be microscopically related to the strength of orbital hybridization, and to the macroscopic bulk formation energy. The presence of oxygen vacancies is found to have profound effects on the electronic properties near the Fermi level, notably the reduction of band gap and gap closure (insulator-to-metal transition). Furthermore, depending on the vacancy sites, the band gap of some defective system may be enlarged with comparison to its defect-free counterpart. Such an anomalous behavior originates from the unexpected formation of Ta-Ta bonds., Comment: 32 pages, 9 figures, 3 tables

Benchmarking a high-precision quantum operation is a big challenge for many quantum systems in the presence of various noises as well as control errors. Here we propose an $O(1)$ benchmarking of a dynamically corrected rotation by taking the quantum advantage of a squeezed spin state in a spin-1 Bose-Einstein condensate. Our analytical and numerical results show that tiny rotation infidelity, defined by $1-F$ with $F$ the rotation fidelity, can be calibrated in the order of $1/N^2$ by only several measurements of the rotation error for $N$ atoms in an optimally squeezed spin state. Such an $O(1)$ benchmarking is possible not only in a spin-1 BEC but also in other many-spin or many-qubit systems if a squeezed or entangled state is available., Comment: 10 pages, 8 figures

While supervised learning has achieved remarkable success, obtaining large-scale labeled datasets in biomedical imaging is often impractical due to high costs and the time-consuming annotations required from radiologists. Semi-supervised learning emerges as an effective strategy to overcome this limitation by leveraging useful information from unlabeled datasets. In this paper, we present a novel semi-supervised learning method, Dual-Decoder Consistency via Pseudo-Labels Guided Data Augmentation (DCPA), for medical image segmentation. We devise a consistency regularization to promote consistent representations during the training process. Specifically, we use distinct decoders for student and teacher networks while maintain the same encoder. Moreover, to learn from unlabeled data, we create pseudo-labels generated by the teacher networks and augment the training data with the pseudo-labels. Both techniques contribute to enhancing the performance of the proposed method. The method is evaluated on three representative medical image segmentation datasets. Comprehensive comparisons with state-of-the-art semi-supervised medical image segmentation methods were conducted under typical scenarios, utilizing 10% and 20% labeled data, as well as in the extreme scenario of only 5% labeled data. The experimental results consistently demonstrate the superior performance of our method compared to other methods across the three semi-supervised settings. The source code is publicly available at https://github.com/BinYCn/DCPA.git.

In this study, we conduct a study on magnetic hyperthermia treatment when a vessel is located near the tumor. The holistic framework is established to solve the process of tumor treatment. The interstitial tissue fluid, MNP distribution, temperature profile, and nanofluids are involved in the simulation. The study evaluates the cancer treatment efficacy by cumulative-equivalent-minutes-at-43 centigrade (CEM43), a widely accepted thermal dose. The influence of the nearby blood vessel is investigated, and parameter studies about the distance to the tumor, the width of the blood vessel, and the vessel direction are also conducted. After that, the effects of the fluid-structure interaction of moving vessel boundaries and blood rheology are discussed. The results demonstrate the cooling effect of a nearby blood vessel, and such effect reduces with the augment of the distance between the tumor and blood vessel. The combination of downward gravity and the cool effect from the lower horizontal vessel leads to the best performance with 97.77% ablation in the tumor and 0.87% injury in healthy tissue at distance d = 4 mm, but the cases of the vertical vessel are relatively poor. The vessel width and blood rheology affect the treatment by velocity gradient near the vessel wall. Additionally, the moving boundary almost has no impact on treatment efficacy. The simulation tool has been well-validated and the outcomes can provide a useful reference to magnetic hyperthermia treatment., Comment: arXiv admin note: substantial text overlap with arXiv:2308.12652

A holistic simulation framework is established on magnetic hyperthermia modeling to solve the treatment process of tumor, which is surrounded by a healthy tissue block. The interstitial tissue fluid, MNP distribution, temperature profile, and nanofluids are involved in the simulation. Study evaluates the cancer treatment efficacy by cumulative-equivalent-minutes-at-43 centigrade (CEM43), a widely accepted thermal dose coming from the cell death curve. Results are separated into the conditions of with or without gravity effect in the computational domain, where two baseline case are investigated and compared. An optimal treatment time 46.55 min happens in the baseline case without gravity, but the situation deteriorates with gravity effect where the time for totally killing tumor cells prolongs 36.11% and meanwhile causing 21.32% ablation in healthy tissue. For the cases without gravity, parameter study of Lewis number and Heat source number are conducted and the variation of optimal treatment time are both fitting to the inverse functions. For the case considering the gravity, parameters Buoyancy ratio and Darcy ratio are investigated and their influence on totally killing tumor cells and the injury on healthy tissue are matching with the parabolic functions. The results are beneficial to the prediction of various conditions, and provides useful guide to the magnetic hyperthermia treatment.

Antiferromagnetic (AF) spintronics is merit on ultra-high operator speed and stability in the presence of magnetic field. To fully use the merit, the device should be pure rather than hybrid with ferromagnet or ferrimagnet. For the magnetism in the antiferromagnet is canceled by that of different sublattices, breaking the symmetry in the material can revive the native magnetism, which can be detected by the magnetoresistance (MR) effect. Achieving noticeable MR effect in the pure AF device is diffcult but essential for the AF spintronic applications. Here, we study the tunnel magnetoresistance(TMR) effect in the Nb/Mn$_2$Au/CdO/Mn$_2$Au/Nb pure AF magnetic tunnel junctions (AF-MTJs) based on a first-principle scattering theory. Giant TMRs with order of 1000% are predicted in some symmetric junctions, which is originated from the interfacial resonance tunneling effect related with the k dependent complex band structures of CdO and Mn$_2$Au in companion with the enhanced spin polarization of the interfacial magnetic atoms. The effect of voltage bias and interfacial disorder such as Oxygen vacancy, Manganese vacancy, and Manganese-Cadmium exchanges at Mn2Au/CdO interfaces are studied also. Our studies suggest Nb/Mn$_2$Au/CdO/Mn$_2$Au/Nb AFMTJs promising material for AF spintronic application, and rocksalt CdO a potential symmetry filtering material for spintronic applications.

Pursuit of superconductivity in light-element systems at ambient pressure is of great experimental and theoretical interest. In this work, we combine a machine learning (ML) method with first-principles calculations to efficiently search for the energetically favorable ternary Ca-B-C compounds. Three new layered borocarbides (stable CaBC5 and metastable Ca2BC11 and CaB3C3) are predicted to be phonon-mediated superconductors at ambient pressure. The hexagonal CaB3C3 possesses the highest Tc of 26.05 K among the three compounds. The {\sigma}-bonging bands around the Fermi level account for the large electron-phonon coupling ({\lambda} = 0.980) of hexagonal CaB3C3. The ML-guided approach opens up a way for greatly accelerating the discovery of new high-Tc superconductors.

Deep learning in computer vision has achieved great success with the price of large-scale labeled training data. However, exhaustive data annotation is impracticable for each task of all domains of interest, due to high labor costs and unguaranteed labeling accuracy. Besides, the uncontrollable data collection process produces non-IID training and test data, where undesired duplication may exist. All these nuisances may hinder the verification of typical theories and exposure to new findings. To circumvent them, an alternative is to generate synthetic data via 3D rendering with domain randomization. We in this work push forward along this line by doing profound and extensive research on bare supervised learning and downstream domain adaptation. Specifically, under the well-controlled, IID data setting enabled by 3D rendering, we systematically verify the typical, important learning insights, e.g., shortcut learning, and discover the new laws of various data regimes and network architectures in generalization. We further investigate the effect of image formation factors on generalization, e.g., object scale, material texture, illumination, camera viewpoint, and background in a 3D scene. Moreover, we use the simulation-to-reality adaptation as a downstream task for comparing the transferability between synthetic and real data when used for pre-training, which demonstrates that synthetic data pre-training is also promising to improve real test results. Lastly, to promote future research, we develop a new large-scale synthetic-to-real benchmark for image classification, termed S2RDA, which provides more significant challenges for transfer from simulation to reality. The code and datasets are available at https://github.com/huitangtang/On_the_Utility_of_Synthetic_Data., Comment: 24 pages, 14 figures, 5 tables, accepted by the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023. The proposed new synthetic-to-real benchmark S2RDA is available at https://pan.baidu.com/s/1fHHaqrEHbUZLXEg9XKpgSg?pwd=w9wa. The project page is available at https://huitangtang.github.io/On_the_Utility_of_Synthetic_Data/

Despite the popularization of deep neural networks (DNNs) in many fields, it is still challenging to deploy state-of-the-art models to resource-constrained devices due to high computational overhead. Model pruning provides a feasible solution to the aforementioned challenges. However, the interpretation of existing pruning criteria is always overlooked. To counter this issue, we propose a novel layer pruning method by exploring the Stochastic Re-initialization. Our SR-init method is inspired by the discovery that the accuracy drop due to stochastic re-initialization of layer parameters differs in various layers. On the basis of this observation, we come up with a layer pruning criterion, i.e., those layers that are not sensitive to stochastic re-initialization (low accuracy drop) produce less contribution to the model and could be pruned with acceptable loss. Afterward, we experimentally verify the interpretability of SR-init via feature visualization. The visual explanation demonstrates that SR-init is theoretically feasible, thus we compare it with state-of-the-art methods to further evaluate its practicability. As for ResNet56 on CIFAR-10 and CIFAR-100, SR-init achieves a great reduction in parameters (63.98% and 37.71%) with an ignorable drop in top-1 accuracy (-0.56% and 0.8%). With ResNet50 on ImageNet, we achieve a 15.59% FLOPs reduction by removing 39.29% of the parameters, with only a drop of 0.6% in top-1 accuracy. Our code is available at https://github.com/huitang-zjut/SR-init.

Unsupervised domain adaptation addresses the problem of classifying data in an unlabeled target domain, given labeled source domain data that share a common label space but follow a different distribution. Most of the recent methods take the approach of explicitly aligning feature distributions between the two domains. Differently, motivated by the fundamental assumption for domain adaptability, we re-cast the domain adaptation problem as discriminative clustering of target data, given strong privileged information provided by the closely related, labeled source data. Technically, we use clustering objectives based on a robust variant of entropy minimization that adaptively filters target data, a soft Fisher-like criterion, and additionally the cluster ordering via centroid classification. To distill discriminative source information for target clustering, we propose to jointly train the network using parallel, supervised learning objectives over labeled source data. We term our method of distilled discriminative clustering for domain adaptation as DisClusterDA. We also give geometric intuition that illustrates how constituent objectives of DisClusterDA help learn class-wisely pure, compact feature distributions. We conduct careful ablation studies and extensive experiments on five popular benchmark datasets, including a multi-source domain adaptation one. Based on commonly used backbone networks, DisClusterDA outperforms existing methods on these benchmarks. It is also interesting to observe that in our DisClusterDA framework, adding an additional loss term that explicitly learns to align class-level feature distributions across domains does harm to the adaptation performance, though more careful studies in different algorithmic frameworks are to be conducted., Comment: A journal paper published by Pattern Recognition in 2022

Nuclear spins in quantum dots are promising candidates for fast and scalable quantum memory. By utilizing the hyperfine interaction between the central electron and its surrounding nuclei, quantum information can be transferred to the collective state of the nuclei and be stored for a long time. However, nuclear spin fluctuations in a partially polarized nuclear bath deteriorate the quantum memory fidelity. Here we introduce a noise-resistant protocol to realize fast and high-fidelity quantum memory through Hamiltonian engineering. With analytics and numerics, we show that high-fidelity quantum state transfer between the electron and the nuclear spins is achievable at relatively low nuclear polarizations, due to the strong suppression of nuclear spin noises. For a realistic quantum dot with $10^4$ nuclear spins, a fidelity surpassing 80% is possible at a polarization as low as 30%. Our approach reduces the demand for high nuclear polarization, making experimentally realizing quantum memory in quantum dots more feasible., Comment: 6 pages, 3 figures

Fermi acceleration by collisionless shocks is believed to be the primary mechanism to produce high energy charged particles in the Universe,where charged particles gain energy successively from multiple reflections off the shock front.Here,we present the first direct experimental evidence of ion energization from reflection off a supercritical quasi perpendicular collisionless shock,an essential component of Fermi acceleration in a laser produced magnetized plasma. We observed a quasi monoenergetic ion beam with 2,4 times the shock velocity in the upstream flow using time of flight method. Our related kinetic simulations reproduced the energy gain and showed that these ions were first reflected and then accelerated mainly by the motional electric field associated with the shock. This mechanism can also explain the quasi monoenergetic fast ion component observed in the Earth's bow shock.

Generating a spin vortex in a $^{87}$Rb dipolar spinor Bose-Einstein condensate in a controllable way is still experimentally challenging. We propose an experimentally easy and tunable way to produce spin vortex by varying the potential barrier height and the width of an additionally applied optical plug. A topological phase transition occurs from the trivial single mode approximation phase to the optical-plug-assisted-vortex one, as the barrier height increases and the width lies in an appropriate range. The optical plug causes radial density variation thus the spin vortex is favored by significantly lowering the intrinsic magnetic dipolar energy. A type of coreless spin vortex, different from the conventional polar core vortex, is predicted by our numerical results. Our proposal removes a major obstacle to investigate the topological phase transition in a $^{87}$Rb dipolar spinor BEC.