Search

Referring expression comprehension (REC) is a vision-language task to locate a target object in an image based on a language expression. Fully fine-tuning general-purpose pre-trained models for REC yields impressive performance but becomes increasingly costly. Parameter-efficient transfer learning (PETL) methods have shown strong performance with fewer tunable parameters. However, applying PETL to REC faces two challenges: (1) insufficient interaction between pre-trained vision and language encoders, and (2) high GPU memory usage due to gradients passing through both heavy encoders. To address these issues, we present M$^2$IST: Multi-Modal Interactive Side-Tuning with M$^3$ISAs: Mixture of Multi-Modal Interactive Side-Adapters. During fine-tuning, we keep the pre-trained vision and language encoders fixed and update M$^3$ISAs on side networks to establish connections between them, thereby achieving parameter- and memory-efficient tuning for REC. Empirical results on three benchmarks show M$^2$IST achieves the best performance-parameter-memory trade-off compared to full fine-tuning and other PETL methods, with only 3.14M tunable parameters (2.11% of full fine-tuning) and 15.44GB GPU memory usage (39.61% of full fine-tuning). Source code will soon be publicly available.

With the advent of image super-resolution (SR) algorithms, how to evaluate the quality of generated SR images has become an urgent task. Although full-reference methods perform well in SR image quality assessment (SR-IQA), their reliance on high-resolution (HR) images limits their practical applicability. Leveraging available reconstruction information as much as possible for SR-IQA, such as low-resolution (LR) images and the scale factors, is a promising way to enhance assessment performance for SR-IQA without HR for reference. In this letter, we attempt to evaluate the perceptual quality and reconstruction fidelity of SR images considering LR images and scale factors. Specifically, we propose a novel dual-branch reduced-reference SR-IQA network, \ie, Perception- and Fidelity-aware SR-IQA (PFIQA). The perception-aware branch evaluates the perceptual quality of SR images by leveraging the merits of global modeling of Vision Transformer (ViT) and local relation of ResNet, and incorporating the scale factor to enable comprehensive visual perception. Meanwhile, the fidelity-aware branch assesses the reconstruction fidelity between LR and SR images through their visual perception. The combination of the two branches substantially aligns with the human visual system, enabling a comprehensive SR image evaluation. Experimental results indicate that our PFIQA outperforms current state-of-the-art models across three widely-used SR-IQA benchmarks. Notably, PFIQA excels in assessing the quality of real-world SR images., Comment: This work has been submitted to the IEEE for possible publication. Copyright may be transferred without notice, after which this version may no longer be accessible

Visual grounding (VG) is a challenging task to localize an object in an image based on a textual description. Recent surge in the scale of VG models has substantially improved performance, but also introduced a significant burden on computational costs during fine-tuning. In this paper, we explore applying parameter-efficient transfer learning (PETL) to efficiently transfer the pre-trained vision-language knowledge to VG. Specifically, we propose \textbf{DARA}, a novel PETL method comprising \underline{\textbf{D}}omain-aware \underline{\textbf{A}}dapters (DA Adapters) and \underline{\textbf{R}}elation-aware \underline{\textbf{A}}dapters (RA Adapters) for VG. DA Adapters first transfer intra-modality representations to be more fine-grained for the VG domain. Then RA Adapters share weights to bridge the relation between two modalities, improving spatial reasoning. Empirical results on widely-used benchmarks demonstrate that DARA achieves the best accuracy while saving numerous updated parameters compared to the full fine-tuning and other PETL methods. Notably, with only \textbf{2.13\%} tunable backbone parameters, DARA improves average accuracy by \textbf{0.81\%} across the three benchmarks compared to the baseline model. Our code is available at \url{https://github.com/liuting20/DARA}., Comment: Accepted by ICME 2024 (Oral)

Contemporary recommender systems predominantly rely on collaborative filtering techniques, employing ID-embedding to capture latent associations among users and items. However, this approach overlooks the wealth of semantic information embedded within textual descriptions of items, leading to suboptimal performance in cold-start scenarios and long-tail user recommendations. Leveraging the capabilities of Large Language Models (LLMs) pretrained on massive text corpus presents a promising avenue for enhancing recommender systems by integrating open-world domain knowledge. In this paper, we propose an Llm-driven knowlEdge Adaptive RecommeNdation (LEARN) framework that synergizes open-world knowledge with collaborative knowledge. We address computational complexity concerns by utilizing pretrained LLMs as item encoders and freezing LLM parameters to avoid catastrophic forgetting and preserve open-world knowledge. To bridge the gap between the open-world and collaborative domains, we design a twin-tower structure supervised by the recommendation task and tailored for practical industrial application. Through offline experiments on the large-scale industrial dataset and online experiments on A/B tests, we demonstrate the efficacy of our approach., Comment: 11 pages, 6 figures

Presently, the task of few-shot object detection (FSOD) in remote sensing images (RSIs) has become a focal point of attention. Numerous few-shot detectors, particularly those based on two-stage detectors, face challenges when dealing with the multiscale complexities inherent in RSIs. Moreover, these detectors present impractical characteristics in real-world applications, mainly due to their unwieldy model parameters when handling large amount of data. In contrast, we recognize the advantages of one-stage detectors, including high detection speed and a global receptive field. Consequently, we choose the YOLOv7 one-stage detector as a baseline and subject it to a novel meta-learning training framework. This transformation allows the detector to adeptly address FSOD tasks while capitalizing on its inherent advantage of lightweight. Additionally, we thoroughly investigate the samples generated by the meta-learning strategy and introduce a novel meta-sampling approach to retain samples produced by our designed meta-detection head. Coupled with our devised meta-cross loss, we deliberately utilize "negative samples" that are often overlooked to extract valuable knowledge from them. This approach serves to enhance detection accuracy and efficiently refine the overall meta-learning strategy. To validate the effectiveness of our proposed detector, we conducted performance comparisons with current state-of-the-art detectors using the DIOR and NWPU VHR-10.v2 datasets, yielding satisfactory results.

Superconducting materials exhibit unique physical properties and hold great scientific value and vast industrial application prospects. However, due to limitations such as critical temperature (TC) and critical current density (JC), the large-scale application of superconducting materials remains challenging. Chemical doping has been a commonly used method to enhance the superconductivity of B(P)SCCO. However, satisfactory enhancement results have been difficult to achieve. In this study, we introduced green light GaN p-n junction particles as inhomogeneous phases into B(P)SCCO polycrystalline particles to form a smart meta-superconductors (SMSCs) structure. Based on the electroluminescence properties of the p-n junction, the Cooper pairs were stimulated and strengthened to enhance the superconductivity of B(P)SCCO. Experimental results demonstrate that the introduction of inhomogeneous phases can indeed enhance the critical temperature TC, critical current density JC, and complete diamagnetism (Meissner effect) of B(P)SCCO superconductors. Moreover, When the particle size of raw material of B(P)SCCO is reduced from 30{\mu}m to 5{\mu}m, the grain size of the sintered samples also decreases, and the optimal doping concentration of the inhomogeneous phases increases from 0.15 wt.% to 0.2 wt.%, further improving the enhancement of superconductivity., Comment: 17 pages,7 figures

Large-scale text-to-image diffusion models have shown impressive capabilities for generative tasks by leveraging strong vision-language alignment from pre-training. However, most vision-language discriminative tasks require extensive fine-tuning on carefully-labeled datasets to acquire such alignment, with great cost in time and computing resources. In this work, we explore directly applying a pre-trained generative diffusion model to the challenging discriminative task of visual grounding without any fine-tuning and additional training dataset. Specifically, we propose VGDiffZero, a simple yet effective zero-shot visual grounding framework based on text-to-image diffusion models. We also design a comprehensive region-scoring method considering both global and local contexts of each isolated proposal. Extensive experiments on RefCOCO, RefCOCO+, and RefCOCOg show that VGDiffZero achieves strong performance on zero-shot visual grounding. Our code is available at https://github.com/xuyang-liu16/VGDiffZero., Comment: Accepted by ICASSP 2024

Improving the critical temperature (TC), critical magnetic field (HC), and critical current (JC) of superconducting materials has always been one of the most significant challenges in the field of superconductivity, but progress has been slow over the years. Based on the concept of injecting energy to enhance electron pairing states, in this study, we have employed a solid-state sintering method to fabricate a series of smart meta-superconductors (SMSCs) consisting of p-n junction nanostructures with a wavelength of 550 nm, doped within an MgB2 matrix. Experimental results demonstrate that compared to pure MgB2 samples, the critical transition temperature (TC) has increased by 1.2 K, the critical current (JC) has increased by 52.8%, and the Meissner effect (HC) shows significant improvement in its diamagnetic properties. This phenomenon of enhanced superconducting performance can be explained by the coupling between superconducting electrons and evanescent waves.

Increasing and improving the critical transition temperature (Tc), current density (Jc) and Meissner effect (Hc) of conventional superconductors are the most important problems in superconductivity research, but progress has been slow for many years. In this study, by introducing the p-n junction electroluminescent inhomogeneous phase with red wavelength to realize energy injection, we found the improved property of smart meta-superconductors MgB2, the critical transition temperature Tc increases by 0.8K, the current density Jc increases by 37%, and the diamagnetism of Meissner effect Hc also significantly improved, compared with pure MgB2. Compared with previous yttrium oxide inhomogeneous phase, p-n junction has higher luminescence intensity, longer stable life and simpler external field requirements. The coupling between superconducting electrons and surface plasmon polaritons may be explain this phenomenon. The realization of smart meta-superconductor by this electroluminescent inhomogeneous phase provides a new way to improve the performance of superconductors.

A smart meta-superconductor Bi(Pb)SrCaCuO (B(P)SCCO) may increase the critical transition temperature (TC) of B(P)SCCO by electroluminescence (EL) energy injection of inhomogeneous phases. However, the increase amplitude {\Delta}TC ({\Delta}TC=TC-T(C,pure)) of TC is relatively small. In this study, a smart meta-superconductor B(P)SCCO with different matrix sizes was designed. Three kinds of raw materials with different particle sizes were used, and different series of Y2O3:Sm3+, Y2O3, Y2O3:Eu3+, and Y2O3:Eu3++Ag doped samples and pure B(P)SCCO were prepared. Results indicated that the TC of the Y2O3 or Y2O3:Sm3+ non-luminescent dopant doping sample is lower than that of pure B(P)SCCO. However, the TC of the Y2O3:Eu3++Ag or Y2O3:Eu3+ luminescent inhomogeneous phase doping sample is higher than that of pure B(P)SCCO. With the decrease of the raw material particle size from 30 to 5 {\mu}m, the particle size of the B(P)SCCO superconducting matrix in the prepared samples decreases, and the doping content of the Y2O3:Eu3++Ag or Y2O3:Eu3+ increases from 0.2% to 0.4%. Meanwhile, the increase of the inhomogeneous phase content enhances the {\Delta}TC. When the particle size of raw material is 5 {\mu}m, the doping concentration of the luminescent inhomogeneous phase can be increased to 0.4%. At this time, the zero-resistance temperature and onset transition temperature of the Y2O3:Eu3++Ag doped sample are 4 and 6.3 K higher than those of pure B(P)SCCO, respectively.

Single image super-resolution (SISR), which aims to reconstruct a high-resolution (HR) image from a low-resolution (LR) observation, has been an active research topic in the area of image processing in recent decades. Particularly, deep learning-based super-resolution (SR) approaches have drawn much attention and have greatly improved the reconstruction performance on synthetic data. Recent studies show that simulation results on synthetic data usually overestimate the capacity to super-resolve real-world images. In this context, more and more researchers devote themselves to develop SR approaches for realistic images. This article aims to make a comprehensive review on real-world single image super-resolution (RSISR). More specifically, this review covers the critical publically available datasets and assessment metrics for RSISR, and four major categories of RSISR methods, namely the degradation modeling-based RSISR, image pairs-based RSISR, domain translation-based RSISR, and self-learning-based RSISR. Comparisons are also made among representative RSISR methods on benchmark datasets, in terms of both reconstruction quality and computational efficiency. Besides, we discuss challenges and promising research topics on RSISR., Comment: 18 pages, 12 figure, 4 tables

Incorporating with inhomogeneous phases with high electroluminescence (EL) intensity to prepare smart meta-superconductors (SMSCs) is an effective method of increasing the superconducting transition temperature (Tc) and has been confirmed in both MgB2 and Bi(Pb)SrCaCuO systems. However, the increase of {\Delta}Tc has been quite small because of the low optimal concentrations of inhomogeneous phases. In this work, three kinds of MgB2 raw materials, namely, aMgB2, bMgB2, and cMgB2, were prepared with particle sizes decreasing in order. Inhomogeneous phases, Y2O3:Eu3+ and Y2O3:Eu3+/Ag, were also prepared and doped into MgB2 to study the influence of doping concentration on the {\Delta}Tc of MgB2 with different particle sizes. Results show that reducing the MgB2 particle size increases the optimal doping concentration of inhomogeneous phases, thereby increasing {\Delta}Tc. The optimal doping concentrations for aMgB2, bMgB2, and cMgB2 are 0.5%, 0.8%, and 1.2%, respectively. The corresponding {\Delta}Tc values are 0.4, 0.9, and 1.2 K, respectively. This work open a new approach to reinforcing increase of {\Delta}Tc in MgB2 SMSCs.

Improving the critical transition temperature (TC) of Bi(Pb)SrCaCuO (B(P)SCCO) high-temperature superconductors is important, however, considerable challenges exist. In this study, on the basis of the metamaterial structure and the idea that the injecting energy will promote the formation of Cooper pairs, a smart meta-superconductor B(P)SCCO consisting of B(P)SCCO microparticles and Y2O3:Eu3++Ag or Y2O3:Eu3+ luminophor was designed. In the applied electric field, the Y2O3:Eu3++Ag or Y2O3:Eu3+ luminophor generates an electroluminescence (EL), thereby promoting the TC via EL energy injection. A series of Y2O3:Eu3++Ag topological luminophor-doped B(P)SCCO samples was prepared. Results showed that Y2O3:Eu3++Ag was dispersed around B(P)SCCO particles, forming a metastructure. Accordingly, the onset transition temperature (T_(C,on)) and zero resistance transition temperature (T_(C,0)) of B(P)SCCO increased. Meanwhile, the B(P)SCCO sample doped with 0.2 wt% Y2O3 or Y2O3:Sm3+ nonluminous inhomogeneous phase was also prepared to further prove the influence of EL on the T_C rather than the rare earth effect. Results indicated that the TC of the Y2O3 or Y2O3:Sm3+ doping sample decreased. However, the TC of the 0.2 wt% Y2O3:Eu3++Ag or Y2O3:Eu3+ luminophor-doped sample improved. This outcome further demonstrated that the smart metastructure method can improve the TC of B(P)SCCO.

Porous media are ubiquitous in both nature and engineering applications, thus their modelling and understanding is of vital importance. In contrast to direct acquisition of three-dimensional (3D) images of such medium, obtaining its sub-region (s) like two-dimensional (2D) images or several small areas could be much feasible. Therefore, reconstructing whole images from the limited information is a primary technique in such cases. Specially, in practice the given data cannot generally be determined by users and may be incomplete or partially informed, thus making existing reconstruction methods inaccurate or even ineffective. To overcome this shortcoming, in this study we proposed a deep learning-based framework for reconstructing full image from its much smaller sub-area(s). Particularly, conditional generative adversarial network (CGAN) is utilized to learn the mapping between input (partial image) and output (full image). To preserve the reconstruction accuracy, two simple but effective objective functions are proposed and then coupled with the other two functions to jointly constrain the training procedure. Due to the inherent essence of this ill-posed problem, a Gaussian noise is introduced for producing reconstruction diversity, thus allowing for providing multiple candidate outputs. Extensively tested on a variety of porous materials and demonstrated by both visual inspection and quantitative comparison, the method is shown to be accurate, stable yet fast ($\sim0.08s$ for a $128 \times 128$ image reconstruction). We highlight that the proposed approach can be readily extended, such as incorporating any user-define conditional data and an arbitrary number of object functions into reconstruction, and being coupled with other reconstruction methods.

On the basis of the idea that the injecting energy will improve the conditions for the formation of Cooper pairs, a smart meta-superconductor (SMSC) was prepared by doping inhomogeneous phase of luminescent nanocomposite Y2O3:Eu3+/Ag, which has the strong luminescence characteristic, in MgB2 to improve the superconducting transition temperature (TC) of the MgB2-based superconductor. Two types of Y2O3:Eu3+/Ag with different sizes were prepared and marked as m-Y2O3:Eu3+/Ag and nY2O3:Eu3+/Ag. MgB2 SMSC was prepared through an ex situ process. Results show that when the inhomogeneous phase content was fixed at 2.0 wt.%, the TC of MgB2 SMSC increased initially then decreased with the increase in the Ag content in the dopant. When the Ag content accounted for 5 wt.% of the inhomogeneous phase weight, the TC of MgB2 SMSC was 37.2-38.0 K, which was similar to that of pure MgB2. Meanwhile, the TC of MgB2 SMSC doped with n-Y2O3:Eu3+/Ag increased initially then decreased basically with the increase in the content of n-Y2O3:Eu3+/Ag, in which Ag accounted for 5 wt.% of the inhomogeneous phase. The TC of MgB2 SMSC doped with 0.5 wt.% n-Y2O3:Eu3+/Ag was 37.6-38.4 K, which was 0.4 K higher than that of pure MgB2. It is thought that the doping inhomogeneous phase of luminescent nanocomposite into the superconductor is a new means to improve the TC of SMSC.

JPEG is one of the widely used lossy compression methods. JPEG-compressed images usually suffer from compression artifacts including blocking and blurring, especially at low bit-rates. Soft decoding is an effective solution to improve the quality of compressed images without changing codec or introducing extra coding bits. Inspired by the excellent performance of the deep convolutional neural networks (CNNs) on both low-level and high-level computer vision problems, we develop a dual pixel-wavelet domain deep CNNs-based soft decoding network for JPEG-compressed images, namely DPW-SDNet. The pixel domain deep network takes the four downsampled versions of the compressed image to form a 4-channel input and outputs a pixel domain prediction, while the wavelet domain deep network uses the 1-level discrete wavelet transformation (DWT) coefficients to form a 4-channel input to produce a DWT domain prediction. The pixel domain and wavelet domain estimates are combined to generate the final soft decoded result. Experimental results demonstrate the superiority of the proposed DPW-SDNet over several state-of-the-art compression artifacts reduction algorithms., Comment: CVPRW 2018

The critical temperature (TC) of MgB2, one of the key factors limiting its application, is highly desired to be improved. On the basis of the meta-material structure, we prepared a smart meta-superconductor structure consisting of MgB2 micro-particles and inhomogeneous phases by an ex situ process. The effect of inhomogeneous phase on the TC of smart meta-superconductor MgB2 was investigated. Results showed that the onset temperature (Ton C) of doping samples was lower than those of pure MgB2. However, the offset temperature (Toff C) of the sample doped with Y2O3:Eu3+ nanosheets with a thickness of 2~3 nm which is much less than the coherence length of MgB2 is 1.2 K higher than that of pure MgB2. The effect of the applied electric field on the TC of sample was also studied. Results indicated that with the increase of current, Ton C is slightly increased in the samples doping with different inhomogeneous phases. When increasing current, the Toff C of the samples doped with nonluminous inhomogeneous phases was decreased. However, the Toff C of the luminescent inhomogeneous phase doping samples increased and then decreased as increasing current.