Search

Image denoising is a fundamental task in computer vision. While prevailing deep learning-based supervised and self-supervised methods have excelled in eliminating in-distribution noise, their susceptibility to out-of-distribution (OOD) noise remains a significant challenge. The recent emergence of contrastive language-image pre-training (CLIP) model has showcased exceptional capabilities in open-world image recognition and segmentation. Yet, the potential for leveraging CLIP to enhance the robustness of low-level tasks remains largely unexplored. This paper uncovers that certain dense features extracted from the frozen ResNet image encoder of CLIP exhibit distortion-invariant and content-related properties, which are highly desirable for generalizable denoising. Leveraging these properties, we devise an asymmetrical encoder-decoder denoising network, which incorporates dense features including the noisy image and its multi-scale features from the frozen ResNet encoder of CLIP into a learnable image decoder to achieve generalizable denoising. The progressive feature augmentation strategy is further proposed to mitigate feature overfitting and improve the robustness of the learnable decoder. Extensive experiments and comparisons conducted across diverse OOD noises, including synthetic noise, real-world sRGB noise, and low-dose CT image noise, demonstrate the superior generalization ability of our method., Comment: Accepted by CVPR2024

Image denoising is a fundamental task in low-level computer vision. While recent deep learning-based image denoising methods have achieved impressive performance, they are black-box models and the underlying denoising principle remains unclear. In this paper, we propose a novel approach to image denoising that offers both clear denoising mechanism and good performance. We view noise as a type of image style and remove it by incorporating noise-free styles derived from clean images. To achieve this, we design novel losses and network modules to extract noisy styles from noisy images and noise-free styles from clean images. The noise-free style induces low-response activations for noise features and high-response activations for content features in the feature space. This leads to the separation of clean contents from noise, effectively denoising the image. Unlike disentanglement-based image editing tasks that edit semantic-level attributes using styles, our main contribution lies in editing pixel-level attributes through global noise-free styles. We conduct extensive experiments on synthetic noise removal and real-world image denoising datasets (SIDD and DND), demonstrating the effectiveness of our method in terms of both PSNR and SSIM metrics. Moreover, we experimentally validate that our method offers good interpretability.

During the process of computed tomography (CT), metallic implants often cause disruptive artifacts in the reconstructed images, impeding accurate diagnosis. Several supervised deep learning-based approaches have been proposed for reducing metal artifacts (MAR). However, these methods heavily rely on training with simulated data, as obtaining paired metal artifact CT and clean CT data in clinical settings is challenging. This limitation can lead to decreased performance when applying these methods in clinical practice. Existing unsupervised MAR methods, whether based on learning or not, typically operate within a single domain, either in the image domain or the sinogram domain. In this paper, we propose an unsupervised MAR method based on the diffusion model, a generative model with a high capacity to represent data distributions. Specifically, we first train a diffusion model using CT images without metal artifacts. Subsequently, we iteratively utilize the priors embedded within the pre-trained diffusion model in both the sinogram and image domains to restore the degraded portions caused by metal artifacts. This dual-domain processing empowers our approach to outperform existing unsupervised MAR methods, including another MAR method based on the diffusion model, which we have qualitatively and quantitatively validated using synthetic datasets. Moreover, our method demonstrates superior visual results compared to both supervised and unsupervised methods on clinical datasets.

Real-world single image denoising is crucial and practical in computer vision. Bayesian inversions combined with score priors now have proven effective for single image denoising but are limited to white Gaussian noise. Moreover, applying existing score-based methods for real-world denoising requires not only the explicit train of score priors on the target domain but also the careful design of sampling procedures for posterior inference, which is complicated and impractical. To address these limitations, we propose a score priors-guided deep variational inference, namely ScoreDVI, for practical real-world denoising. By considering the deep variational image posterior with a Gaussian form, score priors are extracted based on easily accessible minimum MSE Non-$i.i.d$ Gaussian denoisers and variational samples, which in turn facilitate optimizing the variational image posterior. Such a procedure adaptively applies cheap score priors to denoising. Additionally, we exploit a Non-$i.i.d$ Gaussian mixture model and variational noise posterior to model the real-world noise. This scheme also enables the pixel-wise fusion of multiple image priors and variational image posteriors. Besides, we develop a noise-aware prior assignment strategy that dynamically adjusts the weight of image priors in the optimization. Our method outperforms other single image-based real-world denoising methods and achieves comparable performance to dataset-based unsupervised methods., Comment: Accepted by ICCV2023

Denoising low-dose computed tomography (CT) images is a critical task in medical image computing. Supervised deep learning-based approaches have made significant advancements in this area in recent years. However, these methods typically require pairs of low-dose and normal-dose CT images for training, which are challenging to obtain in clinical settings. Existing unsupervised deep learning-based methods often require training with a large number of low-dose CT images or rely on specially designed data acquisition processes to obtain training data. To address these limitations, we propose a novel unsupervised method that only utilizes normal-dose CT images during training, enabling zero-shot denoising of low-dose CT images. Our method leverages the diffusion model, a powerful generative model. We begin by training a cascaded unconditional diffusion model capable of generating high-quality normal-dose CT images from low-resolution to high-resolution. The cascaded architecture makes the training of high-resolution diffusion models more feasible. Subsequently, we introduce low-dose CT images into the reverse process of the diffusion model as likelihood, combined with the priors provided by the diffusion model and iteratively solve multiple maximum a posteriori (MAP) problems to achieve denoising. Additionally, we propose methods to adaptively adjust the coefficients that balance the likelihood and prior in MAP estimations, allowing for adaptation to different noise levels in low-dose CT images. We test our method on low-dose CT datasets of different regions with varying dose levels. The results demonstrate that our method outperforms the state-of-the-art unsupervised method and surpasses several supervised deep learning-based methods. Codes are available in https://github.com/DeepXuan/Dn-Dp.

Diffusion models have emerged as powerful generative models in the text-to-image domain. This paper studies their application as observation-to-action models for imitating human behaviour in sequential environments. Human behaviour is stochastic and multimodal, with structured correlations between action dimensions. Meanwhile, standard modelling choices in behaviour cloning are limited in their expressiveness and may introduce bias into the cloned policy. We begin by pointing out the limitations of these choices. We then propose that diffusion models are an excellent fit for imitating human behaviour, since they learn an expressive distribution over the joint action space. We introduce several innovations to make diffusion models suitable for sequential environments; designing suitable architectures, investigating the role of guidance, and developing reliable sampling strategies. Experimentally, diffusion models closely match human demonstrations in a simulated robotic control task and a modern 3D gaming environment., Comment: Published in ICLR 2023

Operational stability of perovskite solar cells is remarkably influenced by the device temperature, therefore, decreasing the interior temperature of the device is one of the most effective approaches to prolong the service life. Herein, we introduce the spontaneous radiative cooling effect into the perovskite solar cell and amplified this effect via functional structure design of a full-carbon electrode (F-CE). Firstly, with interface engineering, >19% and >23% power conversion efficiencies of F-CE based inorganic CsPbI3 and hybrid perovskite solar cells have been achieved, respectively, both of which are the highest reported efficiencies based on carbon electrode and are comparative to the results for metal electrodes. Highly efficient thermal radiation of this F-CE can reduce the temperature of the operating cell by about 10 {\deg}C. Compared with the conventional metal electrode-based control cells, the operational stability of the above two types of cells have been significantly improved due to this cooling effect. Especially, the CsPbI3 PSCs exhibited no efficiency degradation after 2000 hours of continuous operational tracking.

In the paper, we have given the quantum transmissivity, probability density and probability current density of photon in one-dimensional photonic crystals $(AB)^N$ with the quantum theory approach. We find the quantum transmissivity is identical to the classical transmissivity. When the incident angle $\theta$ and periodic number $N$ change the probability density and probability current density are approximate periodic change, and their amplitude are increased with the incident angles $\theta$ and periodic number $N$ increasing. Otherwise, we find when the frequency of incident photon is corresponding to transmissivity $T=1$, the amplitude of the probability density is the largest. When the frequency of incident photon is corresponding to transmissivity $T=0$, the amplitude of the probability density attenuate rapidly to zero, it indicates that there is the quantum tunnel effect of photon in photonic crystals.

Quantification of local metabolic tumor volume (MTV) chan-ges after Chemo-radiotherapy would allow accurate tumor response evaluation. Currently, local MTV changes in esophageal (soft-tissue) cancer are measured by registering follow-up PET to baseline PET using the same transformation obtained by deformable registration of follow-up CT to baseline CT. Such approach is suboptimal because PET and CT capture fundamentally different properties (metabolic vs. anatomy) of a tumor. In this work we combined PET and CT images into a single blended PET-CT image and registered follow-up blended PET-CT image to baseline blended PET-CT image. B-spline regularized diffeomorphic registration was used to characterize the large MTV shrinkage. Jacobian of the resulting transformation was computed to measure the local MTV changes. Radiomic features (intensity and texture) were then extracted from the Jacobian map to predict pathologic tumor response. Local MTV changes calculated using blended PET-CT registration achieved the highest correlation with ground truth segmentation (R=0.88) compared to PET-PET (R=0.80) and CT-CT (R=0.67) registrations. Moreover, using blended PET-CT registration, the multivariate prediction model achieved the highest accuracy with only one Jacobian co-occurrence texture feature (accuracy=82.3%). This novel framework can replace the conventional approach that applies CT-CT transformation to the PET data for longitudinal evaluation of tumor response., Comment: MICCAI 2018: Workshop on Data-Driven Treatment Response Assessment (Oral Presentation), 11 pages

Part one: In this paper, we propose a key idea and innovation on the Collatz problem. Let an odd Collatz's Sequence be \[ S_{c}(m,t)=\{n_{0}(m,t),n_{1}(m,t),n_{2}(m,t),\ldots,n_{m}(m,t),n_{m+1}(m,t)\} \] produced by a function $n_{i+1}(m,t)=(3n_{i}(m,t)+1)/2$ and ended by an even number $n_{m+1}(m,t)$ where \[ n_{i}(m,t)=2^{m+1-i}\times3^{i}t-1\texttt{ for }i=0,1,2,\ldots,m+1 \] and the positive integer $t$ is odd and not divisible by $3$. Then in two ways we prove that each Collatz's Sequence always returns to 1, i.e., the Collatz problem is solved. Part two: Given an equation $y^{2}=x^{3}+ax+b$ where $a\in Q\texttt{ and }b\in Q$, we prove that its pairs of rational solutions exist and can be determined only in some cases, each pair of which are $x$-axially symmetric, and the number of its pairs of rational solutions except for $x=0$ or $y=0$ or except for $x=\pm x_{0}$ or $y=0$ where $x_{0}\in Q$ is also proved equal to one or equal to two or equal to the number of plus cubic rational numbers only in some cases, and equal to zero in other cases., Comment: 39 pages

Part One: First, we prove that given each natural number $x\geq10^{3}$ we have $$|{\rm Li}(x)-\pi(x)|\leq c\sqrt{x}\log x\texttt{ or } \pi(x)={\rm Li}(x)+O(\sqrt{x}\log x)$$ where $c$ is a constant greater than $1$ and less than $e$. Hence the Riemann Hypothesis is true according to the theorem proved by H. Koch in 1901. Second, with a much more accurate estimation of prime numbers, the error range of which is less than $x^{1/2-0.0327283}$ for $x\geq10^{41}$, we prove a theorem of the number of primes in short intervals: Given a positive real $\beta$, let $\Phi(x):=\beta x^{1/2}$ for $x\geq x_{\beta}$ that satisfies $e(\log x_{\beta})^{3}/x_{\beta}^{0.0327283}=\beta$. Then there are $$\frac{\pi(x+\Phi(x))-\pi(x)}{\Phi(x)/\log x}=1+O(\frac{1}{\log x})$$ and $$\lim_{x \to \infty}\frac{\pi(x+\Phi(x))-\pi(x)}{\Phi(x)/\log x}=1.$$ Part Two: Based on a theorem for the numbers of primes in short intervals $x^{1/2}$, some conjectures of distribution of primes in short intervals, such as Legendre's conjecture, Oppermann's conjecture, Hanssner's conjecture, Brocard's conjecture, Andrica's conjecture, Sierpinski's conjecture and Sierpinski's conjecture of triangular numbers are proved and the Mills' constant can be determined., Comment: 62 pages