Search

Abstract Single image super‐resolution (SISR) is a promising research direction in computer vision and image processing for improving the visual perception of low‐quality images. In recent years, deep learning algorithms have driven tremendous development in SR, and SR methods based on various network architectures have significantly improved the quality of reconstructed images. Although there has been a large amount of reviews focusing on SISR, few studies have focused specifically on network architectures for SISR. This paper aims to provide a systematic overview of the design ideas of SISR using multiple architectures, including Convolutional Neural Networks (CNN), Generative Adversarial Networks (GAN), Transformer, and Diffusion model. In addition, an experimental analysis and comparison of state‐of‐the‐art SR algorithms have been performed on publicly available quantitative and qualitative datasets. Finally, some future directions are discussed that may help other community researchers.

Spectral computed tomography (spectral CT) is a promising medical imaging technology because of its ability to provide information on material characterization and quantification. However, the difficulty of decomposition increases due to the nonlinearity of the measurements and the text ill-condition of the problem, especially in the case of geometric inconsistency, which typically leads to low image qualities. Therefore, it is a crucial issue for inconsistent spectral CT imaging to improve the accuracy of material decomposition while suppressing noise. This paper proposes one-step multi-material algorithms based on a statistical reconstruction model with different priors. In these approaches, the gradient sparsity based and convolutional neural network based methods are designed for the case of the consistent numbers of material and energies. Furthermore, volume conservation constraint is developed while the two numbers are not equal. An efficient Newton descent method is adopted based on a simple surrogate function. For simulation experiments with different noise levels, the largest peak signal-to-noise ratio (PSNR) obtained by the proposed method approximately increases by 20.924 dB and 18.283 dB compared with those of other algorithms. Magnified areas of real data also demonstrated that the proposed methods have a better ability to suppress noise. Numerical experiments verify that the proposed methods efficiently reconstruct the material maps and reduced noise compared with the state-of-the-art methods.

Spectral computed tomography (CT) has a wide range of applications in material discrimination, clinical diagnosis and tissue representation. However, the photon counting detector measurements are subject to serious quantum noise caused by photon starvation, photon accumulation, charge sharing, and other factors, which will lead to a decrease in the quality of the reconstructed image and make clinical diagnosis more difficult. To tackle with this problem, this paper proposes a spectral CT reconstruction technique that exploits the spatial sparsity of inter-channel images and the high correlation of images between different energy channels. Specifically, similar patches from spatial and spectral domains are extracted to form the low-rank tensors. Then the tensor-train rank, which is derived from a well-balanced matricization technique, is adopted to depict the high correlation among different energy channels. To capture the self-similarity of the low-rank tensors, the L0-norm of the image gradient is employed for image smoothing. An efficient algorithm is devised to solve the reconstruction model utilizing the Alternating Direction Method of Multipliers. For the sake of testing and verifying the effectiveness of the proposed algorithm, numerical simulations and real data experiments are conducted. Qualitatively, the designed method demonstrates a clear advantage in image quality over the existing state-of-the-art algorithms. For instance, when taking the full energy bin image as an example, the proposed method reduces the Root Mean Square Error (RMSE) by 52.07%, 38.69%, 35.13%, 12.67%, respectively, compared to the competing methods. Quantitative and qualitative assessment indices have revealed that the suggested method has excellent noise suppression, artifact elimination, and image detail preservation properties.

Spectral computed tomography (spectral CT) is a promising medical imaging technology because of its ability to provide information on material characterization and quantification. However, with an increasing number of basis materials, the nonlinearity of measurements causes difficulty in decomposition. In addition, noise amplification and beam hardening further reduce image quality. Thus, improving the accuracy of material decomposition while suppressing noise is pivotal for spectral CT imaging. This paper proposes a one-step multi-material reconstruction model as well as an iterative proximal adaptive decent method. In this approach, a proximal step and a descent step with adaptive step size are designed under the forward–backward splitting framework. The convergence analysis of the algorithm is further discussed according to the convexity of the optimization objective function. For simulation experiments with different noise levels, the peak signal-to-noise ratio (PSNR) obtained by the proposed method increases approximately 23 dB, 14 dB, and 4 dB compared to those of other algorithms. Magnified areas of thorax data further demonstrated that the proposed method has a better ability to preserve details in tissues, bones, and lungs. Numerical experiments verify that the proposed method efficiently reconstructed the material maps, and reduced noise and beam hardening artifacts compared with the state-of-the-art methods.

In recent years, research on decoding brain activity based on functional magnetic resonance imaging (fMRI) has made eye-catching achievements. However, constraint-free natural image reconstruction from brain activity remains a challenge, as specifying brain activity for all possible images is impractical. The problem was often simplified by using semantic prior information or just reconstructing simple images, including digitals and letters. Without semantic prior information, we present a novel method to reconstruct natural images from the fMRI signals of human visual cortex based on the computation model of convolutional neural network (CNN). First, we extracted the unit output of viewed natural images in each layer of a pre-trained CNN as CNN features. Second, we transformed image reconstruction from fMRI signals into the problem of CNN feature visualization by training a sparse linear regression to map from the fMRI patterns to CNN features. By iteratively optimization to find the matched image, whose CNN unit features become most similar to those predicted from the brain activity, we finally achieved the promising results for the challenging constraint-free natural image reconstruction. The semantic prior information of the stimuli was not used when training decoding model, and any category of images (not constraint by the training set) could be reconstructed theoretically. We found that the reconstructed images resembled the natural stimuli, especially in position and shape. The experimental results suggest that hierarchical visual features may be an effective tool to express the human visual processing.

Deep learning techniques have significantly improved image restoration tasks in recent years. As a crucial component of deep learning, the loss function plays a key role in network optimization and performance enhancement. However, the currently prevalent loss functions assign equal weight to each pixel point during loss calculation, which hampers the ability to reflect the roles of different pixel points and fails to exploit the image’s characteristics fully. To address this issue, this study proposes an asymmetric loss function based on the image and data characteristics of the image recovery task. This novel loss function can adjust the weight of the reconstruction loss based on the grey value of different pixel points, thereby effectively optimizing the network training by differentially utilizing the grey information from the original image. Specifically, we calculate a weight factor for each pixel point based on its grey value and combine it with the reconstruction loss to create a new loss function. This ensures that pixel points with smaller grey values receive greater attention, improving network recovery. In order to verify the effectiveness of the proposed asymmetric loss function, we conducted experimental tests in the image super-resolution task. The experimental results show that the model with the introduction of asymmetric loss weights improves all the indexes of the processing results without increasing the training time. In the typical superresolution network SRCNN, by introducing asymmetric weights, it is possible to improve the peak signal-to-noise ratio (PSNR) by up to about 0.5%, the structural similarity index (SSIM) by up to about 0.3%, and reduce the rootmean- square error (RMSE) by up to about 1.7% with essentially no increase in training time. In addition, we also further tested the performance of the proposed method in the denoising task to verify the potential applicability of the method in the image restoration task. [ABSTRACT FROM AUTHOR]

Sparse-viewimaging is a promising scanning approach which has fast scanning rate and low-radiation dose in X-ray computed tomography (CT). Conventional L1-norm based total variation (TV) has been widely used in image reconstruction since the advent of compressive sensing theory. However, with only the first order information of the image used, the TV often generates dissatisfactory image for some applications. As is widely known, image curvature is among the most important second order features of images and can potentially be applied in image reconstruction for quality improvement. This study incorporates the curvature in the optimization model and proposes a new total absolute curvature (TAC) based reconstruction method. The proposed model contains both total absolute curvature and total variation (TAC-TV), which are intended for better description of the featured complicated image. As for the practical algorithm development, the efficient alternating direction method of multipliers (ADMM) is utilized, which generates a practical and easy-coded algorithm. The TAC-TV iterations mainly contain FFTs, soft-thresholding and projection operations and can be launched on graphics processing unit, which leads to relatively high performance. To evaluate the presented algorithm, both qualitative and quantitative studies were performed using various few view datasets. The results illustrated that the proposed approach yielded better reconstruction quality and satisfied convergence property compared with TV-based methods. [ABSTRACT FROM AUTHOR]

Iterative reconstruction algorithms for computed tomography (CT) through total variation (TV) regularization can provide accurate and stable reconstruction results. TV minimization is the L1-norm of gradient-magnitude images and can be regarded as a convex relaxation method to replace the L0 norm. In this study, a fast and efficient algorithm, which is named a weighted difference of L1 and L2 (L1 - αL2) on the gradient minimization, was proposed and investigated. The new algorithm provides a better description of sparsity for the optimization-based algorithms than TV minimization algorithms. The alternating direction method is an efficient method to solve the proposed model, which is utilized in this study. Both simulations and real CT projections were tested to verify the performances of the proposed algorithm. In the simulation experiments, the reconstructions from the proposed method provided better image quality than TV minimization algorithms with only 7 views in 180 degrees, which is also computationally faster. Meanwhile, the new algorithm enabled to achieve the final solution with less iteration numbers. [ABSTRACT FROM AUTHOR]

Nowadays, diversities of task-specific applications for computed tomography (CT) have already proposed multiple challenges for algorithm design of image reconstructions. Consequently, efficient algorithm design tool is necessary to be established. A fast and efficient algorithm design framework for CT image reconstruction, which is based on alternating direction method (ADM) with ordered subsets (OS), is proposed, termed as OS-ADM. The general ideas of ADM and OS have been abstractly introduced and then they are combined for solving convex optimizations in CT image reconstruction. Standard procedures are concluded for algorithm design which contain 1) model mapping, 2) sub-problem dividing and 3) solving, 4) OS level setting and 5) algorithm evaluation. Typical reconstruction problems are modeled as convex optimizations, including (non-negative) least-square, constrained L1 minimization, constrained total variation (TV) minimization and TV minimizations with different data fidelity terms. Efficient working algorithms for these problems are derived with detailed derivations by the proposed framework. In addition, both simulations and real CT projections are tested to verify the performances of two TV-based algorithms. Experimental investigations indicate that these algorithms are of the state-of-the-art performances. The algorithm instances show that the proposed OS-ADM framework is promising for practical applications. [ABSTRACT FROM AUTHOR]

The optimization-based method that utilizes the additional sparse prior of region-of-interest (ROI) image, such as total variation, has been the subject of considerable research in problems of interior tomography reconstruction. One challenge for optimization-based iterative ROI image reconstruction is to build the relationship between ROI image and truncated projection data. When the reconstruction support region is smaller than the original object, an unsuitable representation of data fidelity may lead to bright truncation artifacts in the boundary region of field of view. In this work, we aim to develop an iterative reconstruction method to suppress the truncation artifacts and improve the image quality for direct ROI image reconstruction. A novel reconstruction approach is proposed based on an optimization problem involving a two-step filtering-based data fidelity. Data filtering is achieved in two steps: the first takes the derivative of projection data; in the second step, Hilbert filtering is applied in the differentiated data. Numerical simulations and real data reconstructions have been conducted to validate the new reconstruction method. Both qualitative and quantitative results indicate that, as theoretically expected, the proposed method brings reasonable performance in suppressing truncation artifacts and preserving detailed features. The presented local reconstruction method based on the two-step filtering strategy provides a simple and efficient approach for the iterative reconstruction from truncated projections. [ABSTRACT FROM AUTHOR]

Iterative reconstruction algorithms for computed tomography (CT) through total variation regularization based on piecewise constant assumption can produce accurate, robust, and stable results. Nonetheless, this approach is often subject to staircase artefacts and the loss of fine details. To overcome these shortcomings, we introduce a family of novel image regularization penalties called total generalized variation (TGV) for the effective production of high-quality images from incomplete or noisy projection data for 3D reconstruction. We propose a new, fast alternating direction minimization algorithm to solve CT image reconstruction problems through TGV regularization. Based on the theory of sparse-view image reconstruction and the framework of augmented Lagrange function method, the TGV regularization term has been introduced in the computed tomography and is transformed into three independent variables of the optimization problem by introducing auxiliary variables. This new algorithm applies a local linearization and proximity technique to make the FFT-based calculation of the analytical solutions in the frequency domain feasible, thereby significantly reducing the complexity of the algorithm. Experiments with various 3D datasets corresponding to incomplete projection data demonstrate the advantage of our proposed algorithm in terms of preserving fine details and overcoming the staircase effect. The computation cost also suggests that the proposed algorithm is applicable to and is effective for CBCT imaging. Theoretical and technical optimization should be investigated carefully in terms of both computation efficiency and high resolution of this algorithm in application-oriented research. [ABSTRACT FROM AUTHOR]

Statistical model is essential for constraint-free visual image reconstruction, as it may overfit training data and have poor generalization. In this study, we investigate the sparsity of the distributed patterns of visual representation and introduce a suitable sparse model for the visual image reconstruction experiment. We use elastic net regularization to model the sparsity of the distributed patterns for local decoder training. We also investigate the relationship between the sparsity of the visual representation and sparse models with different parameters. Our experimental results demonstrate that the sparsity needed by visual reconstruction models differs from the sparsest one, and the l2-norm regularization introduced in the EN model improves not only the robustness of the model but also the generalization performance of the learning results. We therefore conclude that the sparse learning model for visual image reconstruction should reflect the spasity of visual perceptual experience, and have a solution with high but not the highest sparsity, and some robustness as well. [ABSTRACT FROM AUTHOR]

In medical imaging many conventional regularization methods, such as total variation or total generalized variation, impose strong prior assumptions which can only account for very limited classes of images. A more reasonable sparse representation frame for images is still badly needed. Visually understandable images contain meaningful patterns, and combinations or collections of these patterns can be utilized to form some sparse and redundant representations which promise to facilitate image reconstructions. In this work, we propose and study block matching sparsity regularization (BMSR) and devise an optimization program using BMSR for computed tomography (CT) image reconstruction for an incomplete projection set. The program is built as a constrained optimization, minimizing the L1-norm of the coefficients of the image in the transformed domain subject to data observation and positivity of the image itself. To solve the program efficiently, a practical method based on the proximal point algorithm is developed and analyzed. In order to accelerate the convergence rate, a practical strategy for tuning the BMSR parameter is proposed and applied. The experimental results for various settings, including real CT scanning, have verified the proposed reconstruction method showing promising capabilities over conventional regularization. [ABSTRACT FROM AUTHOR]

Accurate reconstruction from a reduced data set is highly essential for computed tomography in fast and/or low dose imaging applications. Conventional total variation (TV)-based algorithms apply the L1 norm-based penalties, which are not as efficient as Lp () quasi-norm-based penalties. TV with a p-th power-based norm can serve as a feasible alternative of the conventional TV, which is referred to as total p-variation (TpV). This paper proposes a TpV-based reconstruction model and develops an efficient algorithm. The total p-variation and Kullback–Leibler (KL) data divergence, which has better noise suppression capability compared with the often-used quadratic term, are combined to build the reconstruction model. The proposed algorithm is derived by the alternating direction method (ADM) which offers a stable, efficient, and easily coded implementation. We apply the proposed method in the reconstructions from very few views of projections (7 views evenly acquired within 180°). The images reconstructed by the new method show clearer edges and higher numerical accuracy than the conventional TV method. Both the simulations and real CT data experiments indicate that the proposed method may be promising for practical applications. [ABSTRACT FROM AUTHOR]

The additional sparse prior of images has been the subject of much research in problems of sparse-view computed tomography (CT) reconstruction. A method employing the image gradient sparsity is often used to reduce the sampling rate and is shown to remove the unwanted artifacts while preserve sharp edges, but may cause blocky or patchy artifacts. To eliminate this drawback, we propose a novel sparsity exploitation-based model for CT image reconstruction. In the presented model, the sparse representation and sparsity exploitation of both gradient and nonlocal gradient are investigated. The new model is shown to offer the potential for better results by introducing a similarity prior information of the image structure. Then, an effective alternating direction minimization algorithm is developed to optimize the objective function with a robust convergence result. Qualitative and quantitative evaluations have been carried out both on the simulation and real data in terms of accuracy and resolution properties. The results indicate that the proposed method can be applied for achieving better image-quality potential with the theoretically expected detailed feature preservation. [ABSTRACT FROM AUTHOR]

With the development of the compressive sensing theory, the image reconstruction from the projections viewed in limited angles is one of the hot problems in the research of computed tomography technology. This paper develops an iterative algorithm for image reconstruction, which can fit most cases. This method gives an image reconstruction flow with the difference image vector, which is based on the concept that the difference image vector between the reconstructed and the reference image is sparse enough. Then the l1-norm minimization method is used to reconstruct the difference vector to recover the image for flat subjects in limited angles. The algorithm has been tested with a thin planar phantom and a real object in limited-view projection data. Moreover, all the studies showed the satisfactory results in accuracy at a rather high reconstruction speed. [ABSTRACT FROM AUTHOR]

Iterative image reconstruction (IIR) with sparsity-exploiting methods, such as total variation (TV) minimization, claims potentially large reductions in sampling requirements. However, the computation complexity becomes a heavy burden, especially in 3D reconstruction situations. In order to improve the performance for iterative reconstruction, an efficient IIR algorithm for cone-beam computed tomography (CBCT) with GPU implementation has been proposed in this paper. In the first place, an algorithm based on alternating direction total variation using local linearization and proximity technique is proposed for CBCT reconstruction. The applied proximal technique avoids the horrible pseudoinverse computation of big matrix which makes the proposed algorithm applicable and efficient for CBCT imaging. The iteration for this algorithm is simple but convergent. The simulation and real CT data reconstruction results indicate that the proposed algorithm is both fast and accurate. The GPU implementation shows an excellent acceleration ratio of more than 100 compared with CPU computation without losing numerical accuracy. The runtime for the new 3D algorithm is about 6.8 seconds per loop with the image size of 256 × 256 × 256 and 36 projections of the size of 512 × 512. [ABSTRACT FROM AUTHOR]