Search

Local and non-local attention-based methods have been well studied in various image restoration tasks while leading to promising performance. However, most of the existing methods solely focus on one type of attention mechanism (local or non-local). Furthermore, by exploiting the self-similarity of natural images, existing pixel-wise non-local attention operations tend to give rise to deviations in the process of characterizing long-range dependence due to image degeneration. To overcome these problems, in this paper we propose a novel collaborative attention network (COLA-Net) for image restoration, as the first attempt to combine local and non-local attention mechanisms to restore image content in the areas with complex textures and with highly repetitive details respectively. In addition, an effective and robust patch-wise non-local attention model is developed to capture long-range feature correspondences through 3D patches. Extensive experiments on synthetic image denoising, real image denoising and compression artifact reduction tasks demonstrate that our proposed COLA-Net is able to achieve state-of-the-art performance in both peak signal-to-noise ratio and visual perception, while maintaining an attractive computational complexity. The source code is available on https://github.com/MC-E/COLA-Net., 11 pages, 6 tables, 9 figures, to be published in IEEE Transactions on Multimedia

ImportanceEnlarged perivascular spaces (ePVSs) have been associated with cerebral small-vessel disease (cSVD). Although their etiology may differ based on brain location, study of ePVSs has been limited to specific brain regions; therefore, their risk factors and significance remain uncertain.ObjectiveToperform a whole-brain investigation of ePVSs in a large community-based cohort.Design, Setting, and ParticipantsThis cross-sectional study analyzed data from the Atrial Fibrillation substudy of the population-based Multi-Ethnic Study of Atherosclerosis. Demographic, vascular risk, and cardiovascular disease data were collected from September 2016 to May 2018. Brain magnetic resonance imaging was performed from March 2018 to July 2019. The reported analysis was conducted between August and October 2022. A total of 1026 participants with available brain magnetic resonance imaging data and complete information on demographic characteristics and vascular risk factors were included.Main Outcomes and MeasuresEnlarged perivascular spaces were quantified using a fully automated deep learning algorithm. Quantified ePVS volumes were grouped into 6 anatomic locations: basal ganglia, thalamus, brainstem, frontoparietal, insular, and temporal regions, and were normalized for the respective regional volumes. The association of normalized regional ePVS volumes with demographic characteristics, vascular risk factors, neuroimaging indices, and prevalent cardiovascular disease was explored using generalized linear models.ResultsIn the 1026 participants, mean (SD) age was 72 (8) years; 541 (53%) of the participants were women. Basal ganglia ePVS volume was positively associated with age (β = 3.59 × 10−3; 95% CI, 2.80 × 10−3 to 4.39 × 10−3), systolic blood pressure (β = 8.35 × 10−4; 95% CI, 5.19 × 10−4 to 1.15 × 10−3), use of antihypertensives (β = 3.29 × 10−2; 95% CI, 1.92 × 10−2 to 4.67 × 10−2), and negatively associated with Black race (β = −3.34 × 10−2; 95% CI, −5.08 × 10−2 to −1.59 × 10−2). Thalamic ePVS volume was positively associated with age (β = 5.57 × 10−4; 95% CI, 2.19 × 10−4 to 8.95 × 10−4) and use of antihypertensives (β = 1.19 × 10−2; 95% CI, 6.02 × 10−3 to 1.77 × 10−2). Insular region ePVS volume was positively associated with age (β = 1.18 × 10−3; 95% CI, 7.98 × 10−4 to 1.55 × 10−3). Brainstem ePVS volume was smaller in Black than in White participants (β = −5.34 × 10−3; 95% CI, −8.26 × 10−3 to −2.41 × 10−3). Frontoparietal ePVS volume was positively associated with systolic blood pressure (β = 1.14 × 10−4; 95% CI, 3.38 × 10−5 to 1.95 × 10−4) and negatively associated with age (β = −3.38 × 10−4; 95% CI, −5.40 × 10−4 to −1.36 × 10−4). Temporal region ePVS volume was negatively associated with age (β = −1.61 × 10−2; 95% CI, −2.14 × 10−2 to −1.09 × 10−2), as well as Chinese American (β = −2.35 × 10−1; 95% CI, −3.83 × 10−1 to −8.74 × 10−2) and Hispanic ethnicities (β = −1.73 × 10−1; 95% CI, −2.96 × 10−1 to −4.99 × 10−2).Conclusions and RelevanceIn this cross-sectional study of ePVSs in the whole brain, increased ePVS burden in the basal ganglia and thalamus was a surrogate marker for underlying cSVD, highlighting the clinical importance of ePVSs in these locations.

Feature dimensionality reduction plays an important role in radiomic studies with a large number of features. However, conventional radiomic approaches may suffer from noise, and feature dimensionality reduction techniques are not equipped to utilize latent supervision information of patient data under study, such as differences in patients, to learn discriminative low dimensional representations. To achieve robustness to noise and feature dimensionality reduction with improved discriminative power, we develop a robust collaborative clustering method to simultaneously cluster patients and radiomic features into distinct groups respectively under adaptive sparse regularization. Our method is built upon matrix tri-factorization enhanced by adaptive sparsity regularization for simultaneous feature dimensionality reduction and denoising. Particularly, latent grouping information of patients with distinct radiomic features is learned and utilized as supervision information to guide the feature dimensionality reduction, and noise in radiomic features is adaptively isolated in a Bayesian framework under a general assumption of Laplacian distributions of transform-domain coefficients. Experiments on synthetic data have demonstrated the effectiveness of the proposed approach in data clustering, and evaluation results on an fDg-PET/CT dataset of rectal cancer patients have demonstrated that the proposed method outperforms alternative methods in terms of both patient stratification and prediction of patient clinical outcomes.

Low-rank approximation-based methods have recently achieved impressive results in image restoration. Generally, the low-rank constraint integrated with the nonlocal self-similarity prior is enforced for image recovery. However, it is still unsatisfactory to recover complex image structures due to the lack of joint modeling based on local and global information, especially when the signal-to-noise ratio is low. In this paper, we propose a novel structure-constrained low-rank approximation method using complementary local and global information, as, respectively, modeled by kernel Wiener filtering and low-rank regularization. The proposed method solves the ill-posed inverse problem associated with image denoising by the alternating direction method of multipliers. Experimental results demonstrate that the proposed method not only removes noise effectively, but also is highly competitive against the state-of-the-art methods both qualitatively and quantitatively.

ImportanceLittle is known about the associations of strict blood pressure (BP) control with microstructural changes in small vessel disease markers.ObjectiveTo investigate the regional associations of intensive vs standard BP control with small vessel disease biomarkers, such as white matter lesions (WMLs), fractional anisotropy (FA), mean diffusivity (MD), and cerebral blood flow (CBF).Design, Setting, and ParticipantsThe Systolic Blood Pressure Intervention Trial (SPRINT) is a multicenter randomized clinical trial that compared intensive systolic BP (SBP) control (SBP target InterventionsAt baseline, 355 participants received intensive SBP treatment and 315 participants received standard SBP treatment.Main Outcomes and MeasuresThe main outcomes were regional changes in WMLs, FA, MD (in white matter regions of interest), and CBF (in gray matter regions of interest).ResultsAt baseline, 355 participants (mean [SD] age, 67.7 [8.0] years; 200 men [56.3%]) received intensive BP treatment and 315 participants (mean [SD] age, 67.0 [8.4] years; 199 men [63.2%]) received standard BP treatment. Intensive treatment was associated with smaller mean increases in WML volume compared with standard treatment (644.5 mm3 vs 1258.1 mm3). The smaller mean increases were observed specifically in the deep white matter regions of the left anterior corona radiata (intensive treatment, 30.3 mm3 [95% CI, 16.0-44.5 mm3]; standard treatment, 80.5 mm3 [95% CI, 53.8-107.2 mm3]), left tapetum (intensive treatment, 11.8 mm3 [95% CI, 4.4-19.2 mm3]; standard treatment, 27.2 mm3 [95% CI, 19.4-35.0 mm3]), left superior fronto-occipital fasciculus (intensive treatment, 3.2 mm3 [95% CI, 0.7-5.8 mm3]; standard treatment, 9.4 mm3 [95% CI, 5.5-13.4 mm3]), left posterior corona radiata (intensive treatment, 26.0 mm3 [95% CI, 12.9-39.1 mm3]; standard treatment, 52.3 mm3 [95% CI, 34.8-69.8 mm3]), left splenium of the corpus callosum (intensive treatment, 45.4 mm3 [95% CI, 25.1-65.7 mm3]; standard treatment, 83.0 mm3 [95% CI, 58.7-107.2 mm3]), left posterior thalamic radiation (intensive treatment, 53.0 mm3 [95% CI, 29.8-76.2 mm3]; standard treatment, 106.9 mm3 [95% CI, 73.4-140.3 mm3]), and right posterior thalamic radiation (intensive treatment, 49.5 mm3 [95% CI, 24.3-74.7 mm3]; standard treatment, 102.6 mm3 [95% CI, 71.0-134.2 mm3]).Conclusions and RelevanceThis study suggests that intensive BP treatment, compared with standard treatment, was associated with a slower increase of WMLs, improved diffusion tensor imaging, and FA and CBF changes in several brain regions that represent vulnerable areas that may benefit from more strict BP control.Trial RegistrationClinicalTrials.gov Identifier: NCT01206062

This paper proposes a robust multi-frame video super-resolution (SR) scheme to obtain high SR performance under large upscaling factors. Although the reference low-resolution frames can provide complementary information for the high-resolution frame, an effective regularizer is required to rectify the unreliable information from the reference frames. As the high-frequency information is mostly contained in the image gradient field, we propose to learn the gradient-mapping function between the high-resolution (HR) and the low-resolution (LR) image to regularize the fusion of multiple frames. In contrast to the existing spatial-domain networks, we train a deep gradient-mapping network to learn the horizontal and vertical gradients. We found that adding the low-frequency information (mainly from the LR image) to the gradient-learning network can boost the performance of the network. A forward and backward motion field prior is used to regularize the estimation of the motion flow between frames. For robust SR reconstruction, a weighting scheme is proposed to exclude the outlier data. Visual and quantitative evaluations on benchmark datasets demonstrate that our method is superior to many state-of-the-art methods and can recover better details with less artifacts.

G-Cast is a wireless visual communication scheme that conveys visual information via image gradient. It is inspired by the characteristics of human vision systems and can provide improved perceptual quality. G-Cast is power efficient but bandwidth demanding, because gradient data have double the size of the original image. This paper presents a scheme named CG-Cast for scalable image transmission in bandwidth-limited wireless scenarios. It employs a compressive-gradient-based image representation to describe perceptually sensitive image details and reduce the bandwidth requirement at the same time, combining gradient-based visual representation with compressive sensing techniques. The compressive gradient data are transmitted in a pseudo-analog way so that it achieves elegant quality transition in a wide channel signal-to-noise ratio (CSNR) range. CG-Cast also sends a small set of low-frequency data in digital a way to provide the global and local luminance of the image. We developed an effective optimization algorithm for the decoder to reconstruct the original image from the received noisy compressive gradient and the low-frequency part of the image. Experimental results demonstrate that the proposed scheme improves the quality of received images remarkably under different CSNR and channel bandwidth conditions.

Image restoration techniques generally use intrinsic correlations of image signals to reduce the uncertainty of the unknown signal and estimate the latent ground truth. Local and non-local correlation are the two major kinds of correlations utilized. They are different sources of correlations reflecting connections between different image data, but such a difference is not taken into consideration in most of the existing schemes. Typically, sparse representation based works use the same image data to exploit both local and non-local correlation in shared regularization. This paper aims to fully exploit local and non-local correlation of image contents separately so that near-optimal sparse representations are achieved and thus the uncertainty of signals is minimized. The proposed scheme adaptively selects different image data to exploit local and non-local correlations respectively. In particular, to exploit local correlation, the image data of interest are extracted from clustered rows of patch groups that consist of similar image contents. Experimental results on image denoising show that the proposed scheme not only outperforms state-of-the-art sparsity and low rank based methods, but also surpasses successful deep learning-based approaches in terms of PSNR, SSIM, and visual quality.

The recently invented retina-inspired spike camera has shown great potential for capturing dynamic scenes. Different from the conventional digital cameras that compact the photoelectric information within the exposure interval into a single snapshot, the spike camera produces a continuous spike stream to record the dynamic light intensity variation process. For spike cameras, image reconstruction remains an important and challenging issue. To this end, this paper develops a spike-to-image neural network (Spk2ImgNet) to reconstruct the dynamic scene from the continuous spike stream. In particular, to handle the challenges brought by both noise and high-speed motion, we propose a hierarchical architecture to exploit the temporal correlation of the spike stream progressively. Firstly, a spatially adaptive light inference subnet is proposed to exploit the local temporal correlation, producing basic light intensity estimates of different moments. Then, a pyramid deformable alignment is utilized to align the intermediate features such that the feature fusion module can exploit the long-term temporal correlation, while avoiding undesired motion blur. In addition, to train the network, we simulate the working mechanism of spike camera to generate a large-scale spike dataset composed of spike streams and corresponding ground truth images. Experimental results demonstrate that the proposed network evidently outperforms the state-of-the-art spike camera reconstruction methods.

Lobar cerebral microbleeds (CMBs) and localized non-hemorrhage iron deposits in the basal ganglia have been associated with brain aging, vascular disease and neurodegenerative disorders. Particularly, CMBs are small lesions and require multiple neuroimaging modalities for accurate detection. Quantitative susceptibility mapping (QSM) derived from in vivo magnetic resonance imaging (MRI) is necessary to differentiate between iron content and mineralization. We set out to develop a deep learning-based segmentation method suitable for segmenting both CMBs and iron deposits. We included a convenience sample of 24 participants from the MESA cohort and used T2-weighted images, susceptibility weighted imaging (SWI), and QSM to segment the two types of lesions. We developed a protocol for simultaneous manual annotation of CMBs and non-hemorrhage iron deposits in the basal ganglia. This manual annotation was then used to train a deep convolution neural network (CNN). Specifically, we adapted the U-Net model with a higher number of resolution layers to be able to detect small lesions such as CMBs from standard resolution MRI. We tested different combinations of the three modalities to determine the most informative data sources for the detection tasks. In the detection of CMBs using single class and multiclass models, we achieved an average sensitivity and precision of between 0.84–0.88 and 0.40–0.59, respectively. The same framework detected non-hemorrhage iron deposits with an average sensitivity and precision of about 0.75–0.81 and 0.62–0.75, respectively. Our results showed that deep learning could automate the detection of small vessel disease lesions and including multimodal MR data (particularly QSM) can improve the detection of CMB and non-hemorrhage iron deposits with sensitivity and precision that is compatible with use in large-scale research studies.