Search

Medical image segmentation is fundamental and essential for the analysis of medical images. Although prevalent success has been achieved by convolutional neural networks (CNN), challenges are encountered in the domain of medical image analysis by two aspects: 1) lack of discriminative features to handle similar textures of distinct structures and 2) lack of selective features for potential blurred boundaries in medical images. In this paper, we extend the concept of contrastive learning (CL) to the segmentation task to learn more discriminative representation. Specifically, we propose a novel patch-dragsaw contrastive regularization (PDCR) to perform patch-level tugging and repulsing. In addition, a new structure, namely uncertainty-aware feature re- weighting block (UAFR), is designed to address the potential high uncertainty regions in the feature maps and serves as a better feature re- weighting. Our proposed method achieves state-of-the-art results across 8 public datasets from 6 domains. Besides, the method also demonstrates robustness in the limited-data scenario. The code is publicly available at https://github.com/lzh19961031/PDCR_UAFR-MIShttps://github.com/lzh19961031/PDCR_UAFR-MIS. [ABSTRACT FROM AUTHOR]

Optical flow estimation is a fundamental task in computer vision and image processing. Due to the difficulty in obtaining the ground truth of flow field, unsupervised learning approaches attract more and more research interests in recent years. However, despite of their good generalization capability, unsupervised optical flow methods suffer in the scenarios with large displacement, small objects, and occlusions. In this work, we propose a novel optical flow network based on decoder with multi-scale kernels. Different from previous U-Net like or pyramidal methods, we design our network based on RAFT architecture that with a 4D correlation layer and recurrent decoder. More importantly, we incorporate three novel ideas with regard to the input, information processing and output of the update units improve the performance. Firstly, we utilize various motion-related information as input to the update units. Secondly, we propose a module of multi-scale update unit. Thirdly, for the final flow up-sampling procedure, we propose an image-guided up-sampling loss to guide the learning of up-sampling masks. Our model is trained by the occlusion-aware photometric loss, edge-aware smoothness loss, self-supervised loss, and image-guided up-sampling loss. Experimental results demonstrate that our model achieves the state-of-the-art performance on both Sintel and KITTI and outperforms other unsupervised optical flow methods remarkably. [ABSTRACT FROM AUTHOR]

The edge information plays a key role in the restoration of a depth map. Most conventional methods assume that the color image and depth map are consistent in edge areas. However, complex texture regions in the color image do not match exactly with edges in the depth map. In this paper, firstly, we point out that in most cases the consistency between normal map and depth map is much higher than that between RGB-D pairs. Then we propose a dual normal-depth regularization term to guide the restoration of depth map, which constrains the edge consistency between normal map and depth map back and forth. Moreover, considering the bimodal characteristic of weight distribution that exists in depth discontinuous areas, a reweighted graph Laplacian regularizer is proposed to promote this bimodal characteristic. And this regularization is incorporated into a unified optimization framework to effectively protect the piece-wise smoothness(PWS) characteristics of depth map. By treating depth image as graph signal, the weight between two nodes is adapted according to its content. The proposed method is tested for both noise-free and noisy cases, and is compared against the state-of-the-art methods on both synthesis and real captured datasets. Extensive experimental results demonstrate the superior performance of our method compared with most state-of-the-art works in terms of both objective and subjective quality evaluations. Specifically, our method is more effective on edge areas and more robust to noises. [ABSTRACT FROM AUTHOR]

Inter prediction is a crucial part of hybrid video coding frameworks, utilized to exploit the temporal redundancy in video sequences and improve the coding performance. During inter prediction, a predicted block is typically derived from reference pictures using motion estimation and motion compensation. To improve the coding performance of inter prediction, a neural network based enhancement to inter prediction (NNIP) is proposed in this paper. NNIP is composed of three networks, namely residue estimation network, combination network, and deep refinement network. Specifically, first, a residue estimation network is designed to estimate the residue between current block and its predicted block using their available spatial neighbors. Second, the feature maps of the estimated residue and the predicted block are extracted and concatenated in a combination network. Finally, the concatenated feature maps are fed into a deep refinement network to generate a refined residue, which is added back to the predicted block to derive a more accurate predicted block. NNIP is integrated in HEVC to evaluate its efficiency. The experimental results demonstrate that NNIP can achieve 4.6%, 3.0%, and 2.7% BD-rate reduction on average under LDP, LDB, and RA configurations compared to HEVC. [ABSTRACT FROM AUTHOR]

In this letter, we compose a new three-stage deep convolutional neural network (NTSDCN) for image demosaicking, and it consists of our proposed Laplacian energy-constrained local residual unit (LC-LRU) and a feature-guided prior fusion unit (FG-PFU). Specifically, the LC-LRU is used to refine the learning target of the specific residual blocks in the network and enhance the dominant information of the residual features. The FG-PFU is designed to guide the feature extraction of the red (R) and blue (B) channels by utilizing prior information from the reconstructed green (G) channel. In our proposed NTSDCN, we recover the G channel image in the first stage with the CFA image and reconstruct the R and B images in the second stage. Finally, we fine-tune the resulting R, G and B images in the third stage to compose a full-color RGB image. The experimental results show that our proposed method achieves better performance than the state-of-the-art methods. The code is available at https://github.com/wyannn/NTSDCN. [ABSTRACT FROM AUTHOR]

Image paragraph generation aims to produce a complete description of a given image. This task is more challenging than image captioning, which only generates one sentence to describe the entire image. Traditional paragraph generation methods usually produce paragraph descriptions based on individual regions that are detected by a Region Proposal Network (RPN). However, relationships among visual objects are either ignored or utilized in an implicit manner in previous work. In this paper, we attempt to explore more visual information through a novel paragraph generation network that explicitly incorporates visual relationship semantics when producing descriptions. First, a novel Relation Pair Generative Adversarial Network (RP-GAN) is designed to locate regions that may cover subjective or objective elements. Then, their relationships are inferred through an attention-based network. Finally, the visual features and relationship semantics of valid relation pairs are taken as inputs by a Long Short-Term Memory (LSTM) network for generating sentences. The experimental results show that by explicitly utilizing the predicted relationship information, our proposed method obtains more accurate and informative paragraph descriptions than previous methods. [ABSTRACT FROM AUTHOR]

Visually unpleasant compression artifacts frequently appear in block-based transform coding, especially at low bit rates. This paper presents a new artifact reduction scheme based on Bayesian sparse modeling and artifacts-free PCA basis learning. To avoid the effect of blocking artifacts, we propose to learn artifacts-free PCA basis from clean images. We concatenate the clean patches and their compressed counterparts to learn paired distribution prior via the Gaussian Mixture Model (GMM). By this way, the GMM characterizes the mapping between the clean image and its compressed version. To restore a compressed patch, the best matched GMM component is assigned using the patch in the compressed image subspace. The artifacts-free PCA basis is obtained according to the mapping learned by the paired GMM. In practice, the statistical distributions of different sparse coefficients in different patches may dramatically vary with image contents. Instead of using a global zero-mean distribution for all coefficients, we propose to adaptively model the prior of each band in a Bayesian framework. The expectation and variance of each band are adaptively learned from the similar patches within the image. Thus, different transform bands are regularized unequally according to the learned priors. Experimental results show that the proposed scheme outperforms most of the compared schemes in terms of both objective quality and perceptual quality. [ABSTRACT FROM AUTHOR]

Block transform coded images usually suffer from annoying artifacts at low bit-rates, because of the independent quantization of DCT coefficients. Image prior models play an important role in compressed image reconstruction. Natural image patches in a small neighborhood of the high-dimensional image space usually exhibit an underlying sub-manifold structure. To model the distribution of signal, we extract sub-manifold structure as prior knowledge. We utilize graph Laplacian regularization to characterize the sub-manifold structure at patch level. And similar patches are exploited as samples to estimate distribution of a particular patch. Instead of using Euclidean distance as similarity metric, we propose to use graph-domain distance to measure the patch similarity. Then we perform low-rank regularization on the similar-patch group, and incorporate a non-convex $l_{p}$ penalty to surrogate matrix rank. Finally, an alternatively minimizing strategy is employed to solve the non-convex problem. Experimental results show that our proposed method is capable of achieving more accurate reconstruction than the state-of-the-art methods in both objective and perceptual qualities. [ABSTRACT FROM AUTHOR]

Wavelet transform is a powerful tool for multiresolution time-frequency analysis. It has been widely adopted in many image processing tasks, such as denoising, enhancement, fusion, and especially compression. Wavelets lead to the successful image coding standard JPEG-2000. Traditionally, wavelets were designed from the signal processing theory with certain assumption on the signal, but natural images are not as ideal as assumed by the theory. How to design content-adaptive wavelets for natural images remains a difficulty. Inspired by the recent progress of convolutional neural network (CNN), we propose iWave as a framework for deriving wavelet-like transform that is more suitable for natural image compression. iWave adopts an update-first lifting scheme, where the prediction filter is a trained CNN, to achieve wavelet-like transform. The CNN can be embedded into a deep network that is analogous to an auto-encoder, which is trained end-to-end. The trained wavelet-like transform still possesses the lifting structure, which ensures perfect reconstruction, supports multiresolution analysis, and is more interpretable than the deep networks trained as “black boxes.” We perform experiments to verify the generality as well as the speciality of iWave in comparison with JPEG-2000. When trained with a generic set of natural images and tested on the Kodak dataset, iWave achieves on average 4.4% and up to 14% BD-rate reductions. When trained and tested with a specific kind of textures, iWave provides as high as 27% BD-rate reduction. [ABSTRACT FROM AUTHOR]

G-Cast is a wireless visual communication scheme that conveys visual information via image gradient. It is inspired by the characteristics of human vision systems and can provide improved perceptual quality. G-Cast is power efficient but bandwidth demanding, because gradient data have double the size of the original image. This paper presents a scheme named CG-Cast for scalable image transmission in bandwidth-limited wireless scenarios. It employs a compressive-gradient-based image representation to describe perceptually sensitive image details and reduce the bandwidth requirement at the same time, combining gradient-based visual representation with compressive sensing techniques. The compressive gradient data are transmitted in a pseudo-analog way so that it achieves elegant quality transition in a wide channel signal-to-noise ratio (CSNR) range. CG-Cast also sends a small set of low-frequency data in digital a way to provide the global and local luminance of the image. We developed an effective optimization algorithm for the decoder to reconstruct the original image from the received noisy compressive gradient and the low-frequency part of the image. Experimental results demonstrate that the proposed scheme improves the quality of received images remarkably under different CSNR and channel bandwidth conditions. [ABSTRACT FROM AUTHOR]

In image restoration tasks, image priors generally utilize correlation within image contents to predict the latent image signal. In this paper, we propose to jointly exploit both intra- and inter-patch correlation of the input image, so as to further reduce the uncertainty of the unknown signal, and thus improve the prediction of the latent image. The proposed scheme evolves from the low-rank regularization for non-local highly-correlated image contents. Since the underlying cost function to pursue minimal rank is hard to solve, we use non-convex smooth surrogates for the rank penalty. Two such surrogates are utilized in order to incorporate both intra- and inter-patch correlation. To tackle the optimization problem, we use iterative alternating direction technique to divide the problem into two subproblems, each of which is solved via an empirical Bayesian procedure built upon variational approximation. Experimental results on image denoising show that the proposed approach outperforms several state-of-the-art methods in terms of peak signal-to-noise ratio, structural similarity, and perceptual quality. [ABSTRACT FROM AUTHOR]