Search

Highlights • Latent sharp image and the blur PSF are estimated jointly via a multi-functional optimization problem addressed by a first-order primal-dual solver. • This approach generates a multi-resolution pyramid of the input blurred image. • We introduce a PSF initiation method that benefits from the salient structure obtained via an l 0 gradient smoothing of blurred image. • A first-order primal-dual solution to the l 0 gradient smoothing problem is introduced in this paper. Abstract In this paper, a blind deblurring algorithm that is well converged towards the latent sharp image and the blur PSF is proposed. The proposed algorithm employs our recently proposed ringing suppression regularizer in a solution to the blind deblurring problem. Latent sharp image and the blur PSF are estimated jointly via a multi-functional optimization problem addressed by a first-order primal-dual solver. This approach generates a multi-resolution pyramid of the input blurred image and propagates the estimation results of the lower resolution levels to the higher resolution ones. By considering the PSF initiation as an important step in the blind deblurring process, we introduce a PSF initiation method that benefits from the salient structure obtained via an l 0 -gradient smoothing of blurred image. A first-order primal-dual solution to the l 0 -gradient smoothing problem is introduced in this paper. Experimental results show that the proposed deblurring algorithm significantly enhances the results favorably against other state-of- the-art image deblurring methods. [ABSTRACT FROM AUTHOR]

Highlights • A MRF-based inpainting algorithm for maintaining structure coherence is proposed. • Selecting proper offsets for energy function determines the inpainting results. • Appropriate direction structure feature is helpful to improve the inpainting performance and efficiency. Abstract Image inpainting provides a means for restoration of an image with some damaged portions and has been widely applied to many fields. However, many existing inpainting methods still suffer the difficulties of effectively and efficiently maintaining structure coherence. To tackle the problems, this paper proposes a direction structure distribution analysis strategy for Markov Random Field (MRF) -based image inpainting algorithms. In the strategy, the desired direction structures (DDS) are selected and applied to guide the inpainting process. Given a degraded image, four kinds of direction edge information are extracted by super-wavelet transform, canny and dilation operators. Then local direction edge gradient magnitude is applied for direction structure distribution analysis and DDS are selected according to gradient variance. Afterwards, offsets of similar patches are calculated and only a few dominated offsets are chosen as candidate offsets. Finally, graph-cut optimization technology is utilized to solve energy function. Experimental results show that the proposed method achieves generally better performance than nine state-of-the-art methods in terms of the abilities of maintaining structure coherence and the computational cost on inpainting different kinds of degraded images. [ABSTRACT FROM AUTHOR]

Single image super-resolution (SR) aims at generating a plausible and visually pleasing high-resolution (HR) image from a low-resolution (LR) input. In this paper, we propose an effective single image SR algorithm by using collaborative representation and exploiting non-local self-similarity of natural images. In particular, the collaborative-representation-based method is applied to build the so-called self-projection matrices from a training set of HR images. Then the learned self-projection matrices are used to establish the collaborative-representation-based regularization (CRR), which is responsible for introducing the external HR information. Furthermore, to guarantee a reliable estimation of the HR image, the non-local low-rank regularization (NLR) which exploits internal prior of images is also taken into consideration. Since the CRR term and NLR term are complementary, they are assembled together to form a new reconstruction-based framework for SR recovery. Finally, to implement the proposed framework, an iterative algorithm is designed to gradually improve the quality of the SR results. Extensive experimental results indicate that the proposed approach is capable of delivering higher quality of SR results than several state-of-the-art SR methods. [ABSTRACT FROM AUTHOR]

Eye fixation models, which try to quantitatively predict human eye attended areas in visual fields, have received increasing interest in recent years. In this paper, a novel framework is proposed for the detection of eye fixations. First, a multi-channel detection module, which extracts information of color contrast, salient object proposals and center bias from input image, is conducted to introduce various useful information into the subsequent fixations detection. In salient object detection channel, we employ the multi-instance learning (MIL) algorithm to determine which object proposal can attract attention, which avoids the fuzzyness of positive sample selection. Second, an adaptive weighted fusion method achieved by deep learning framework is proposed to fuse the multi-level information (i.e., contrast, objective, center bias) together for the detection of fixations, so that the integration of information between each level becomes more scientific. Finally, the detection result is optimized by embedding semantic information. Experimental results show that the algorithm has achieved competitive results in MIT1003, MIT300 and Toronto120 dataset. [ABSTRACT FROM AUTHOR]

In recent years face hallucination or super-resolution (SR) is getting much attention due to its wide applicability in real world scenarios. The existing SR methods and models perform well for noise free or small camera/atmospheric noisy faces. However, when suffering from mixed Impulse-Gaussian (MIG) noise, face hallucination becomes a challenging task. To address this problem, a novel error shrunk nearest neighbors representation (ESNNR) based face hallucination algorithm is proposed in this paper. Here, local content prior is incorporated to identify the high variance content (HVC) in the input images. The proposed algorithm suppresses the identified HVC in the input face to minimize the squared error. Moreover, the similarity matching between the input and training images is improved to achieve the locality and sparsity in the presence of MIG noise. Simulation results performed on public FEI, CAS-PEAL, CMU+MIT face databases, and locally captured surveillance video frames show that the proposed algorithm is computationally efficient, suitable for practical applications and give better performance than the existing face SR methods. [ABSTRACT FROM AUTHOR]

In multiscale transform (MST)-based multifocus image fusion, the fusion rules of different subbands are a significant factor that affects the fusion performance. However, dependence only on new fusion rule will see no significant performance gain for a MST-based method. To address this problem, this paper proposes two novel multifocus image fusion techniques based on multi-scale and multi-direction neighbor distance (MMND), in which the improvements of the fusion performance are respectively achieved by two new developed updating schemes. These two schemes are constructed according to the fact that the difference between a low quality fused result and the source image in the focused region is sharper than those generated by a high quality fused result. Based on this fact, the pixels of the source images are classified into three types in the updating mechanism, namely, pixels of focused significant regions, pixels of smooth regions, pixels of transition area between the focused and defocused regions. According to the categories of source images pixels, we can update the fused result produced by the MMND method in spatial and the MMND domain. Extensive experimental results validate that the proposed two fusion schemes can achieve better results than some state-of-the art algorithms. [ABSTRACT FROM AUTHOR]

• An image denoising algorithm based on an image-dependent basis inspired by the quantum many-body theory. • The local structural similarity of images is formalized through a term akin to interaction in quantum physics. • The versatility of the adaptive basis extends its scope of applications regardless of the image characteristics and noise statistics. [Display omitted] Sparse representation of real-life images is a very effective approach in imaging applications, such as denoising. In recent years, with the growth of computing power, data-driven strategies exploiting the redundancy within patches extracted from one or several images to increase sparsity have become more prominent. This paper presents a novel image denoising algorithm exploiting such an image-dependent basis inspired by the quantum many-body theory. Based on patch analysis, the similarity measures in a local image neighborhood are formalized through a term akin to interaction in quantum mechanics that can efficiently preserve the local structures of real images. The versatile nature of this adaptive basis extends the scope of its application to image-independent or image-dependent noise scenarios without any adjustment. We carry out a rigorous comparison with contemporary methods to demonstrate the denoising capability of the proposed algorithm regardless of the image characteristics, noise statistics and intensity. We illustrate the properties of the hyperparameters and their respective effects on the denoising performance, together with automated rules of selecting their values close to the optimal one in experimental setups with ground truth not available. Finally, we show the ability of our approach to deal with practical images denoising problems such as medical ultrasound image despeckling applications. [ABSTRACT FROM AUTHOR]

Learning-to-rank techniques have shown promising results in the domain of image ranking recently, where dimensionality reduction is a critical step to overcome the “curse of dimensionality”. However, conventional dimensionality reduction approaches cannot guarantee the satisfying performance because the important ranking information is ignored. This paper presents a novel “Ranking Dimensionality Reduction” scheme specifically designed for learning-to-rank based image ranking, which aims at not only discovering the intrinsic structure of data but also keeping the ordinal information. Within this scheme, a new dimensionality reduction algorithm called Relevance Marginal Fisher Analysis (RMFA) is proposed. RMFA models the proposed pairwise constraints of relevance-link and irrelevance-link into the relevance graph and the irrelevance graph, and applies the graphs to build the objective function with the idea of Marginal Fisher Analysis (MFA). Further, a semi-supervised RMFA algorithm called Semi-RMFA is developed to offer a more general solution for the real-world application. Extensive experiments are carried on two popular, real-world image search reranking datasets. The promising results demonstrate the robustness and effectiveness of the proposed scheme and methods. [ABSTRACT FROM AUTHOR]

Abstract: The basic theories and techniques in compressive sensing (CS) are established on the sampling and reconstruction of one-dimensional (1D) signals. When it is applied to two-dimensional (2D) images, the images are first stacked in a large vector. However, this vectorization not only destroys the spatial structure of the 2D image, but also increases computational complexity and memory requirements. As a result, some researchers proposed the concept of 2D CS. The major challenge of 2D CS is to design a reconstruction algorithm that can directly reconstruct the 2D image data from the 2D random projection. In this paper, a 2D CS sparse image reconstruction algorithm based on iterative gradient projection is proposed. In the proposed algorithm, the sparse solution is searched iteratively in the 2D solution space and then updated by gradient descent of the total variation (TV) and bivariate shrinkage in the dual-tree discrete wavelet transform (DDWT) domain. Numerous experiments are performed on several natural images. Compared with several state-of-the-art reconstruction algorithms, the proposed algorithm is more efficient and robust, not only yielding higher peak-signal-to-noise ratio but also reconstructing images of better subjective visual quality. [Copyright &y& Elsevier]

• In this paper, a multi-focus image fusion method based on two stage of convolutional neural network is proposed, which effectively solves two key problems of how to find the perfect decision map and deal with the boundary of the fused image. • In the first stage, an improved densenet model is used, which deepens the structure of the network to improves the ability to extract features and ensures that the gradient does not disappear. Besides, so as to prevent the loss of detailed information caused by the use of the pooling layer, most of the pooling layers are replaced with atrous convolution layers. • In addition, we create an appropriate training set for the improved densenet model. The traditional training set only uses the high-definition original image as the only focus patch, which can cause the network's generalization ability to deteriorate. So as to improve the accuracy of classification, a multi-version blurred dataset is applied, which fully simulates the situation of focusing and defocusing in reality, including 15 negative versions with label 0 and 15 positive versions with label 1. • In the second stage, an EDGAN model is utilized, which solves the problem of boundary blur or block effect caused by the decision map for fusion. To remove blurry edges in the fused image and ensure that the image generated by EDGAN does not lose texture features, five types of loss functions with different effects are proposed. • Besides, the performance of the network often depends on the quality of the training set. Therefore, we process the images from the COCO dataset to simulate boundary blur in the fused image. A large dataset based on the second stage is created using the above method. This paper proposes a two-stage convolutional neural network (CNN) fusion method that can obtain accurate decision map and deal with the problem of unclear fusion boundaries. In the first stage, an improved densenet is trained to classify whether the image patch is in focus or defocus, and then the corresponding fusion rule is utilized to acquire a perfect decision map. In addition, a multi-version blurred dataset is designed to improve the generalization ability of the network. In the second stage, edge-deblurring generative adversarial networks (EDGAN) is introduced to process the boundary. Furthermore, five different loss functions are applied to generate approving boundary deblurred images. At the same time, natural images are selected from the COCO dataset for special processing to simulate the boundary blurring situation to create the second-stage dataset. After two stages of processing, an image with rich details and decent fusion boundaries are attained. Experimental results demonstrate that the proposed algorithm is superior to other fusion algorithms in subjective vision and objective assessment. [ABSTRACT FROM AUTHOR]

• This article proposes a novel nonconvex TGV model for adaptive image restoration. • We develop in detail a highly efficient alternating direction method of multipliers. • The regularization parameter λ can be automatically estimated during the iteration. • Based on convex analysis, a novel convergence proof of our algorithm is presented. • Our scheme provide promising performance in restoration accuracy and visual quality. Overcoming the staircase effect and simultaneously preserving edge details is an important and challenging issue in image processing. To this aim, this paper investigates the nonconvex total generalized variation regularization model for image restoration. Numerically, a highly efficient alternating direction method of multipliers is constructed to deal with the optimization problem in detail, which closely incorporates the superiorities of iteratively reweighted ℓ 1 algorithm and variable splitting technique. The proposed strategy can efficiently achieve image restoration and adaptive parameter estimation by applying Morozov's discrepancy principle. Furthermore, the convergence property of our novel algorithm with the variable regularization parameter is presented. Finally, numerical simulations concertedly illustrate that our approach outperforms several state-of-the-art regularization models, in terms of reconstruction accuracy and edge-preserving ability. [ABSTRACT FROM AUTHOR]

• Novel DoPHTs are constructed and the performance of IoPHTs is improved effectively. • A color image triple zero-watermarking algorithm that can provide triple image copyright protection is proposed. • The proposed algorithm has improved resistance and high robustness to geometric attacks. • The proposed algorithm features a high security level. Existing zero-watermarking algorithms basically construct single zero-watermark information for the entire image and hence their efficiency is relatively low. In this paper, a color image triple zero-watermarking algorithm based on decimal-order polar harmonic transforms (DoPHTs) and chaotic system is proposed to achieve triple color image copyright protection, which is robust to geometric attacks. Firstly, the traditional integer-order polar harmonic transforms (IoPHTs) are extended to the decimal-order to construct DoPHTs, which effectively improve the performance of IoPHTs. Secondly, the three forms of DoPHTs, namely, decimal-order polar complex exponential transform (DoPCET), decimal-order polar cosine transform (DoPCT) and decimal-order polar sine transform (DoPST), are used to construct triple zero-watermark information. This algorithm uses geometrically invariant DoPHTs to improve its robustness to geometric attacks and uses the chaotic system to scramble logo image and feature images to enhance its security. The experimental results show that the proposed algorithm is highly robust to common image processing attacks and geometric attacks and is superior to other algorithms. [ABSTRACT FROM AUTHOR]

• Quaternion representation shows advantage in colour image processing. • Some quality-aware features to reflect the naturalness, structural, and texture of a colour in the quaternion domain are extracted. • An unsupervised blind colour image quality assessment model is proposed. • The proposed model generates high prediction accuracy and high robustness in evaluation. Without the limitations of labelled distorted images for training, the unsupervised blind colour image quality assessment (UBCIQA) model shows excellent generalization performance. However, existing UBCIQA methods fail to consider the correlation between different colour channels. This paper proposes an efficient UQBCIA algorithm. Placing R, G, and B channels of a colour image into the three imaginary parts of a quaternion, the proposed UQBCIA obtains the quaternion representation of a colour image, which allows UQBCIA to process the colour image as a whole while keeping its three colour channels dependent. The naturalness, structural and texture statistic features are then extracted to fit a multivariate Gaussian (MVG) model. The quality of the test colour image is calculated as the MVG distance between the fitted pristine images and the test image. The proposed method is evaluated from three aspects: (1) prediction accuracy, (2) computational complexity and (3) robustness on five colour databases. The results demonstrate that the proposed UQBCIA algorithm outperforms state-of-the-art unsupervised and classical supervised BIQA methods. [ABSTRACT FROM AUTHOR]

• Appropriate local blur scales are selected when image block is reblurred. • Sub-band fusion of DCT coefficients is proposed in calculating blur amount. • Edge blur maps are computed in multi-scale and multi-orientation image windows. • Blur points are added to initialize input image in image matting stage. In common natural image blur, objects that not lie in the focal length of a digital camera generate defocus areas in the photographed image. In this paper, we propose a novel edge-based method for spatially varying defocus blur detection based on reblurred DCT coefficients ratios of the corresponding local patches. This method selects appropriate local reblur scales while detecting the edge points to deal with the problem of different blur degree and texture richness of the local image blocks. A sub-band fusion method of DCT coefficients is proposed to expand the difference between DCT features of in-focus and out-of-focus regions. Edge points blur maps are computed in multi-scale and multi-orientation image windows and more blur points are added to initialize sparse blur maps, finally Matting Laplacian method is used along with multi-scale fusion algorithm to obtain a more accurate blur segmentation. Experimental results present the proposed method has strong advantages in image detail processing and outperforms state-of-the-art methods for blur detection. [ABSTRACT FROM AUTHOR]

• We propose a deep spectral convolution network with spectral library that can be applied for a series of HSIs after training. • We design a deeper network architecture to efficiently extract local spectral features and achieve better estimation results. • We construct a new loss function, which includes reconstruction error, abundance sparsity, and abundance cross-entropy. Spectral unmixing is an important task for hyperspectral remote sensing image processing, which infers the pure spectral signatures (endmembers) in hyperspectral image (HSI) and their corresponding fractions (abundances). Recently, deep learning has become a powerful tool for HSI analysis, such as HSI classification and HSI super-resolution. In this paper, we propose a new unmixing algorithm that uses the convolutional neural network (CNN) for hyperspectral data incorporating spectral library, which can be applied for a series of HSIs after training. The proposed deep spectral convolution network extracts features and then executes the estimating process from these extracted spectral characteristics to acquire the fractional abundances on a fixed spectral library. Meanwhile, considering the incorporation of spectral library, a deeper convolutional network has been adopted to achieve better results. Moreover, we construct a new loss function, which includes pixel reconstruction error, abundance sparsity, and abundance cross-entropy to train the aforementioned network in an end-to-end manner. Experiments on both simulated and real HSIs indicate the advantage of the proposed method, which can obviously enhance the abundance estimation accuracy. [ABSTRACT FROM AUTHOR]

• We develop a connection from GSC to low-rank matrix minimization via an effective dictionary learning approach, where the sparse representation coefficient vector is also the singular value vector of matrix in our proposed model, then we can convert the sparsity-inducing optimization problem into the low-rank matrix minimization problem. • For a more accurate approximation of the rank of group matrix, we develop a generalized low-rank minimization model. Our proposed model employs a generalized and flexible weighted scheme and a generalized nonconvex nonsmooth surrogate function on singular values of the group matrix. • To solve the generalized rank minimization problem for GSC, we will first convert the optimization model into a derivative double nonconvex nonsmooth rank (DNNR) minimization problem, and then develop an iterative reweighted singular-value function thresholding (IRSFT) algorithm for GSC. • We develop an alternative direction method of multipliers (ADMM) framework to integrate the proposed GSC model into IR problem, where the sparse vector for GSC and the group matrix of desired image will be achieved simultaneously. We evaluate the proposed framework for four classic IR problems, including image CS reconstruction, inpainting, deblurring, and impulsive noise removal. Recently, low-rank matrix recovery theory has been emerging as a significant progress for various image processing problems. Meanwhile, the group sparse coding (GSC) theory has led to great successes in image restoration (IR) problem with each group contains low-rank property. In this paper, we propose a novel low-rank minimization based denoising model for IR tasks under the perspective of GSC, an important connection between our denoising model and rank minimization problem has been put forward. To overcome the bias problem caused by convex nuclear norm minimization (NNM) for rank approximation, a more generalized and flexible rank relaxation function is employed, namely weighted nonconvex relaxation. Accordingly, an efficient iteratively-reweighted algorithm is proposed to handle the resulting minimization problem combing with the popular L 1/2 and L 2/3 thresholding operators. Finally, our proposed denoising model is applied to IR problems via an alternating direction method of multipliers (ADMM) strategy. Typical IR experiments on image compressive sensing (CS), inpainting, deblurring and impulsive noise removal demonstrate that our proposed method can achieve significantly higher PSNR/FSIM values than many relevant state-of-the-art methods. [ABSTRACT FROM AUTHOR]