Search

In hyperspectral image (HSI) processing, a fundamental issue is to restore HSI data from various degradations such as noise corruption and information missing. However, most existing methods more or less ignore the abundant prior knowledge on HSIs and the embedded noise, leading to suboptimal performance in practice. In this article, we propose a novel HSI restoration method by fully considering the intrinsic image structures and the complex noise characteristics. For HSIs, the global correlation is captured by the Kronecker-basis-representation-based tensor low-rankness measure, which integrates the insights delivered by both CP and Tucker decompositions; the local regularity is depicted by a plug-and-play spatial-spectral convolutional neural network with strong fitting ability to complex image features. For realistic noise, its statistical characteristics are encoded by a nonidentical and nonindependent distributed mixture of Gaussians distribution with flexible fitting capability. Then, we incorporate these image and noise priors into a probabilistic model based on the maximum a posteriori principle, and develop a solving scheme by combining expectation-maximization and alternating direction method of multipliers. Extensive experimental results on both simulated and real scenarios demonstrate the effectiveness of the proposed method and its superiority over the compared state-of-the- arts.

As low-rank modeling has achieved great success in tensor recovery, many research efforts devote to defining the tensor rank. Among them, the recent popular tensor tubal rank, defined based on the tensor singular value decomposition (t-SVD), obtains promising results. However, the framework of the t-SVD and the tensor tubal rank are applicable only to three-way tensors and lack of flexibility to handle different correlations along different modes. To tackle these two issues, we define a new tensor unfolding operator, named mode-$k_1k_2$ tensor unfolding, as the process of lexicographically stacking the mode-$k_1k_2$ slices of an $N$-way tensor into a three-way tensor, which is a three-way extension of the well-known mode-$k$ tensor matricization. Based on it, we define a novel tensor rank, the tensor $N$-tubal rank, as a vector whose elements contain the tubal rank of all mode-$k_1k_2$ unfolding tensors, to depict the correlations along different modes. To efficiently minimize the proposed $N$-tubal rank, we establish its convex relaxation: the weighted sum of tensor nuclear norm (WSTNN). Then, we apply WSTNN to low-rank tensor completion (LRTC) and tensor robust principal component analysis (TRPCA). The corresponding WSTNN-based LRTC and TRPCA models are proposed, and two efficient alternating direction method of multipliers (ADMM)-based algorithms are developed to solve the proposed models. Numerical experiments demonstrate that the proposed models significantly outperform the compared ones.

Recently, the tensor train (TT) rank has received much attention for tensor completion, due to its ability to explore the global low-rankness of tensors. However, existing methods still leave room for improvement, since the low-rankness itself is generally not sufficient to recover the underlying data. Inspired by this, we consider a novel tensor completion model by simultaneously exploiting the global low-rankness and local smoothness of visual data. In particular, we use low-rank matrix factorization to characterize the global TT low-rankness, and framelet and total variation regularization to enhance the local smoothness. We develop an efficient proximal alternating minimization algorithm to solve the proposed new model with guaranteed convergence. Extensive experiments on various data demonstrated that the proposed method outperforms compared methods in terms of visual and quantitative measures.

The tensor tubal rank, defined based on the tensor singular value decomposition (t-SVD), has obtained promising results in hyperspectral image (HSI) denoising. However, the framework of the t-SVD lacks flexibility for handling different correlations along different modes of HSIs, leading to suboptimal denoising performance. This article mainly makes three contributions. First, we introduce a new tensor rank named tensor fibered rank by generalizing the t-SVD to the mode- ${k}$ t-SVD, to achieve a more flexible and accurate HSI characterization. Since directly minimizing the fibered rank is NP-hard, we suggest a three-directional tensor nuclear norm (3DTNN) and a three-directional log-based tensor nuclear norm (3DLogTNN) as its convex and nonconvex relaxation to provide an efficient numerical solution, respectively. Second, we propose a fibered rank minimization model for HSI mixed noise removal, in which the underlying HSI is modeled as a low-fibered-rank component. Third, we develop an efficient alternating direction method of multipliers (ADMMs)-based algorithm to solve the proposed model, especially, each subproblem within ADMM is proven to have a closed-form solution, although 3DLogTNN is nonconvex. Extensive experimental results demonstrate that the proposed method has superior denoising performance, as compared with the state-of-the-art competing methods on low-rank matrix/tensor approximation and noise modeling.

In this paper, we propose a novel model for remote sensing images destriping, which includes the Schatten 1 ∕ 2 -norm and the unidirectional first-order and high-order total variation regularization. The main idea is that the stripe layer is low-rank, and the desired image possesses smoothness across stripes. Therefore, we use the Schatten 1 ∕ 2 -norm regularization to depict the low-rankness of stripes, and use the unidirectional total variation and the unidirectional high-order total variation to guarantee the smoothness of the underlying image. We develop the alternating direction method of multipliers algorithm to solve the proposed model. Extensive experiments on synthetic and real data are reported to show the superiority of the proposed method over state-of-the-art methods in terms of both quantitative and qualitative assessments.

Destriping has attracted a lot of interest in the field of remote sensing images processing. Recent research works consider the characteristics of stripes, among which low-rankness and group sparsity regularizations are widely used. However, for real scenarios, these regularizations sometimes fail to work when the underlying assumptions are violated. Motivated by this observation, we propose a novel and flexible regularization, named as the reweighted block sparsity (RBS), for complex stripes (e.g., the partial stripes). RBS divides the stripe layer into several blocks along the stripe-perpendicular direction and boosts the group sparsity of each block, where the sparsity level is adaptively controlled by updating weights. Based on RBS, we further propose a destriping model by integrating RBS and unidirectional total variation regularization, which can better detect and remove complex stripes. Moreover, we develop an alternating direction method of multipliers algorithm to solve the proposed model. Experimental results demonstrate that the proposed method outperforms the state-of-the-art competitors qualitatively and quantitatively.

Low-rank modeling has achieved great success in tensor completion. However, the low-rank prior is not sufficient for the recovery of the underlying tensor, especially when the sampling rate (SR) is extremely low. Fortunately, many real world data exhibit the piecewise smoothness prior along both the spatial and the third modes (e.g., the temporal mode in video data and the spectral mode in hyperspectral data). Motivated by this observation, we propose a novel low-rank tensor completion model using smooth matrix factorization (SMF-LRTC), which exploits the piecewise smoothness prior along all modes of the underlying tensor by introducing smoothness constraints on the factor matrices. An efficient block successive upper-bound minimization (BSUM)-based algorithm is developed to solve the proposed model. The developed algorithm converges to the set of the coordinate-wise minimizers under some mild conditions. Extensive experimental results demonstrate the superiority of the proposed method over the compared ones.

Recently, the tensor nuclear norm (TNN), which is the convex surrogate for tensor multi rank, has been successfully applied to the low-rank tensor completion (LRTC) problem. However, treating each singular value equally restricts the capability of TNN. In this work, we suggest the Laplace function based surrogate for tensor multi rank, which adaptively assigns weights to different singular values. We propose the corresponding surrogate based LRTC model and develop an efficient alternating direction method of multipliers (ADMM) to tackle the proposed model. Extensive experiments demonstrate that the proposed method outperforms state-of-the-art methods both quantitatively and qualitatively.

Rain streak removal is an important issue of the outdoor vision system and has been investigated extensively. In this paper, we propose a novel tensor optimization model for video rain streak removal by fully considering the discriminatively intrinsic characteristics of the rain streaks and the clean video. In specific, the rain streaks are group sparse and smooth along the rain streaks direction; the clean video is smooth along the perpendicular direction of the rain streaks and the temporal direction. For the rain streaks, we use l 2 , 1 norm to characterize the group sparsity and the unidirectional total variation (UTV) to promote the smoothness along the rain streaks direction. For the clean video, we use two UTV regularizers to enhance the smoothness along the perpendicular direction of the rain streaks and the temporal direction. To solve the proposed model we develop an efficient alternating direction method of multipliers (ADMM) algorithm. Experiments on synthetic and real data demonstrate the superiority of the proposed method over state-of-the-art methods both quantitatively and qualitatively.

In this paper, we propose a novel model to restore an image corrupted by blur and Cauchy noise. The model is composed of a data fidelity term and two regularization terms including total variation and high-order total variation. Total variation provides well-preserved edge features, but suffers from staircase effects in smooth regions, whereas high-order total variation can alleviate staircase effects. Moreover, we introduce a strategy for adaptively selecting regularization parameters. We develop an efficient alternating minimization algorithm for solving the proposed model. Numerical examples suggest that the proposed method has the advantages of better preserving edges and reducing staircase effects.

Retinex aims at estimating real reflectance images by removing the effect of illumination. We propose a nonconvex variational model for Retinex with novel priors for reflectance and illumination. Based on the statistics of the gradients of reflectance and illumination, we use the hyper-Laplacian prior to characterize the gradients of reflectance, and the hybrid hyper-Laplacian and Tikhonov prior to characterize the gradients of illumination. An efficient alternating direction method of multipliers (ADMM) is developed to solve the proposed model. Extensive numerical experiments show that the proposed method is comparable to the state-of-the-art methods quantitatively and qualitatively.

Remote sensing hyperspectral images (HSIs) are inevitably corrupted by several types of noise in the process of acquisition and transmission. In this paper, we propose a non-convex low-rank tensor approximation (NonLRTA) model for mixed noise removal, which can estimate the intrinsic structure of the underlying HSI from its noisy observation. The clean HSI component is characterized by the ϵ -norm, which is a non-convex surrogate to Tucker rank. The mixed noise is modeled as the sum of sparse and Gaussian components, which are regularized by the l 1 -norm and the Frobenius norm, respectively. An efficient augmented Lagrange multiplier (ALM) algorithm is developed to solve the proposed model. Experiments implemented on simulated and real HSIs validate the superiority of the proposed method, as compared to the state-of-the-art matrix-based and tensor-based methods.

We address the problem of restoring an original image from its blurry and noisy observation together with inaccurate information of the blurring process. For this purpose, we propose an enhanced regularized structured total least squares (RSTLS) model that can estimate the latent image and blur kernel simultaneously. In the proposed model, both the image and the blur kernel are characterized by a group-based low-rank prior, which assumes that a group of vectorized similar data patches can be well approximated by a low-rank matrix. We develop an alternating minimization algorithm to solve the proposed model efficiently. Numerical experiments demonstrate the effectiveness of our method in terms of both quantitative measures and visual quality.

Low-rankness has been widely exploited for the tensor completion problem. Recent advances have suggested that the tensor nuclear norm often leads to a promising approximation for the tensor rank. It treats the singular values equally to pursue the convexity of the objective function, while the singular values for the practical images have clear physical meanings with different importance and should be treated differently. In this paper, we propose a non-convex logDet function as a smooth approximation for tensor rank instead of the convex tensor nuclear norm and introduce it into the low-rank tensor completion problem. An alternating direction method of multiplier (ADMM)-based method is developed to solve the problem. Experimental results have shown that the proposed method can significantly outperform existing state-of-the-art nuclear norm-based methods for tensor completion.

Methods based on low-rank regularization have been successfully used to decompose an image into its cartoon and texture components. However, most of the existing low-rank regularized methods are formulated as a convex nuclear norm minimization, which is in practice suboptimal due to equally punishing each singular value. Recent works have shown that non-convex low-rank approximations adaptively treating the singular values at different scales yield better results than those convex ones. In this paper, we consider a non-convex log det function as the low-rank regularization to characterize the texture component in image decomposition, which treats singular values with varying degrees to facilitate a better characterization of the texture component. Then we obtain a non-convex cartoon–texture image decomposition model, where the cartoon and texture components are characterized simultaneously by minimizing the total variation norm and log det function. We integrate the self-similarity of texture component and the piecewise smooth of cartoon component into one model. The model can handle various types of image degradations, including blur, missing pixels and noise. Moreover, we develop an efficient alternating direction method of multiplier to solve the proposed model. The proposed method gives both a decomposition of cartoon and texture components and the restored image. Results of numerical experiments demonstrate the outstanding performance of the proposed method in image decomposition.

Retinex is a theory on simulating and explaining how human visual system perceives colors under different illumination conditions. The main contribution of this paper is to put forward a new convex optimization model for Retinex. Different from existing methods, the main idea is to rewrite a multiplicative form such that the illumination variable and the reflection variable are decoupled in spatial domain. The resulting objective function involves three terms including the Tikhonov regularization of the illumination component, the total variation regularization of the reciprocal of the reflection component, and the data-fitting term among the input image, the illumination component, and the reciprocal of the reflection component. We develop an alternating direction method of multipliers (ADMM) to solve the convex optimization model. Numerical experiments demonstrate the advantages of the proposed model which can decompose an image into the illumination and the reflection components.

We study a truncated difference of $l_1$ and $l_2$ norms as a nonconvex metric for recovering sparse vectors and low-rank matrices from linear measurements. By discarding large magnitude entries/singular values in penalization, the proposed metric, denoted as truncated $l_{1-2}$, achieves a nearly unbiased approximation of the vector sparsity/matrix rank. We establish exact and stable recovery conditions of truncated $l_{1-2}$ minimization under the restricted isometry property (RIP) framework. Computationally, we apply the difference of convex functions algorithm (DCA) to efficiently solve truncated $l_{1-2}$ minimization with guaranteed convergence. Our method is validated on sparse vector recovery, matrix completion, and magnetic resonance imaging (MRI) reconstruction, with performance comparable to the state of the art. Particularly for MRI reconstruction, it succeeds in reconstructing the $256\times 256$ Shepp--Logan phantom image from merely 7 radial lines.