Search

Missing information, such as dead pixel values and cloud effects, is very common image quality degradation problems in remote sensing. Missing information can reduce the accuracy of the subsequent image processing, in applications such as classification, unmixing, and target detection, and even the quantitative retrieval process. The main aim of this paper is to study an adaptive weighted tensor completion (AWTC) method for the recovery of remote sensing images with missing data. Our idea is to collectively make use of the spatial, spectral, and temporal information to build a new weighted tensor low-rank regularization model for recovering the missing data. In the model, the weights are determined adaptively by considering the contribution of the spatial, spectral, and temporal information in each dimension. Experimental results based on both simulated and real data sets are presented to verify that the proposed method can recover missing data, and its performance is found to be better than the other tested methods. In the simulated experiments, the peak signal-to-noise ratio is improved by more than 3 dB, compared with the original tensor completion model. In the real data experiments, the proposed AWTC model can better recover the dead line problem in Aqua Moderate Resolution Imaging Spectroradiometer band 6 and the scan-line corrector-off problem in enhanced thematic mapper plus images, with the smallest spectral distortion. [ABSTRACT FROM PUBLISHER]

Data assimilation is a process used to fuse information from priors, observations of nature, and numerical models, in order to obtain best estimates of the parameters or state of a physical system of interest. Presence of large errors in some observational data, e.g., data collected from a faulty instrument, negatively affect the quality of the overall assimilation results. This work develops a systematic framework for robust data assimilation. The new algorithms continue to produce good estimates of parameters or state in the presence of observation outliers. The approach is based on replacing the traditional L2 norm formulation of data assimilation problems with formulations based on L1 and Huber norms. Numerical experiments using the Lorenz-96 and the shallow water on the sphere models illustrate how the new algorithms outperform traditional data assimilation approaches in the presence of data outliers. [ABSTRACT FROM AUTHOR]

Outlier detection is an important data analysis task in its own right and removing the outliers from clusters can improve the clustering accuracy. In this paper, we extend the k -means algorithm to provide data clustering and outlier detection simultaneously by introducing an additional “cluster” to the k -means algorithm to hold all outliers. We design an iterative procedure to optimize the objective function of the proposed algorithm and establish the convergence of the iterative procedure. Numerical experiments on both synthetic data and real data are provided to demonstrate the effectiveness and efficiency of the proposed algorithm. [ABSTRACT FROM AUTHOR]

Over the past few decades, much attention has been drawn to large-scale incremental data analysis, where researchers are faced with huge amounts of high-dimensional data acquired incrementally. In such a case, conventional algorithms that compute the result from scratch whenever a new sample comes are highly inefficient. To conquer this problem, we propose a new incremental algorithm incremental regularized least squares (IRLS) that incrementally computes the solution to the regularized least squares (RLS) problem with multiple columns on the right-hand side. More specifically, for an RLS problem with c (c > 1) columns on the right-hand side, we update its unique solution by solving an RLS problem with a single column on the right-hand side whenever a new sample arrives, instead of solving an RLS problem with c columns on the right-hand side from scratch. As a direct application of IRLS, we consider the supervised dimensionality reduction of large-scale data and focus on linear discriminant analysis (LDA). We first propose a new batch LDA model that is closely related to the RLS problem, and then apply IRLS to develop a new incremental LDA algorithm. Experimental results on real-world datasets demonstrate the effectiveness and efficiency of our algorithms. [ABSTRACT FROM AUTHOR]

This paper proposes a subspace clustering algorithm with automatic feature grouping for clustering high-dimensional data. In this algorithm, a new component is introduced into the objective function to capture the feature groups and a new iterative process is defined to optimize the objective function so that the features of high-dimensional data are grouped automatically. Experiments on both synthetic data and real data show that the new algorithm outperforms the FG- k -means algorithm in terms of accuracy and choice of parameters. [ABSTRACT FROM AUTHOR]

This paper proposes a subspace clustering algorithm by introducing attribute weights in the affinity propagation algorithm. A new step is introduced to the affinity propagation process to iteratively update the attribute weights based on the current partition of the data. The relative magnitude of the attribute weights can be used to identify the subspaces in which clusters are embedded. Experiments on both synthetic data and real data show that the new algorithm outperforms the affinity propagation algorithm in recovering clusters from data. [ABSTRACT FROM AUTHOR]

Robust low-rank tensor completion plays an important role in multidimensional data analysis against different degradations, such as Gaussian noise, sparse noise, and missing entries, and has a variety of applications in image processing and computer vision. In this paper, we investigate the problem of low-rank tensor completion with different degradations for third-order tensors, and propose a transformed tensor nuclear norm method combined the tensor ℓ 1 norm with total variational (TV) regularization. Our model is based on a recently proposed algebraic framework in which the transformed tensor nuclear norm is introduced to capture lower transformed multi-rank by using suitable unitary transformations. We adopt the tensor ℓ 1 norm to detect the sparse noise, and the TV regularization to preserve the piecewise smooth structure along the spatial and tubal dimensions. Moreover, a symmetric Gauss–Seidel based alternating direction method of multipliers is developed to solve the resulting model and its global convergence is established under very mild conditions. Extensive numerical examples on both hyperspectral images and video datasets are carried out to demonstrate the superiority of the proposed model compared with several existing state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Abstract Most of existing image categorizations assume that the given datasets have a good resolution and quality. However, the assumption is often violated in real applications. In this paper, we study the low-resolution (LR) image categorization. By utilizing labeled high-resolution (HR) images as auxiliary information, we formulate the problem as a heterogeneous domain adaptation problem and propose a Discriminative Joint Distribution Adaptation (DJDA) model to solve it. The DJDA model projects both LR and HR images into an intermediate subspace with a well-designed objective function, where the distance between classes is expected to be enlarged and the distribution divergence to be reduced. As a result, the discriminative knowledge for HR images can be transferred effectively to LR images. Experimental results demonstrate the proposed DJDA method produces significantly superior categorization accuracies against state-of-the-art competitors. [ABSTRACT FROM AUTHOR]

In this paper, we propose a new random forest (RF) based ensemble method, ForesTexter, to solve the imbalanced text categorization problems. RF has shown great success in many real-world applications. However, the problem of learning from text data with class imbalance is a relatively new challenge that needs to be addressed. A RF algorithm tends to use a simple random sampling of features in building their decision trees. As a result, it selects many subspaces that contain few, if any, informative features for the minority class. Furthermore, the Gini measure for data splitting is considered to be skew sensitive and bias towards the majority class. Due to the inherent complex characteristics of imbalanced text datasets, learning RF from such data requires new approaches to overcome challenges related to feature subspace selection and cut-point choice while performing node splitting. To this end, we propose a new tree induction method that selects splits, both feature subspace selection and splitting criterion, for RF on imbalanced text data. The key idea is to stratify features into two groups and to generate effective term weighting for the features. One group contains positive features for the minority class and the other one contains the negative features for the majority class. Then, for feature subspace selection, we effectively select features from each group based on the term weights. The advantage of our approach is that each subspace contains adequate informative features for both minority and majority classes. One difference between our proposed tree induction method and the classical RF method is that our method uses Support Vector Machines (SVM) classifier to split the training data into smaller and more balance subsets at each tree node, and then successively retrains the SVM classifiers on the data partitions to refine the model while moving down the tree. In this way, we force the classifiers to learn from refined feature subspaces and data subsets to fit the imbalanced data better. Hence, the tree model becomes more robust for text categorization task with imbalanced dataset. Experimental results on various benchmark imbalanced text datasets (Reuters-21578, Ohsumed, and imbalanced 20 newsgroup) consistently demonstrate the effectiveness of our proposed ForesTexter method. The performance of our proposed approach is competitive against the standard random forest and different variants of SVM algorithms. [ABSTRACT FROM AUTHOR]