Search

Data reconstruction, which aims at preserving statistical properties of the data during the reconstruction has become a new criterion for feature selection. Although feature selection could benefit from the perspective of data reconstruction, it is unable to exploit other crucial information, namely, graph structure and pairwise constraints. To address previously mentioned deficiency, we propose a novel feature selection approach in this paper, known as unsupervised feature selection via data reconstruction and side information. More specifically, the proposed method takes advantage of the prior knowledge regarding pairwise constraints (side information), the minimization of data reconstruction error, and the graph embedding simultaneously, such that pivotal features are selected with preserving data manifold structure. To obtain the robust solution, a robust loss function is applied to the feature selection problem, which interpolates between $\ell _{1}$ -norm and $\ell _{2}$ -norm. Eventually, extensive experiments are conducted to demonstrate the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

People in crowd scenes always exhibit consistent behaviors and form collective motions. The analysis of collective motion has motivated a surge of interest in computer vision. Nevertheless, the effort is hampered by the complex nature of collective motions. Considering the fact that collective motions are formed by individuals, this paper proposes a new framework for both quantifying and detecting collective motion by investigating the spatio-temporal behavior of individuals. The main contributions of this work are threefold: 1) an intention-aware model is built to fully capture the intrinsic dynamics of individuals; 2) a structure-based collectiveness measurement is developed to accurately quantify the collective properties of crowds; 3) a multi-stage clustering strategy is formulated to detect both the local and global behavior consistency in crowd scenes. Experiments on real world data sets show that our method is able to handle crowds with various structures and time-varying dynamics. Especially, the proposed method shows nearly 10% improvement over the competitors in terms of NMI, Purity and RI. Its applicability is illustrated in the context of anomaly detection and semantic scene segmentation. [ABSTRACT FROM AUTHOR]

The existing unsupervised feature extraction methods frequently explore low-redundant features by an uncorrelated constraint. However, the constrained models might incur trivial solutions, due to the singularity of scatter matrix triggered by high-dimensional data. In this paper, we propose a regularized regression model with a generalized uncorrelated constraint for feature selection, which leads to three merits: 1) exploring the low-redundant and discriminative features; 2) avoiding the trivial solutions and 3) simplifying the optimization. Besides that, the local cluster structure is achieved via a novel constrained spectral analysis for the unsupervised learning, where Must-Links and Cannot-Links are transformed into a intrinsic graph and a penalty graph respectively, rather than incorporated into a mixed affinity graph. Accordingly, a discriminative and uncorrelated feature selection with constrained spectral analysis (DUCFS) is proposed with adopting $\sigma $ -norm regularization for interpolating between F-norm and $\ell _{2,1}$ -norm. Due to the flexible gradient and global differentiability, our model converges fast. Extensive experiments on benchmark datasets among several state-of-the-art approaches verify the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Graph clustering plays an important role in data mining. Based on an input data graph, data points are partitioned into clusters. However, most existing methods keep the data graph fixed during the clustering procedure, so they are limited to exploit the implied data manifold and highly dependent on the initial graph construction. Inspired by the recent development on manifold learning, this paper proposes an Adaptive Consistency Propagation (ACP) method for graph clustering. In order to utilize the features captured from different perspectives, we further put forward the Multi-view version of the ACP model (MACP). The main contributions are threefold: (1) the manifold structure of input data is sufficiently exploited by propagating the topological connectivities between data points from near to far; (2) the optimal graph for clustering is learned by taking graph learning as a part of the optimization procedure; and (3) the negotiation among the heterogeneous features is captured by the multi-view clustering model. Extensive experiments on real-world datasets validate the effectiveness of the proposed methods on both single- and multi-view clustering, and show their superior performance over the state-of-the-arts. [ABSTRACT FROM AUTHOR]

Non-negative Matrix Factorization (NMF) has been one of the most popular clustering techniques in machine leaning, and involves various real-world applications. Most existing works perform matrix factorization on high-dimensional data directly. However, the intrinsic data structure is always hidden within the low-dimensional subspace. And, the redundant features within the input space may affect the final result adversely. In this paper, a new unsupervised matrix factorization method, Discrimination-aware Projected Matrix Factorization (DPMF), is proposed for data clustering. The main contributions are threefold: (1) The linear discriminant analysis is jointly incorporated into the unsupervised matrix factorization framework, so the clustering can be accomplished in the discriminant subspace. (2) The manifold regularization is introduced to perceive the geometric information, and the ${\ell _{2,1}}$ ℓ 2 , 1 -norm is utilized to improve the robustness. (3) An efficient optimization algorithm is designed to solve the proposed problem with proved convergence. Experimental results on one toy dataset and eight real-world benchmarks show the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Multi-Task Learning attempts to explore and mine the sufficient information within multiple related tasks for the better solutions. However, the performance of the existing multi-task approaches would largely degenerate when dealing with the polluted data, i.e., outliers. In this paper, we propose a novel robust multi-task model by incorporating a flexible manifold constraint (FMC-MTL) and a robust loss. Specifically speaking, multi-task subspace is embedded with a relaxed and generalized Stiefel Manifold for considering point-wise correlation and preserving the data structure simultaneously. In addition, a robust loss function is developed to ensure the robustness to outliers by smoothly interpolating between ℓ2,1-norm and squared Frobenius norm. Equipped with an efficient algorithm, FMC-MTL serves as a robust solution to tackling the severely polluted data. Moreover, extensive experiments are conducted to verify the superiority of our model. Compared to the state-of-the-art multi-task models, the proposed FMC-MTL model demonstrates remarkable robustness to the contaminated data. [ABSTRACT FROM AUTHOR]

Unsupervised feature selection has attracted more and more attention due to the rapid growth of the large amount of unlabelled and high-dimensional data. The performance of traditional spectral-based unsupervised methods always depends on the quality of constructed similarity matrix. However, real world data always contain a large number of noise samples and features that make the similarity matrix created by original data cannot be fully relied. We propose an unsupervised feature selection method which conducts feature selection and local structure learning simultaneously. Moreover, we add an important constraint on the similarity matrix to allow it to capture more accurate information of the data structure. To perform feature selection, orthogonal constraint and ℓ2,p-norm are adopted on the projection matrix. An efficient and simple algorithm is derived to tackle the problem. We conduct comprehensive experiments on various benchmark data sets, including handwritten digit, face image, and biomedical data, to validate the effectiveness of the proposed approach. [ABSTRACT FROM AUTHOR]

To meet the required huge data analysis, organization, and storage demand, the hashing technique has got a lot of attention as it aims to learn an efficient binary representation from the original high-dimensional data. In this paper, we focus on the unsupervised spectral hashing due to its effective manifold embedding. Existing spectral hashing methods mainly suffer from two problems, i.e., the inefficient spectral candidate and intractable binary constraint for spectral analysis. To overcome these two problems, we propose to employ spectral rotation to seek a better spectral solution and adopt the alternating projection algorithm to settle the complex code constraints, which are therefore named as Spectral Hashing with Spectral Rotation and Alternating Discrete Spectral Hashing, respectively. To enjoy the merits of both methods, the spectral rotation technique is finally combined with the original spectral objective, which aims to simultaneously learn better spectral solution and more efficient discrete codes and is called as Discrete Spectral Hashing. Furthermore, the efficient optimization algorithms are also provided, which just take comparable time complexity to existing hashing methods. To evaluate the proposed three methods, extensive comparison experiments and studies are conducted on four large-scale data sets for the image retrieval task, and the noticeable performance beats several state-of-the-art spectral hashing methods on different evaluation metrics. [ABSTRACT FROM AUTHOR]

Nonnegative matrix factorization (NMF), known as a famous matrix factorization technique, has been widely used in pattern recognition and computer vision. NMF represents the input data matrix as a product of two nonnegative factors. As NMF is based on the Euclidean distance, which is sensitive to noise or errors in the data, some robust NMF methods are proposed. Mainly focusing on parts-based representation, these robust NMF methods often neglect global representation of data. In fact, the global geometry information of data is more robust than the local information about the noisy data in terms of image classification. In order to effectively improve the robustness of NMF and learn part-based and global representation of the data, a novel method low-rank nonnegative factorization (LRNF) is proposed in this paper. First, we assume that the data are grossly corrupted, and the $L_{1} $ norm is used as a sparse constraint on the assumed noise matrix. Then, LRNF learns a low-rank matrix with the global representation ability. Finally, we make a nonnegative factorization of the learned low-rank matrix. We can obtain a base matrix, which preserves locality and globality properties of the data in the meantime. Extensive experiments have been conducted on nine real-world image databases to verify the performance of the proposed LRNF method by comparing with the state-of-the-art algorithms on robust dimensionality reduction. [ABSTRACT FROM AUTHOR]

Linear discriminant analysis (LDA) has been proven to be effective in dimensionality reduction. However, the performance of LDA depends on the consistency assumption of the global structure and the local structure. Some work extended LDA along this line of research and proposed local formulations of LDA. Unfortunately, the learning scheme of these algorithms is suboptimal in that the intrinsic relationship between data points is pre-learned in the original space, which is usually affected by the noise and redundant features. Besides, the time cost is relatively high. To alleviate these drawbacks, we propose a Fast Locality Discriminant Analysis framework (FLDA), which has three advantages: (1) It can divide a non-Gaussian distribution class into many sub-blocks that obey Gaussian distributions by using the anchor-based strategy. (2) It captures the manifold structure of data by learning the fuzzy membership relationship between data points and the corresponding anchor points, which can reduce computation time. (3) The weights between data points and anchor points are adaptively updated in the subspace where the irrelevant information and the noise in high-dimensional space have been effectively suppressed. Extensive experiments on toy data sets, UCI benchmark data sets and imbalanced data sets demonstrate the efficiency and effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Linear discriminant analysis (LDA) has been proven to be effective in dimensionality reduction. However, the performance of LDA depends on the consistency assumption of the global structure and the local structure. Some work extended LDA along this line of research and proposed local formulations of LDA. Unfortunately, the learning scheme of these algorithms is suboptimal in that the intrinsic relationship between data points is pre-learned in the original space, which is usually affected by the noise and redundant features. Besides, the time cost is relatively high. To alleviate these drawbacks, we propose a Fast Locality Discriminant Analysis framework (FLDA), which has three advantages: (1) It can divide a non-Gaussian distribution class into many sub-blocks that obey Gaussian distributions by using the anchor-based strategy. (2) It captures the manifold structure of data by learning the fuzzy membership relationship between data points and the corresponding anchor points, which can reduce computation time. (3) The weights between data points and anchor points are adaptively updated in the subspace where the irrelevant information and the noise in high-dimensional space have been effectively suppressed. Extensive experiments on toy data sets, UCI benchmark data sets and imbalanced data sets demonstrate the efficiency and effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Semi-supervised learning as one of most attractive problems in machine learning research field has aroused broad attentions in recent years. In this paper, we propose a novel locality preserved dimensionality reduction framework, named Semi-supervised Adaptive Local Embedding learning (SALE), which learns a local discriminative embedding by constructing a $k_1$ k 1 Nearest Neighbors ($k_1$ k 1 NN) graph on labeled data, so as to explore the intrinsic structure, i.e., sub-manifolds from non-Gaussian labeled data. Then, mapping all samples into learned embedding and constructing another $k_2$ k 2 NN graph on all embedded data to explore the global structure of all samples. Therefore, the unlabeled data and their corresponding labeled neighbors can be clustered into same sub-manifold, so as to improve the discriminative power of embedded data. Furthermore, we propose two semi-supervised dimensionality reduction methods with orthogonal and whitening constraints based on proposed SALE framework. An efficient alternatively iterative optimization algorithm is developed to solve the NP-hard problem in our models. Extensive experiments conducted on several synthetic and real-world data sets demonstrate the superiorities of our methods on local structure exploration and classification task. [ABSTRACT FROM AUTHOR]

Principal component analysis (PCA) is warmly welcomed in dimensionality reduction and its applications. Due to the high sensitivity of PCA to outliers, a series of PCA methods are proposed to enhance the robustness of PCA. Besides, the representation ability of the existing PCA methods has limitations as well. To enhance the robustness and representation ability of robust PCA, we elaborate a novel Graph Convolution Robust PCA method (GRPCA) to incorporate the manifold structure into PCA. It constructs a sparse graph based on the local connectivity structure of samples. Graph auto-encoder is utilized to solve the robust PCA problem under the low-rank and sparse constraints. With the dual-decoder, GRPCA learns the low-dimensional embeddings that reconstruct the manifold structure and low-rank approximation simultaneously. Furthermore, since the graph suffers from misconnection triggered by occlusions, the local connectivity structure of low-dimensional embeddings is utilized to modify the graph. Our proposed method excels in both the clustering of low-dimensional embeddings and the low-rank recovery. Lastly, extensive experiments conducted on six real-world datasets demonstrated the efficiency and superiority of the proposed GRPCA. [ABSTRACT FROM AUTHOR]

Data clustering, which is to partition the given data into different groups, has attracted much attention. Recently various effective algorithms have been developed to tackle the task. Among these methods, non-negative matrix factorization (NMF) has been demonstrated to be a powerful tool. However, there are still some problems. First, the standard NMF is sensitive to noises and outliers. Although $\ell _{2,1}$ ℓ 2 , 1 norm based NMF improves the robustness, it is still affected easily by large noises. Second, for most graph regularized NMF, the performance highly depends on the initial similarity graph. Third, many graph-based NMF models perform the graph construction and matrix factorization in two separated steps. Thus the learned graph structure may not be optimal. To overcome the above drawbacks, we propose a robust bi-stochastic graph regularized matrix factorization (RBSMF) framework for data clustering. Specifically, we present a general loss function, which is more robust than the commonly used $L_2$ L 2 and $L_1$ L 1 functions. Besides, instead of keeping the graph fixed, we learn an adaptive similarity graph. Furthermore, the graph updating and matrix factorization are processed simultaneously, which can make the learned graph more appropriate for clustering. Extensive experiments have shown the proposed RBSMF outperforms other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

The results of traditional clustering methods are usually unreliable as there is not any guidance from the data labels, while the class labels can be predicted more reliable by the semisupervised learning if the labels of partial data are given. In this paper, we propose an actively self-training clustering method, in which the samples are actively selected as training set to minimize an estimated Bayes error, and then explore semisupervised learning to perform clustering. Traditional graph-based semisupervised learning methods are not convenient to estimate the Bayes error; we develop a specific regularization framework on graph to perform semisupervised learning, in which the Bayes error can be effectively estimated. In addition, the proposed clustering algorithm can be readily applied in a semisupervised setting with partial class labels. Experimental results on toy data and real-world data sets demonstrate the effectiveness of the proposed clustering method on the unsupervised and the semisupervised setting. It is worthy noting that the proposed clustering method is free of initialization, while traditional clustering methods are usually dependent on initialization. [ABSTRACT FROM AUTHOR]

Semisupervised dimensionality reduction has been attracting much attention as it not only utilizes both labeled and unlabeled data simultaneously, but also works well in the situation of out-of-sample. This paper proposes an effective approach of semisupervised dimensionality reduction through label propagation and label regression. Different from previous efforts, the new approach propagates the label information from labeled to unlabeled data with a well-designed mechanism of random walks, in which outliers are effectively detected and the obtained virtual labels of unlabeled data can be well encoded in a weighted regression model. These virtual labels are thereafter regressed with a linear model to calculate the projection matrix for dimensionality reduction. By this means, when the manifold or the clustering assumption of data is satisfied, the labels of labeled data can be correctly propagated to the unlabeled data; and thus, the proposed approach utilizes the labeled and the unlabeled data more effectively than previous work. Experimental results are carried out upon several databases, and the advantage of the new approach is well demonstrated. [ABSTRACT FROM AUTHOR]

In the past few years, scene contextual information has been increasingly used for action understanding with promising results. However, unsupervised video action clustering using context has been less explored, and existing clustering methods cannot achieve satisfactory performances. In this paper, we propose a novel unsupervised video action clustering method by using the motion-scene interaction constraint (MSIC). The proposed method takes the unique static scene and dynamic motion characteristics of video action into account, and develops a contextual interaction constraint model under a self-representation subspace clustering framework. First, the complementarity of multi-view subspace representation in each context is explored by single-view and multi-view constraints. Afterward, the context-constrained affinity matrix is calculated and the MSIC is introduced to mutually regularize the disagreement of subspace representation in scene and motion. Finally, by jointly constraining the complementarity of multi-views and the consistency of multi-contexts, an overall objective function is constructed to guarantee the video action clustering result. The experiments on four video benchmark datasets (Weizmann, KTH, UCFsports, and Olympic) demonstrate that the proposed method outperforms the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

As a popular dimensionality reduction method, nonnegative matrix factorization (NMF) has been widely used in image classification. However, the NMF does not consider discriminant information from the data themselves. In addition, most NMF-based methods use the Euclidean distance as a metric, which is sensitive to noise or outliers in data. To solve these problems, in this paper, we introduce structural incoherence and low-rank to NMF and propose a novel nonnegative factorization method, called structurally incoherent low-rank NMF (SILR-NMF), in which we jointly consider structural incoherence and low-rank properties of data for image classification. For the corrupted data, we use the $L_{1}$ norm as a constraint to ensure the noise is sparse. SILR-NMF learns a clean data matrix from the noisy data by low-rank learning. As a result, the SILR-NMF can capture the global structure information of the data, which is more robust than local information to noise. By introducing the structural incoherence of the learned clean data, SILR-NMF ensures the clean data points from different classes are as independent as possible. To verify the performance of the proposed method, extensive experiments are conducted on six image databases. The experimental results demonstrate that our proposed method has substantial gain over existing NMF approaches. [ABSTRACT FROM AUTHOR]

We consider the problem of image representation from the perspective of statistical design. Recent studies have shown that images are possibly sampled from a low dimensional manifold despite of the fact that the ambient space is usually very high dimensional. Learning low dimensional image representations is crucial for many image processing tasks such as recognition and retrieval. Most of the existing approaches for learning low dimensional representations, such as principal component analysis (PCA) and locality preserving projections (LPP), aim at discovering the geometrical or discriminant structures in the data. In this paper, we take a different perspective from statistical experimental design, and propose a novel dimensionality reduction algorithm called A-Optimal Projection (AOP). AOP is based on a linear regression model. Specifically, AOP finds the optimal basis functions so that the expected prediction error of the regression model can be minimized if the new representations are used for training the model. Experimental results suggest that the proposed approach provides a better representation and achieves higher accuracy in image retrieval. [ABSTRACT FROM PUBLISHER]

The neighbor-embedding (NE) algorithm for single-image super-resolution (SR) reconstruction assumes that the feature spaces of low-resolution (LR) and high-resolution (HR) patches are locally isometric. However, this is not true for SR because of one-to-many mappings between LR and HR patches. To overcome or at least to reduce the problem for NE-based SR reconstruction, we apply a joint learning technique to train two projection matrices simultaneously and to map the original LR and HR feature spaces onto a unified feature subspace. Subsequently, the k-nearest neighbor selection of the input LR image patches is conducted in the unified feature subspace to estimate the reconstruction weights. To handle a large number of samples, joint learning locally exploits a coupled constraint by linking the LR–HR counterparts together with the K-nearest grouping patch pairs. In order to refine further the initial SR estimate, we impose a global reconstruction constraint on the SR outcome based on the maximum a posteriori framework. Preliminary experiments suggest that the proposed algorithm outperforms NE-related baselines. [ABSTRACT FROM AUTHOR]

The Gaussian process latent variable model (GP-LVM) has been identified to be an effective probabilistic approach for dimensionality reduction because it can obtain a low-dimensional manifold of a data set in an unsupervised fashion. Consequently, the GP-LVM is insufficient for supervised learning tasks (e.g., classification and regression) because it ignores the class label information for dimensionality reduction. In this paper, a supervised GP-LVM is developed for supervised learning tasks, and the maximum a posteriori algorithm is introduced to estimate positions of all samples in the latent variable space. We present experimental evidences suggesting that the supervised GP-LVM is able to use the class label information effectively, and thus, it outperforms the GP-LVM and the discriminative extension of the GP-LVM consistently. The comparison with some supervised classification methods, such as Gaussian process classification and support vector machines, is also given to illustrate the advantage of the proposed method. [ABSTRACT FROM AUTHOR]