Search

Non-negative Matrix Factorization (NMF) has been one of the most popular clustering techniques in machine leaning, and involves various real-world applications. Most existing works perform matrix factorization on high-dimensional data directly. However, the intrinsic data structure is always hidden within the low-dimensional subspace. And, the redundant features within the input space may affect the final result adversely. In this paper, a new unsupervised matrix factorization method, Discrimination-aware Projected Matrix Factorization (DPMF), is proposed for data clustering. The main contributions are threefold: (1) The linear discriminant analysis is jointly incorporated into the unsupervised matrix factorization framework, so the clustering can be accomplished in the discriminant subspace. (2) The manifold regularization is introduced to perceive the geometric information, and the ${\ell _{2,1}}$ ℓ 2 , 1 -norm is utilized to improve the robustness. (3) An efficient optimization algorithm is designed to solve the proposed problem with proved convergence. Experimental results on one toy dataset and eight real-world benchmarks show the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

In this paper, a novel low-rank structural model is proposed for segmenting data drawn from a high-dimensional space. Our method is based on the fact that all groups clustered from a high-dimensional dataset are distributed in multiple low-rank subspaces. In general, it’s a very difficult task to find the low-rank structures hidden in data. Different from the classical sparse subspace clustering (SSC) and low-rank representation (LRR) which all take two steps including building the affinity matrix and spectral clustering, we introduce a new rank constraint into our model. This constraint allows our model to learn a subspace indicator which can capture different clusters directly from the data without any postprocessing. To further approximate the rank constraint, a piecewise function is utilized as the relaxing item for the proposed model. Besides, under the subspace indicator constraints, the integer programming problem is avoided, which makes our algorithm more efficient and scalable. In addition, we prove the convergence of the proposed algorithm in theory and further discuss the general case in which subspaces don’t pass through the origin. Experiment results on both synthetic and real-world datasets demonstrate that our algorithm significantly outperforms the state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Fuzzy K-Means (FKM) clustering is of great importance for analyzing unlabeled data. FKM algorithms assign each data point to multiple clusters with some degree of certainty measured by the membership function. In these methods, the fuzzy membership degree matrix is obtained based on the calculation of the distance between data points in the original space. However, this operation may lead to suboptimal results because of the influence of noises and redundant features. Besides, some FKM clustering methods ignore the importance of the weighting exponent. In this paper, we propose a novel FKM method called Fuzzy K-Means Clustering With Discriminative Embedding. Within this method, we simultaneously conduct dimensionality reduction along with fuzzy membership degree learning. To retain most information in the embedding subspace and improve the robustness of this method, principal component analysis is incorporated into our framework. An iterative optimization algorithm is proposed to solve the model. To validate the efficacy of the proposed method, we perform comprehensive analyses, including convergence behavior, parameter determination and computational complexity. Moreover, we also match a appropriate weighting exponent for each data set. Experimental results on benchmark data sets show that the proposed method is more discriminative and effective for clustering tasks. [ABSTRACT FROM AUTHOR]

Feature selection is one of the most important techniques to deal with the high-dimensional data for a variety of machine learning and data mining tasks, such clustering, classification, and retrieval, etc. Fuzziness is a widespread nature of data in nature human society. However, most existing feature selection methods ignore the existence of fuzziness in the data, resulting in sub-optimal feature subsets. To address the problem, we propose a novel unsupervised feature selection method, called Unsupervised Discriminative Projection for Feature Selection (UDPFS) to select discriminative features by conducting fuzziness learning and sparse learning, simultaneously. Specifically, we use projection matrix transform data as its low-dimensional representation, which are partitioned into clusters by using membership matrix with sparse constraint. In addition, $\ell _{2, 1}$ ℓ 2 , 1 -norm regularization is applied to the projection matrix. Then, a discriminative projection matrix with row sparse is obtained by perform fuzziness learning and sparse learning, simultaneously. An effective alternative optimization algorithm is proposed to solve the objective function. Evaluate experimental results on several real-world datasets show the effectiveness and superiority of the proposed unsupervised feature selection method. [ABSTRACT FROM AUTHOR]

Spectral clustering is a quite simple but effective method for solving graph clustering problem. It projects the original data points into a lower dimensional space with spectral embedding, and then relies on an algorithm to obtain the cluster labels. Since it involves eigendecomposition of the graph Laplacian matrix for embedding, spectral clustering has high time complexity and is not able to process large scale data. The performance of spectral clustering is also limited by a post-processing algorithm such as kmeans. To tackle the two issues, we propose a method called Orthogonal and Nonnegative Graph Reconstruction (ONGR) for large scale clustering. The two constraints serve as a structure constraint with which the graph reconstructed by the indicator matrix is structured. The proposed method has linear time complexity with respect to the data size that it mainly needs to implicitly construct a graph and iteratively perform economical singular value decomposition for a small size matrix. Moreover, the interpretability of the indicator matrix is offered due to the nonnegative constraint, and thus our method can provide the cluster labels with no post-processing. The experiments on benchmark datasets show the effectiveness of the proposed scalable clustering method. [ABSTRACT FROM AUTHOR]

Sparse regression based feature selection method has been extensively investigated these years. However, because it has a non-convex constraint, i.e., $\ell _{2,0}$ℓ2,0-norm constraint, this problem is very hard to solve. In this paper, unlike most of the other methods which only solve its slack version by introducing sparsity regularization into objective function forcibly, a novel framework is proposed by us to solve the original $\ell _{2,0}$ℓ2,0-norm constrained sparse regression based feature selection problem. We transform our objective function into Linear Discriminant Analysis (LDA) by using a new label coding method, thus enabling our model to calculate the ratio of inter-class scatter to intra-class scatter of features which is the most widely used feature discrimination evaluation metric. According to that ratio, features can be selected by a simple sorting method. The projection gradient descent algorithm is introduced to further improve the performance of our algorithm by using the solution obtained before as its initial solution. This ensures the stability of this iterative algorithm. We prove that the proposed method can get the global optimal solution of this non-convex problem when all features are statistically independent. For the general case where features are statistically dependent, extensive experiments on six small sample size datasets and one large-scale dataset show that our algorithm has comparable or better classification capability comparing with other eight state-of-the-art feature selection methods by the SVM classifier. We also show that our algorithm can obtain a low loss value, which means the solution of our algorithm can get very close to this NP-hard problem's real solution. What is more, because we solve the original $\ell _{2,0}$ℓ2,0-norm constrained problem, we avoid the heavy work of tuning the regularization parameter because its meaning is explicit in our method, i.e., the number of selected features. At last, we evaluate the stability of our algorithm from two perspectives, i.e., the objective function values and the selected features, by experiments. From both perspectives, our algorithm shows satisfactory stability performance. [ABSTRACT FROM AUTHOR]