Search

Nonnegative matrix factorization (NMF) has been successfully applied in several data mining tasks. Recently, there is an increasing interest in the acceleration of NMF, due to its high cost on large matrices. On the other hand, the privacy issue of NMF over federated data is worthy of attention, since NMF is prevalently applied in image and text analysis which may involve leveraging privacy data (e.g, medical image and record) across several parties (e.g., hospitals). In this paper, we study the acceleration and security problems of distributed NMF. First, we propose a distributed sketched alternating nonnegative least squares(DSANLS) framework for NMF, which utilizes a matrix sketching technique to reduce the size of nonnegative least squares subproblems with a convergence guarantee. For the second problem, we show that DSANLS with modification can be adapted to the security setting, but only for one or limited iterations. Consequently, we propose four efficient distributed NMF methods in both synchronous and asynchronous settings with a security guarantee. We conduct extensive experiments on several real datasets to show the superiority of our proposed methods. The implementation of our methods is available at https://github.com/qianyuqiu79/DSANLS. [ABSTRACT FROM AUTHOR]

Agnostic matrix phase retrieval (AMPR) is a general low-rank matrix recovery problem given a set of noisy high-dimensional data samples. To be specific, AMPR is targeting at recovering an $r$ -rank matrix $\mathbf {M}^*\in \mathbb {R}^{d_1\times d_2}$ as the parametric component from $n$ instantiations/samples of a semi-parametric model $y=f(\langle \mathbf {M}^*, \mathbf {X}\rangle, \epsilon)$ , where the predictor matrix is denoted as $\mathbf {X}\in \mathbb {R}^{d_1\times d_2}$ , link function $f(\cdot, \epsilon)$ is agnostic under some mild distribution assumptions on $\mathbf {X}$ , and $\epsilon$ represents the noise. In this paper, we formulate AMPR as a rank-restricted largest eigenvalue problem by applying the second-order Stein's identity and propose a new spectrum truncation power iteration (STPower) method to obtain the desired matrix efficiently. Also, we show a favorable rank recovery result by adopting the STPower method, i.e., a near-optimal statistical convergence rate under some relatively general model assumption from a wide range of applications. Extensive simulations verify our theoretical analysis and showcase the strength of STPower compared with the other existing counterparts. [ABSTRACT FROM AUTHOR]

Incomplete multi-view clustering (IMVC) optimally combines multiple pre-specified incomplete views to improve clustering performance. Among various excellent solutions, the recently proposed multiple kernel $k$ k -means with incomplete kernels (MKKM-IK) forms a benchmark, which redefines IMVC as a joint optimization problem where the clustering and kernel matrix imputation tasks are alternately performed until convergence. Though demonstrating promising performance in various applications, we observe that the manner of kernel matrix imputation in MKKM-IK would incur intensive computational and storage complexities, over-complicated optimization and limitedly improved clustering performance. In this paper, we first propose an Efficient and Effective Incomplete Multi-view Clustering (EE-IMVC) algorithm to address these issues. Instead of completing the incomplete kernel matrices, EE-IMVC proposes to impute each incomplete base matrix generated by incomplete views with a learned consensus clustering matrix. Moreover, we further improve this algorithm by incorporating prior knowledge to regularize the learned consensus clustering matrix. Two three-step iterative algorithms are carefully developed to solve the resultant optimization problems with linear computational complexity, and their convergence is theoretically proven. After that, we theoretically study the generalization bound of the proposed algorithms. Furthermore, we conduct comprehensive experiments to study the proposed algorithms in terms of clustering accuracy, evolution of the learned consensus clustering matrix and the convergence. As indicated, our algorithms deliver their effectiveness by significantly and consistently outperforming some state-of-the-art ones. [ABSTRACT FROM AUTHOR]