Search

Graph convolutional networks are capable of handling non-Euclidean data with sparse features, and some research has begun to apply them to the field of recommendation systems. Graph convolutional network's aggregation and propagation mechanism can learn features well and improve the embedding quality. However, simply applying GCN to the recommendation domain can only show some of its advantages, and the complex structure makes it difficult for the model to handle the massive amount of data in the industrial domain. Some work has been done to integrate GCN with recommendation systems better. However, most related work pursues the model's simplicity and ignores the large amount of hidden auxiliary information. In this paper, we propose a GCN-SA model, which adds a multi-head self-attention mechanism to the aggregation and propagation process to learn the weights of neighboring nodes and analyze the importance of the relationships between nodes; we also design a new embedding splicing layer for the graph convolutional network, which dynamically adjusts the embedding of different layers to achieve adaptive layer smoothing and mitigate the over-smoothing phenomenon. After experimental results on five benchmark datasets, we show that the GCN-SA model outperforms previous related work, captures a large amount of auxiliary information, and enhances the expressive ability of the model. [ABSTRACT FROM AUTHOR]

Learning large-scale data sets with high dimensionality is a main concern in research areas including machine learning, visual recognition, information retrieval, to name a few. In many practical uses such as images, video, audio, and text processing, we have to face with high-dimension and large-sample data problems. The trace-ratio problem is a key problem for feature extraction and dimensionality reduction to circumvent the high dimensional space. However, it has been long believed that this problem has no closed-form solution, and one has to solve it by using some inner-outer iterative algorithms that are very time consuming. Therefore, efficient algorithms for high-dimension and large-sample trace-ratio problems are still lacking, especially for dense data problems. In this work, we present a closed-form solution for the trace-ratio problem, and propose two algorithms to solve it. Based on the formula and the randomized singular value decomposition, we first propose a randomized algorithm for solving high-dimension and large-sample dense trace-ratio problems. For high-dimension and large-sample sparse trace-ratio problems, we then propose an algorithm based on the closed-form solution and solving some consistent under-determined linear systems. Theoretical results are established to show the rationality and efficiency of the proposed methods. Numerical experiments are performed on some real-world data sets, which illustrate the superiority of the proposed algorithms over many state-of-the-art algorithms for high-dimension and large-sample dimensionality reduction problems. [ABSTRACT FROM AUTHOR]

Recently, some matrix exponential-based discriminant analysis methods were proposed for high dimensionality reduction. It has been shown that they often have more discriminant power than the corresponding discriminant analysis methods. However, one has to solve some large-scale matrix exponential eigenvalue problems which constitutes the bottleneck in this type of methods. The main contribution of this paper is twofold: First, we propose a framework of fast implementation on general matrix exponential-based discriminant analysis methods. The key is to equivalently transform large-scale matrix computation problems into much smaller ones. On the other hand, it was mentioned in Wang et al. (IEEE Trans Image Process 23:920–930, 2014) that the exponential model is more reliable than the original one and suppresses the sensitivity to pertubations. However, the interpretation is only heuristic, and to the best of our knowledge, there is no theoretical justification for reliability and stability of the matrix discriminant analysis methods. To fill in this gap, the second contribution of our work is to provide stability analysis for the fast exponential discriminant analysis method from a theoretical point of view. Numerical experiments illustrate the numerical behavior of the proposed algorithm, and demonstrate that our algorithm is more stable than many state-of-the-art algorithms for high dimensionality reduction. [ABSTRACT FROM AUTHOR]