Hyperbolic Factorization Machine

Factorization machine (FM) has been widely applied in recommender systems since its publication, and to capture feature interactions, FM represents its 2nd-order coefficient of any two features as inner product of the corresponding embedding vectors in Euclidean space. Considering that objects in recommender systems such as items, users, properties and contexts can be described as a heterogeneous network exhibiting hierarchical structures, whereas flat Euclidean space is not able to capture this kind of structure, restricting the feature representation ability of FM, this paper proposes hyperbolic FM (HFM). It represents each feature as a vector in hyperbolic space rather than in Euclidean space, and evaluates the 2nd-order feature interaction strength with hyperbolic distance measure. The reason for adopting hyperbolic geometry is that it has been shown to be the underlying embedding space of hierarchical structures, like trees, graphs and vocabulary. This paper designs two HFMs based on Poincaré ball model and hyperboloid model, respectively, and derives the corresponding Riemannian gradient descent algorithm for optimization. Experiments conducted on various datasets indicate that HFM achieves better performance than original FM with identical number of trainable parameters, and reveals the hierarchical structure of features which is missing in FM, offering explanability to some extent.