Search

On-device session-based recommendation systems have been achieving increasing attention on account of the low energy/resource consumption and privacy protection while providing promising recommendation performance. To fit the powerful neural session-based recommendation models in resource-constrained mobile devices, tensor-train decomposition and its variants have been widely applied to reduce memory footprint by decomposing the embedding table into smaller tensors, showing great potential in compressing recommendation models. However, these model compression techniques significantly increase the local inference time due to the complex process of generating index lists and a series of tensor multiplications to form item embeddings, and the resultant on-device recommender fails to provide real-time response and recommendation. To improve the online recommendation efficiency, we propose to learn compositional encoding-based compact item representations. Specifically, each item is represented by a compositional code that consists of several codewords, and we learn embedding vectors to represent each codeword instead of each item. Then the composition of the codeword embedding vectors from different embedding matrices (i.e., codebooks) forms the item embedding. Since the size of codebooks can be extremely small, the recommender model is thus able to fit in resource-constrained devices and meanwhile can save the codebooks for fast local inference.Besides, to prevent the loss of model capacity caused by compression, we propose a bidirectional self-supervised knowledge distillation framework. Extensive experimental results on two benchmark datasets demonstrate that compared with existing methods, the proposed on-device recommender not only achieves an 8x inference speedup with a large compression ratio but also shows superior recommendation performance., Extension of Our SIGIR'22 Paper (On-Device Next-Item Recommendation with Self-Supervised Knowledge Distillation), accepted by TOIS. arXiv admin note: text overlap with arXiv:2204.11091

Modern recommender systems operate in a fully server-based fashion. To cater to millions of users, the frequent model maintaining and the high-speed processing for concurrent user requests are required, which comes at the cost of a huge carbon footprint. Meanwhile, users need to upload their behavior data even including the immediate environmental context to the server, raising the public concern about privacy. On-device recommender systems circumvent these two issues with cost-conscious settings and local inference. However, due to the limited memory and computing resources, on-device recommender systems are confronted with two fundamental challenges: (1) how to reduce the size of regular models to fit edge devices? (2) how to retain the original capacity? Previous research mostly adopts tensor decomposition techniques to compress the regular recommendation model with limited compression ratio so as to avoid drastic performance degradation. In this paper, we explore ultra-compact models for next-item recommendation, by loosing the constraint of dimensionality consistency in tensor decomposition. Meanwhile, to compensate for the capacity loss caused by compression, we develop a self-supervised knowledge distillation framework which enables the compressed model (student) to distill the essential information lying in the raw data, and improves the long-tail item recommendation through an embedding-recombination strategy with the original model (teacher). The extensive experiments on two benchmarks demonstrate that, with 30x model size reduction, the compressed model almost comes with no accuracy loss, and even outperforms its uncompressed counterpart in most cases., To Appear in SIGIR'22

The traffic load has grown significantly in recent years, which might be a threat for the service safety of existing bridges. Thus, it is an urgent task to assess the actual traffic load effects on bridges, considering actual heavy traffic load instead of design traffic load. This study presents a framework for extrapolating maximum dynamic traffic load effects on large bridges using site-specific traffic monitoring data. The framework involves vehicle–bridge interaction analysis and probabilistic modelling of extreme values. The weigh-in-motion measurements of a busy highway in China were collected for stochastic traffic load modelling. Case studies of two long-span cable-supported bridge based on the weigh-in-motion measurements were conducted to demonstrate the effectiveness of the proposed framework. It is demonstrated that Rice’s level-crossing approach can capture both dynamic and probabilistic characteristics of the traffic load effects. The root-mean-square displacement of the cable-stayed bridge follows a C-type distribution, and the one for the suspension bridge follows an M-type distribution, which is associated with the first-order mode shapes of the two types of bridges. The amplification factors for the cable-stayed bridge and the suspension bridge are 5.9% and 3.6%, respectively. The numerical analysis indicates that the dynamic effect for extrapolation is weaker with the increase in bridge span length, but the effect of traffic volume growth will be more significant.

As mobile devices, such as smartphones and wearable watches, gain popularity and become an indispensable part of our daily life, people interact with extensive mobile applications every day for different purposes. While there are a lot of similar mobile applications in the same category that need to intensely compete with each other to stand out, the performance of their personalized recommender system is considered as a major factor in their success. On the other hand, deep learning (DL) has been a trending technique for developing recommender systems recently due to their outstanding effectiveness. As a result, most state-of-the-art recommender systems in real-world scenarios are already deep learning-based. To deliver personalized results from DL-based recommender systems (DLRSs) to users of mobile devices, the cloud-based paradigm is currently the most commonly adopted solution in real-world scenarios. In this paradigm, the DLRSs are first trained and deployed on powerful cloud servers, and then generate recommendation results to mobile users on demand. However, DLRSs running within this cloud-based paradigm are subject to network bottlenecks, privacy issues and energy consumption. Therefore, instead of the cloud-based paradigm, we propose to develop lightweight DLRSs that can securely run or even be trained on mobile devices locally to avoid those problems. This thesis identifies several critical challenges of running such novel local DLRS services and proposes corresponding novel solutions, which are three-fold. First, a well-known drawback of DLRSs is that they are large and have many more parameters than conventional machine learning methods, hence they are memory- and computation-intensive. However, mobile devices are resource-constrained, i.e., they lack abundant computation and storage capacities to support running or training such complex large models. Therefore, it is expected that the size of DLRSs should be compatible with limited memory space and the complexity of DLRSs should suit the computation power. Second, existing approaches to the DLRS training process require mobile users to upload their sensitive personal data, such as check-in history data, to servers. As awareness of personal privacy rises rapidly for both mobile users and the public, obtaining user data and storing it in servers is getting difficult. It is practical to devise a privacy-by-design solution to train DLRSs while letting mobile users keep their own data. Third, deep pure collaborative filtering (CF) models, as representative DLRSs, use simple neural structures and are only dependent on large-scale user-item interaction data to achieve good performance, which make them quite efficient and suitable to run on mobile devices. However, mobile recommender systems are known to suffer from data sparsity since most mobile users are subject to access constraints (such as physical constraints or time constraints) and can only visit limited items. Existing studies addressing this problem are neither efficient nor generic because they tend to incorporate the user/item features available and then develop ad-hoc models. The contributions of this thesis are made to address the aforementioned challenges and comprise of: (i) methodologies to compress DLRSs to fit in resource-constrained mobile devices and a knowledge distillation-based training algorithm to substantially improve the compressed model performance, which are applied to a next point-of-interest (POI) DLRS, (ii) a federated learning-based DLRS that moves the training from cloud servers to local devices to protect users' privacy, and a differential privacy-based mechanism to defend against potential malicious participating users, and (iii) a generic and end-to-end collaborated model to efficiently and effectively alleviate the data sparsity problem by generating reliable augmentation to the user-item interaction datasets.

It has been an important task for recommender systems to suggest satisfying activities to a group of users in people's daily social life. The major challenge in this task is how to aggregate personal preferences of group members to infer the decision of a group. Conventional group recommendation methods applied a predefined strategy for preference aggregation. However, these static strategies are too simple to model the real and complex process of group decision-making, especially for occasional groups which are formed ad-hoc. Moreover, group members should have non-uniform influences or weights in a group, and the weight of a user can be varied in different groups. Therefore, an ideal group recommender system should be able to accurately learn not only users' personal preferences but also the preference aggregation strategy from data. In this paper, we propose a novel end-to-end group recommender system named CAGR (short for Centrality Aware Group Recommender"), which takes Bipartite Graph Embedding Model (BGEM), the self-attention mechanism and Graph Convolutional Networks (GCNs) as basic building blocks to learn group and user representations in a unified way. Specifically, we first extend BGEM to model group-item interactions, and then in order to overcome the limitation and sparsity of the interaction data generated by occasional groups, we propose a self-attentive mechanism to represent groups based on the group members. In addition, to overcome the sparsity issue of user-item interaction data, we leverage the user social networks to enhance user representation learning, obtaining centrality-aware user representations. We create three large-scale benchmark datasets and conduct extensive experiments on them. The experimental results show the superiority of our proposed CAGR by comparing it with state-of-the-art group recommender models., To appear in TKDE

The task of dialogue generation has attracted increasing attention due to its diverse downstream applications, such as question-answering systems and chatbots. Recently, the deep neural network (DNN)-based dialogue generation models have achieved superior performance against conventional models utilizing statistical machine learning methods. However, despite that an enormous number of state-of-the-art DNN-based models have been proposed, there lacks detailed empirical comparative analysis for them on the open Chinese corpus. As a result, relevant researchers and engineers might find it hard to get an intuitive understanding of the current research progress. To address this challenge, we conducted an empirical study for state-of-the-art DNN-based dialogue generation models in various Chinese corpora. Specifically, extensive experiments were performed on several well-known single-turn and multi-turn dialogue corpora, including KdConv, Weibo, and Douban, to evaluate a wide range of dialogue generation models that are based on the symmetrical architecture of Seq2Seq, RNNSearch, transformer, generative adversarial nets, and reinforcement learning respectively. Moreover, we paid special attention to the prevalent pre-trained model for the quality of dialogue generation. Their performances were evaluated by four widely-used metrics in this area: BLEU, pseudo, distinct, and rouge. Finally, we report a case study to show example responses generated by these models separately.

Group Recommendation (GR) is the task of suggesting relevant items/events for a group of users in online systems, whose major challenge is to aggregate the preferences of group members to infer the decision of a group. Prior group recommendation methods applied predefined static strategies for preference aggregation. However, these static strategies are insufficient to model the complicated decision making process of a group, especially for occasional groups which are formed adhoc. Compared to conventional individual recommendation task, GR is rather dynamic and each group member may contribute differently to the final group decision. Recent works argue that group members should have non-uniform weights in forming the decision of a group, and try to utilize a standard attention mechanism to aggregate the preferences of group members, but they do not model the interaction behavior among group members, and the decision making process is largely unexplored.In this work, we study GR in a more general scenario, that is Occasional Group Recommendation (OGR), and focus on solving the preference aggregation problem and the data sparsity issue of group-item interactions. Instead of exploring new heuristic or vanilla attention-based mechanism, we propose a new social self-attention based aggregation strategy by directly modeling the interactions among group members, namely Group Self-Attention (GroupSA). In GroupSA, we treat the group decision making process as multiple voting processes, and develop a stacked social self-attention network to simulate how a group consensus is reached. To overcome the data sparsity issue, we resort to the relatively abundant user-item and user-user interaction data, and enhance the representation of users by two types of aggregation methods. In the training process, we further propose a joint training method to learn the user/item embeddings in the group-item recommendation task and the user-item recommendation task simultaneously. Finally, we conduct extensive experiments on two real-world datasets. The experimental results demonstrate the superiority of our proposed GroupSA method compared to several state-of-the-art methods in terms of HR and NDCG.

On E-commerce platforms, understanding the relationships (e.g., substitute and complement) among products from user's explicit feedback, such as users' online transactions, is of great importance to boost extra sales. However, the significance of such relationships is usually neglected by existing recommender systems. In this paper, we propose a semisupervised deep embedding model, namely, Substitute Products Embedding Model (SPEM), which models the substitutable relationships between products by preserving the second-order proximity, negative first-order proximity and semantic similarity in a product co-purchasing graph based on user's purchasing behaviours. With SPEM, the learned representations of two substitutable products align closely in the latent embedding space. Extensive experiments on real-world datasets are conducted, and the results verify that our model outperforms state-of-the-art baselines.

Collaborative filtering (CF) has become one of the most popular and widely used methods in recommender systems, but its performance degrades sharply for users with rare interaction data. Most existing hybrid CF methods try to incorporate side information such as review texts to alleviate the data sparsity problem. However, the process of exploiting and integrating side information is computationally expensive. Existing hybrid recommendation methods treat each user equally and ignore that the pure CF methods have already achieved both effective and efficient recommendation performance for active users with sufficient interaction records and the little improvement brought by side information to these active users is ignorable. Therefore, they are not cost-effective solutions. One cost-effective idea to bypass this dilemma is to generate sufficient "real" interaction data for the inactive users with the help of side information, and then a pure CF method could be performed on this augmented dataset effectively. However, there are three major challenges to implement this idea. Firstly, how to ensure the correctness of the generated interaction data. Secondly, how to combine the data augmentation process and recommendation process into a unified model and train the model end-to-end. Thirdly, how to make the solution generalizable for various side information and recommendation tasks. In light of these challenges, we propose a generic and effective CF model called AugCF that supports a wide variety of recommendation tasks. AugCF is based on Conditional Generative Adversarial Nets that additionally consider the class (like or dislike) as a feature to generate new interaction data, which can be a sufficiently real augmentation to the original dataset. Also, AugCF adopts a novel discriminator loss and Gumbel-Softmax approximation to enable end-to-end training. Finally, extensive experiments are conducted on two large-scale recommendation datasets, and the experimental results show the superiority of our proposed model.

Session-based Recommendation (SR) is the task of recommending the next item based on previously recorded user interactions. In this work, we study SR in a practical streaming scenario, namely Streaming Session-based Recommendation (SSR), which is a more challenging task due to (1) the uncertainty of user behaviors, and (2) the continuous, large-volume, high-velocity nature of the session data. Recent studies address (1) by exploiting the attention mechanism in Recurrent Neural Network (RNN) to better model the user's current intent, which leads to promising improvements. However, the proposed attention models are based solely on the current session. Moreover, existing studies only perform SR under static offline settings and none of them explore (2). In this work, we target SSR and propose a Streaming Session-based Recommendation Machine (SSRM) to tackle these two challenges. Specifically, to better understand the uncertainty of user behaviors, we propose a Matrix Factorization (MF) based attention model, which improves the commonly used attention mechanism by leveraging the user's historical interactions. To deal with the large-volume and high-velocity challenge, we introduce a reservoir-based streaming model where an active sampling strategy is proposed to improve the efficiency of model updating. We conduct extensive experiments on two real-world datasets. The experimental results demonstrate the superiority of the SSRM method compared to several state-of-the-art methods in terms of MRR and Recall.