Search

Traditional recommendation systems are subject to a strong feedback loop by learning from and reinforcing past user-item interactions, which in turn limits the discovery of novel user interests. To address this, we introduce a hybrid hierarchical framework combining Large Language Models (LLMs) and classic recommendation models for user interest exploration. The framework controls the interfacing between the LLMs and the classic recommendation models through "interest clusters", the granularity of which can be explicitly determined by algorithm designers. It recommends the next novel interests by first representing "interest clusters" using language, and employs a fine-tuned LLM to generate novel interest descriptions that are strictly within these predefined clusters. At the low level, it grounds these generated interests to an item-level policy by restricting classic recommendation models, in this case a transformer-based sequence recommender to return items that fall within the novel clusters generated at the high level. We showcase the efficacy of this approach on an industrial-scale commercial platform serving billions of users. Live experiments show a significant increase in both exploration of novel interests and overall user enjoyment of the platform.

Electroencephalogram (EEG) is a non-invasive technique to record bioelectrical signals. Integrating supervised deep learning techniques with EEG signals has recently facilitated automatic analysis across diverse EEG-based tasks. However, the label issues of EEG signals have constrained the development of EEG-based deep models. Obtaining EEG annotations is difficult that requires domain experts to guide collection and labeling, and the variability of EEG signals among different subjects causes significant label shifts. To solve the above challenges, self-supervised learning (SSL) has been proposed to extract representations from unlabeled samples through well-designed pretext tasks. This paper concentrates on integrating SSL frameworks with temporal EEG signals to achieve efficient representation and proposes a systematic review of the SSL for EEG signals. In this paper, 1) we introduce the concept and theory of self-supervised learning and typical SSL frameworks. 2) We provide a comprehensive review of SSL for EEG analysis, including taxonomy, methodology, and technique details of the existing EEG-based SSL frameworks, and discuss the difference between these methods. 3) We investigate the adaptation of the SSL approach to various downstream tasks, including the task description and related benchmark datasets. 4) Finally, we discuss the potential directions for future SSL-EEG research., Comment: 35 pages, 12 figures

Due to the abundant neurophysiological information in the electroencephalogram (EEG) signal, EEG signals integrated with deep learning methods have gained substantial traction across numerous real-world tasks. However, the development of supervised learning methods based on EEG signals has been hindered by the high cost and significant label discrepancies to manually label large-scale EEG datasets. Self-supervised frameworks are adopted in vision and language fields to solve this issue, but the lack of EEG-specific theoretical foundations hampers their applicability across various tasks. To solve these challenges, this paper proposes a knowledge-driven cross-view contrastive learning framework (KDC2), which integrates neurological theory to extract effective representations from EEG with limited labels. The KDC2 method creates scalp and neural views of EEG signals, simulating the internal and external representation of brain activity. Sequentially, inter-view and cross-view contrastive learning pipelines in combination with various augmentation methods are applied to capture neural features from different views. By modeling prior neural knowledge based on homologous neural information consistency theory, the proposed method extracts invariant and complementary neural knowledge to generate combined representations. Experimental results on different downstream tasks demonstrate that our method outperforms state-of-the-art methods, highlighting the superior generalization of neural knowledge-supported EEG representations across various brain tasks., Comment: 14pages,7 figures

Cross-lingual entity alignment is the task of finding the same semantic entities from different language knowledge graphs. In this paper, we propose a simple and novel unsupervised method for cross-language entity alignment. We utilize the deep learning multi-language encoder combined with a machine translator to encode knowledge graph text, which reduces the reliance on label data. Unlike traditional methods that only emphasize global or local alignment, our method simultaneously considers both alignment strategies. We first view the alignment task as a bipartite matching problem and then adopt the re-exchanging idea to accomplish alignment. Compared with the traditional bipartite matching algorithm that only gives one optimal solution, our algorithm generates ranked matching results which enabled many potentials downstream tasks. Additionally, our method can adapt two different types of optimization (minimal and maximal) in the bipartite matching process, which provides more flexibility. Our evaluation shows, we each scored 0.966, 0.990, and 0.996 Hits@1 rates on the DBP15K dataset in Chinese, Japanese, and French to English alignment tasks. We outperformed the state-of-the-art method in unsupervised and semi-supervised categories. Compared with the state-of-the-art supervised method, our method outperforms 2.6% and 0.4% in Ja-En and Fr-En alignment tasks while marginally lower by 0.2% in the Zh-En alignment task., Comment: 17 pages,5 figures, Accepted by ECML PKDD 2023(Research Track)

Emotion recognition in user-generated videos plays an important role in human-centered computing. Existing methods mainly employ traditional two-stage shallow pipeline, i.e. extracting visual and/or audio features and training classifiers. In this paper, we propose to recognize video emotions in an end-to-end manner based on convolutional neural networks (CNNs). Specifically, we develop a deep Visual-Audio Attention Network (VAANet), a novel architecture that integrates spatial, channel-wise, and temporal attentions into a visual 3D CNN and temporal attentions into an audio 2D CNN. Further, we design a special classification loss, i.e. polarity-consistent cross-entropy loss, based on the polarity-emotion hierarchy constraint to guide the attention generation. Extensive experiments conducted on the challenging VideoEmotion-8 and Ekman-6 datasets demonstrate that the proposed VAANet outperforms the state-of-the-art approaches for video emotion recognition. Our source code is released at: https://github.com/maysonma/VAANet., Comment: Accepted by AAAI 2020

Deep neural networks suffer from performance decay when there is domain shift between the labeled source domain and unlabeled target domain, which motivates the research on domain adaptation (DA). Conventional DA methods usually assume that the labeled data is sampled from a single source distribution. However, in practice, labeled data may be collected from multiple sources, while naive application of the single-source DA algorithms may lead to suboptimal solutions. In this paper, we propose a novel multi-source distilling domain adaptation (MDDA) network, which not only considers the different distances among multiple sources and the target, but also investigates the different similarities of the source samples to the target ones. Specifically, the proposed MDDA includes four stages: (1) pre-train the source classifiers separately using the training data from each source; (2) adversarially map the target into the feature space of each source respectively by minimizing the empirical Wasserstein distance between source and target; (3) select the source training samples that are closer to the target to fine-tune the source classifiers; and (4) classify each encoded target feature by corresponding source classifier, and aggregate different predictions using respective domain weight, which corresponds to the discrepancy between each source and target. Extensive experiments are conducted on public DA benchmarks, and the results demonstrate that the proposed MDDA significantly outperforms the state-of-the-art approaches. Our source code is released at: https://github.com/daoyuan98/MDDA., Comment: Accepted by AAAI 2020

Simulation-to-real domain adaptation for semantic segmentation has been actively studied for various applications such as autonomous driving. Existing methods mainly focus on a single-source setting, which cannot easily handle a more practical scenario of multiple sources with different distributions. In this paper, we propose to investigate multi-source domain adaptation for semantic segmentation. Specifically, we design a novel framework, termed Multi-source Adversarial Domain Aggregation Network (MADAN), which can be trained in an end-to-end manner. First, we generate an adapted domain for each source with dynamic semantic consistency while aligning at the pixel-level cycle-consistently towards the target. Second, we propose sub-domain aggregation discriminator and cross-domain cycle discriminator to make different adapted domains more closely aggregated. Finally, feature-level alignment is performed between the aggregated domain and target domain while training the segmentation network. Extensive experiments from synthetic GTA and SYNTHIA to real Cityscapes and BDDS datasets demonstrate that the proposed MADAN model outperforms state-of-the-art approaches. Our source code is released at: https://github.com/Luodian/MADAN., Comment: Accepted by NeurIPS 2019

Falls are one of the important causes of accidental or unintentional injury death worldwide. Therefore, this paper presents a reliable fall detection algorithm and a mobile cloud collaboration system for fall detection. The algorithm is an ensemble learning method based on decision tree, named Falldetection Ensemble Decision Tree (FEDT). The mobile cloud collaboration system can be divided into three stages: 1) mobile stage: use a light-weighted threshold method to filter out the activities of daily livings (ADLs), 2) collaboration stage: transmit data to cloud and meanwhile extract features in the cloud, 3) cloud stage: deploy the model trained by FEDT to give the final detection result with the extracted features. Experiments show that the performance of the proposed FEDT outperforms the others' over 1-3% both on sensitivity and specificity, and more importantly, the system can provide reliable fall detection in practical scenario., Comment: 7 pages

Foot-mounted inertial positioning (FMIP) can face problems of inertial drifts and unknown initial states in real applications, which renders the estimated trajectories inaccurate and not obtained in a well defined coordinate system for matching trajectories of different users. In this paper, an approach adopting received signal strength (RSS) measurements for Wifi access points (APs) are proposed to align and calibrate the trajectories estimated from foot mounted inertial measurement units (IMUs). A crowd-sourced radio map (RM) can be built subsequently and can be used for fingerprinting based Wifi indoor positioning (FWIP). The foundation of the proposed approach is graph based simultaneously localization and mapping (SLAM). The nodes in the graph denote users poses and the edges denote the pairwise constrains between the nodes. The constrains are derived from: (1) inertial estimated trajectories; (2) vicinity in the RSS space. With these constrains, an error functions is defined. By minimizing the error function, the graph is optimized and the aligned/calibrated trajectories along with the RM are acquired. The experimental results have corroborated the effectiveness of the approach for trajectory alignment, calibration as well as RM construction., Comment: 9 figures, 6 pages, paper under review of IPIN 2017

Fingerprinting based WLAN indoor positioning system (FWIPS) provides a promising indoor positioning solution to meet the growing interests for indoor location-based services (e.g., indoor way finding or geo-fencing). FWIPS is preferred because it requires no additional infrastructure for deploying an FWIPS and achieving the position estimation by reusing the available WLAN and mobile devices, and capable of providing absolute position estimation. For fingerprinting based positioning (FbP), a model is created to provide reference values of observable features (e.g., signal strength from access point (AP)) as a function of location during offline stage. One widely applied method to build a complete and an accurate reference database (i.e. radio map (RM)) for FWIPS is carrying out a site survey throughout the region of interest (RoI). Along the site survey, the readings of received signal strength (RSS) from all visible APs at each reference point (RP) are collected. This site survey, however, is time-consuming and labor-intensive, especially in the case that the RoI is large (e.g., an airport or a big mall). This bottleneck hinders the wide commercial applications of FWIPS (e.g., proximity promotions in a shopping center). To diminish the cost of site survey, we propose a probabilistic model, which combines fingerprinting based positioning (FbP) and RM generation based on stochastic variational Bayesian inference (SVBI). This SVBI based position and RSS estimation has three properties: i) being able to predict the distribution of the estimated position and RSS, ii) treating each observation of RSS at each RP as an example to learn for FbP and RM generation instead of using the whole RM as an example, and iii) requiring only one time training of the SVBI model for both localization and RSS estimation. These benefits make it outperforms the previous proposed approaches., Comment: 10 pages, 16 figures, and 2 tables. A paper under review of IPIN 2017

Foot-mounted inertial positioning (FMIP) and fingerprinting based WiFi indoor positioning (FWIP) are two promising solutions for indoor positioning. However, FMIP suffers from accumulative positioning errors in the long term while FWIP involves a very labor-intensive offline training phase. A new approach combining the two solutions is proposed in this paper, which can limit the error growth in FMIP and is free of any offline site survey phase. This approach is realized in the framework of a particle filter, where each particle denotes a potential trajectory of the user and is weighted according to its consistency in signal strength space. Compared with the traditional Gaussian process based approaches, the proposed one has less computational cost and is free from any prior information in the position domain, such as the positions of access points, received signal strengths at certain positions and so on. An experiment is carried out to demonstrate the performance of the proposed approach compared to the traditional Gaussian process based approach., Comment: 12 figures. Accepted by European Navigation Conference 2017, Lausanne