Search

The key to multimodal sarcasm detection is effective to align and fuse the features of different modes. However, the existing multimodal data fusion methods ignore the relationship between multimodal intercomponent structures. Also, the importance of common features associated with sarcastic emotions in multimodal data is overlooked in the process of recognizing sarcasm. To address the above problems, this paper proposed a model based on cross-modal hierarchical interaction networks and contrastive learning (CMHICL). Firstly, the cross-modal hierarchical interaction network employed a minimal unit alignment module based on the cross-attention mechanism and a compositional structure fusion module based on the graph attention network to identify inconsistencies between text and images at different levels, and determined the samples with low consistency as sarcasm samples. Secondly, two contrastive learning tasks, based on data enhancement and category enhancement, helped to learn common features related to sarcasm and reduce false correlations within the modality. The experimental results show that the CMHICL model has increased the Acc by 0.81% and the F₁ value by 1.6% compared to the baseline models, which verifies the key role of the hierarchical interactive network and contrastive learning method proposed in this paper in multimodal sarcasm detection. [ABSTRACT FROM AUTHOR]

In recommender systems, extracting higher order information about the graph using methods such as graph convolutional network alleviates the cold start problem. In order to fuse the deep feature extraction capability of neural network collaborative filtering on this basis, this paper proposed a two-channel collaborative filtering recommendation algorithm (GCNCF-2C) based on graph convolution. Firstly, it divided the recommendation problem into an upstream task and a downstream task. Secondly, to form a feature representation of nodes, in the upstream task, a pre-trained encoder extracted node features and higher-order features of the graph using a one-dimensional convolutional layer containing residuals and multiple graph convolutional layers in two independent channels. Finally, the decoder performed end-to-end training by node features for rating prediction. Experiments on the datasets MovieLens-100k and MovieLens-1M show that compared to the baseline models, the algorithm improves RMSE metrics by 1.72% and 1.76% on average and MAE metrics by 2.7% and 1.98% on average on the two datasets. Meanwhile, it improves RMSE metrics by 5.9% on average in the cold-start experiments based on users and projects, which shows overall performance. [ABSTRACT FROM AUTHOR]

Copyright of Journal of Computer Engineering & Applications is the property of Beijing Journal of Computer Engineering & Applications Journal Co Ltd. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)