1. Pre-Training and Prompting for Few-Shot Node Classification on Text-Attributed Graphs
- Author
-
Zhao, Huanjing, Yang, Beining, Cen, Yukuo, Ren, Junyu, Zhang, Chenhui, Dong, Yuxiao, Kharlamov, Evgeny, Zhao, Shu, and Tang, Jie
- Subjects
Computer Science - Social and Information Networks ,Computer Science - Artificial Intelligence ,Computer Science - Machine Learning - Abstract
The text-attributed graph (TAG) is one kind of important real-world graph-structured data with each node associated with raw texts. For TAGs, traditional few-shot node classification methods directly conduct training on the pre-processed node features and do not consider the raw texts. The performance is highly dependent on the choice of the feature pre-processing method. In this paper, we propose P2TAG, a framework designed for few-shot node classification on TAGs with graph pre-training and prompting. P2TAG first pre-trains the language model (LM) and graph neural network (GNN) on TAGs with self-supervised loss. To fully utilize the ability of language models, we adapt the masked language modeling objective for our framework. The pre-trained model is then used for the few-shot node classification with a mixed prompt method, which simultaneously considers both text and graph information. We conduct experiments on six real-world TAGs, including paper citation networks and product co-purchasing networks. Experimental results demonstrate that our proposed framework outperforms existing graph few-shot learning methods on these datasets with +18.98% ~ +35.98% improvements., Comment: Accepted to KDD'24
- Published
- 2024
- Full Text
- View/download PDF