Search

Abstractive Text summarization aims to generate a short summary for a document while preserving salient information. Recently, contrastive learning has been extended from visual representation to summarization tasks. At present, the methods of contrastive learning summarization focus on modeling the global semantics of source documents, targets and candidate summaries to maximize their similarities. However, they ignore the influence of sentence semantics in the document. In this paper, we propose a sentence-level salience contrastive learning method to help the model capture salient information and denoise. The model expresses the sentence salience according to the semantic similarity between the summaries and sentences of the source document, and integrates the similarity distance into the contrastive loss in the form of soft weights. Therefore, our model maximize the similarity between summaries and salient information, while minimizing the similarity between summaries and potential noise. We have verified our method in three widely used datasets, CNN/Daily Mail, XSum and PubMed. The experimental results show that the proposed method can significantly improve the baseline performance and achieve competitive performance in the existing contrastive learning methods. • We propose a contrastive learning method for abstractive text summarization. • We construct contrastive samples in the source sentences. • We incorporate the salient distance-aware weightsinto the contrastive loss. • Experiments show that our model has obviously better performance. [ABSTRACT FROM AUTHOR]

Abstractive dialogue summarization requires distilling and rephrasing key information from noisy multi-speaker documents. Combining pre-trained language models with input augmentation techniques has recently led to significant research progress. However, existing solutions still struggle to select relevant chat segments, primarily relying on open-domain and unsupervised annotators not tailored to the actual needs of the summarization task. In this paper, we propose DearWatson , a task-aware utterance-level annotation framework for improving the effectiveness and interpretability of pre-trained dialogue summarization models. Precisely, we learn relevant utterances in the source document and mark them with special tags, that then act as supporting evidence for the generated summary. Quantitative experiments are conducted on two datasets made up of real-life messenger conversations. The results show that DearWatson allows model attention to focus on salient tokens, achieving new state-of-the-art results in three evaluation metrics, including semantic and factuality measures. Human evaluation proves the superiority of our solution in semantic consistency and recall. Finally, extensive ablation studies confirm each module's importance, also exploring different annotation strategies and parameter-efficient fine-tuning of large generative language models. [ABSTRACT FROM AUTHOR]

In this paper, we focus on analyzing the relationship between the input of source text and source image, and then through the integration and generalization of the multi-modal information (e.g., texts and images), outputs the multi-modal summarization including text and image. However, existing multi-modal summarization methods face several challenges: (1) Different modalities exist in different semantic spaces, and expressing the multi-modal information of source modalities in a similar semantic representation space is crucial. (2) For the result of text summarization, it is crucial to explore how to capture both the differences and similarities within the source modalities, which can reduce redundancy to improve the quality of text summarization. In order to overcome the challenge above, Multi-Modal Anchor Adaptation Learning for Multi-Modal Summarization (MA-Sum) has been proposed. Specifically, MA-Sum employs a novel and highly efficient image anchor selection method, which selects the object sample containing the richest image information as the image anchor. Simultaneously, it carefully selects the text sentence closely intertwined with the image semantics to serve as the language anchor. Therefore, the multi-modal anchor can be seen as a bridge for multi-modal alignment to alleviate the semantic gap between textual and visual. Moreover, based on the distance between the anchors and the semantic information in the respective modal, the positive and negative semantic information of each modal will be distinguished. Based on negative semantic information, the counterfactual learning mechanism is constructed to optimize the result of multi-modal summarization. Finally, the process of multi-modal features interaction is optimized by image summary which is chosen by using multi-modal anchors. According to the experimental results, compared with the state-of-art multi-modal summarization, our proposed MA-Sum can be optimized in terms of summarization consistency and completeness, so as to obtain the optimal multi-modal summarization metric. [ABSTRACT FROM AUTHOR]

At present, most Chinese text summarization algorithms use the sequence-to-sequence model, but this model is prone to the problems of unknown words and incomplete content generation. To address these problems, we propose a new two-stage automatic text summarization method using keyword information and adversarial learning in this paper. On the one hand, the proposed method integrates the keyword information into the sequence-to-sequence model. The main information and keywords of the article are considered simultaneously through the attention mechanism to improve the information of summary generation. On the other hand, adversarial learning is introduced into the proposed model to avoid the problem that the semantic vector after passing through the encoder cannot save the context information better. Experiments are carried out on the Chinese dataset LCSTS, and the comparison results show that the proposed method has advantages in abstractive summarization. [ABSTRACT FROM AUTHOR]

Although pretrained language models achieve high performance on various natural language processing tasks, they still require further improvements in the sentence embedding task. Many studies have improved performance in this task using pre-trained language models and contrastive learning, but these approaches are limited because they are based on naive average pooling and CLS tokens. Therefore, we propose an advanced sentence-embedding method based on weighted pooling that considers token importance. Specifically, the token importance is calculated by combining an explainable artificial-intelligence module with a text summarization model, and the final sentence embedding is derived through weighted pooling token embedding and token importance. Thus, we derive a sentence embedding that considers both the local information of the token embedding and the global information of the entire sentence. Experimental results reveal that our proposed sentence embedding outperforms other models on both text similarity tasks and text classification. Moreover, the proposed method's robustness is verified through the results of an ablation study. [ABSTRACT FROM AUTHOR]

Abstractive text summarization aims to generate a brief version of a given sentence while attempting to express its main meaning. Although some models based on the sequence-to-sequence framework have achieved remarkable results recently, there are still many problems that cannot be ignored. In this paper, we propose a selective reinforced sequence-to-sequence (i.e. Seq2Seq) attention model for abstractive social media text summarization. We add a selective gate after the encoder module for better filtering out invalid information. Specifically, we combine the cross-entropy and reinforcement learning policy for optimizing the ROUGE score directly. Evaluations on a famous social media dataset (i.e. LCSTS) demonstrate that our model outperforms most of famous baseline models, and the proposed model is 2.6% , 2.1% and 2.5% higher than the basic Seq2Seq attention model on the F 1 score of ROUGE-1, ROUGE-2, and ROUGE-L, respectively. [ABSTRACT FROM AUTHOR]

Generative transformer-based models have achieved state-of-the-art performance in text summarization. Nevertheless, they still struggle in real-world scenarios with long documents when trained in low-resource settings of a few dozen labeled training instances, namely in low-resource summarization (LRS). This paper bridges the gap by addressing two key research challenges when summarizing long documents, i.e., long-input processing and document representation, in one coherent model trained for LRS. Specifically, our novel align-then-abstract representation learning model (Athena) jointly trains a segmenter and a summarizer by maximizing the alignment between the chunk-target pairs in output from the text segmentation. Extensive experiments reveal that Athena outperforms the current state-of-the-art approaches in LRS on multiple long document summarization datasets from different domains. [ABSTRACT FROM AUTHOR]