Search

We introduce a top-down approach to discourse parsing that is conceptually simpler than its predecessors (Kobayashi et al., 2020; Zhang et al., 2020). By framing the task as a sequence labelling problem where the goal is to iteratively segment a document into individual discourse units, we are able to eliminate the decoder and reduce the search space for splitting points. We explore both traditional recurrent models and modern pre-trained transformer models for the task, and additionally introduce a novel dynamic oracle for top-down parsing. Based on the Full metric, our proposed LSTM model sets a new state-of-the-art for RST parsing., Accepted at EACL 2021

In this paper, we propose FFCI, a framework for fine-grained summarization evaluation that comprises four elements: faithfulness (degree of factual consistency with the source), focus (precision of summary content relative to the reference), coverage (recall of summary content relative to the reference), and inter-sentential coherence (document fluency between adjacent sentences). We construct a novel dataset for focus, coverage, and inter-sentential coherence, and develop automatic methods for evaluating each of the four dimensions of FFCI based on cross-comparison of evaluation metrics and model-based evaluation methods, including question answering (QA) approaches, semantic textual similarity (STS), next-sentence prediction (NSP), and scores derived from 19 pre-trained language models. We then apply the developed metrics in evaluating a broad range of summarization models across two datasets, with some surprising findings., Comment: Accepted at Journal of Artificial Intelligence Research (JAIR 2022)

In this study, we propose InSet, an Indonesian sentiment lexicon built to identify written opinion and categorize it into positive or negative opinion, which could be utilized to analyze public sentiment towards particular topic, event, or product. Composed using collection of words from Indonesian tweet, InSet was constructed by manually weighting each words and enhanced by adding stemming and synonym set. As the result, we obtained 3,609 positive words and 6,609 negative words with score ranging between −5 and +5. Based on the experiment utilizing the InSet, our method outperforms other rarely found Indonesian lexicon that we used as baseline.

In this paper, we report the first work ever of detecting the summarizable chat conversation in order to improve the quality of summarization and system performance, especially in real time server-based system like online instant messaging. Summarizable chat conversation means that the document assessed could produce a meaningful summary for human. Our study intends to answer the question: what are the characteristics of a summarizable chat and how to distinguish it with non-summarizable chat conversation. To conduct the experiment, corpora of 536 chat conversations was constructed manually. Technically, we used 19 attributes and grouped them by feature sets of (1) chat attribute, (2) lexical, and (3) Rapid Automatic Keyword Extraction (RAKE). As result, our work reveals that the features can classify summarizable chat by 78.36 % as our highest accuracy, performed by feature selection with SVM.

As one of the largest Social Media in providing public data every day, Twitter has attracted the attention of researcher to investigate, in order to mine public opinion, which is known as Sentiment Analysis. Consequently, many techniques and studies related to Sentiment Analysis over Twitter have been proposed in recent years. However, there is no study that discuss about sentence pattern of positive/negative sentence and neither subjective/objective sentence. In this paper we propose POS sequence as feature to investigate pattern or word combination of tweets in two domains of Sentiment Analysis: subjectivity and polarity. Specifically we utilize Information Gain to extract POS sequence in three forms: sequence of 2-tags, 3-tags, and 5-tags. The results reveal that there are some tendencies of sentence pattern which distinguish between positive, negative, subjective and objective tweets. Our approach also shows that feature of POS sequence can improve Sentiment Analysis accuracy.

This paper presents a study on natural expressive speech during public talks. Specifically, we focus on how people convey important messages that may be retained in the audience’s consciousness. Our study aims to answer several questions. Why are some public speeches memorable and inspirational for the audience, while others are not? Why are some memorable/inspirational spoken quotes more popular than others? Being able to evaluate why certain spoken words are memorable/inspirational is not a trivial matter, and most studies on memorable quote detection are only limited to textual data. In this study, we use both linguistic and acoustic features of public speeches in TED talks. The results reveal that based on those linguistic and acoustic features, we are able to distinguish memorable spoken quotes and non-memorable spoken quotes with 70.4 % accuracy. Furthermore, we also analyze the important factors that affect the memorableness and popularity of spoken quotes.

In this paper, we address the problem of automatic speech summarization on open-domain TED talks. The large vocabulary and diversity of topics from speaker-to-speaker presents significant difficulties. The challenges increase not only how to handle disfluencies and fillers, but also how to extract topic-related meaningful messages within the free talks. Here, we propose to incorporate semantic and acoustic features within the speech summarization technique. In addition, we also propose a new evaluation method for speech summarization by checking semantic similarity between system and human summarization. Experiments results reveal that the proposed methods are effective in spontaneous speech summarization.

In this paper we present the construction and analysis of memorable spoken quote corpora from TED public speaking. Memorable quotes are interesting and useful words which usually contain generic pearls of wisdom that could achieve public awareness and retained in people consciousness. Our study aims to reveal why can some public speeches can be retained in people mind and make their consciousness to like and share it, while some others can not. To achieve this purpose, the relevance corpora is required to perform system quantitative evaluation. In this study, we start with the collection of the corpus from TED public speaking. Specifically, we utilize 899 video files of TED Talks and more than 2000 speech quotes annotated by TED team. We then complement the data with non-memorable quotes. According to shares number of quotes which are provided by TED, we also annotate memorable quotes with popularity factor. Analysis of memorable spoken quotes is done based on speech duration, F0, and popularity.