Search

The Institute for Defense Analyses (IDA) produces a variety of researchdeliverables such as reports, memoranda, slides, and other formats for oursponsors. Summarizing keywords from these products quickly and foreffi cient retrieval of information on given research topics poses achallenge. IDA has numerous initiatives for tagging products with IDA-defi ned taxonomies of research terms, but this is a manual and time-consuming process and must be repeated periodically to tag newerproducts. To address this challenge, we developed a Python-basedautomated tagging pipeline. In this article, we introduce the mechanics ofthis pipeline, highlight current results, and discuss future applications foranalyzing IDA’s research in terms of these tags. [ABSTRACT FROM AUTHOR]

Identifying and understanding visitor needs and expectations is of the utmost importance for a number of stakeholders and policymakers involved in the touristic domain. Apart from traditional forms of feedback, an abundance of related information exists online, scattered across various data sources like online social media, tourism-related platforms, traveling blogs, forums, etc. Retrieving and analyzing the aforementioned content is not a straightforward task and in order to address this challenge, we have developed the StreetLines platform, a novel information system that is able to collect, analyze and produce insights from the available tourism-related data. Its highly modular architecture allows for the continuous monitoring of varying pools of heterogeneous data sources whose contents are subsequently stored, after preprocessing, in a data repository. Following that, the aforementioned data feed a number of independent and parallel processing modules that extract useful information for all individuals involved in the tourism domain, like place recommendation for visitors and sentiment analysis and keyword extraction reports for professionals in the tourism industry. The presented platform is an outcome of the StreetLines project and apart from the contributions of its individual components, its novelty lies in the holistic approach to knowledge extraction and tourism data mining. [ABSTRACT FROM AUTHOR]

During the information retrieval process, individuals locate relevant web pages by entering specific keywords. Nevertheless, if users provide inaccurate keywords or if these keywords are absent from the intended page, the effectiveness of information retrieval will be significantly compromised. Thus, the role of keywords in text processing remains of utmost importance. Particularly in intricate contexts, relying on manual analysis by readers can prove to be both time-intensive and unfeasible. Most existing methods are addressed with limited accuracy, leading to elevated error rates and compromised training capabilities. To overcome these limitations, the proposed approach introduces an automated keyword extraction and ranking system based on deep learning. Several key stages, like data acquisition, pre-processing, tokenization, word-to-vector transformation, keyword classification, and ranking, are used. The effectiveness of this keyword extraction process is evaluated using 500N-KPCrowd, KPTimes, and KP20k datasets. During text pre-processing, eliminating stop words, applying Parts of Speech (PoS) tagging, stemming, and sentence segmentation are undertaken. The pre-processed text is fed into the Deep-KeywordNet model, while the pre-processed input is tokenized into individual words. The Word2Vec (W2V) Skip-gram embedding layer facilitates the categorization of distributed vector representations. The Attention Bidirectional Long Short-Term Memory Gated Convolutional Neural Network (Attn Bi-GCNN), along with the softmax layer, assign class labels, and the network's loss optimization employs the Dwarf Mongoose Algorithm (DMA). Significant keywords are ranked using the Term Frequency-Inverse Average Document Frequency (TF-IADF) model. Remarkably, the overall accuracy achieved through the implementation in PYTHON stands at 98.87%, with a minimized time complexity. [ABSTRACT FROM AUTHOR]

A large petrochemical construction project is typically executed by multiple parties, all bound by contract agreement. During the execution phase, issues and problems may arise because the work details are not clearly specified in the contractual agreement. These issues are formally communicated and documented through written correspondence letters. By identifying important keywords within these formal letters, a comprehensive narrative of the project, including its associated issues, can be identified and analyzed. In this research, we introduce an adjusted TextRank algorithm that integrates external features from the Indonesian FastText language model and term frequency-inverse document frequency (TF-IDF) scores to identify important keywords within a dataset of correspondence letters of petrochemical projects. This enhancement involves refining phrase detection, semantic relationship estimation between words, and part-ofspeech (POS) identification for words or phrases. Our results show that the proposed adjustments result in improved evaluation scores compared to the baseline standard TextRank and standard TF-IDF, respectively by 24.1% and 25% in terms of F-1 scores. [ABSTRACT FROM AUTHOR]

Identifying and understanding visitor needs and expectations is of the utmost importance for a number of stakeholders and policymakers involved in the touristic domain. Apart from traditional forms of feedback, an abundance of related information exists online, scattered across various data sources like online social media, tourism-related platforms, traveling blogs, forums, etc. Retrieving and analyzing the aforementioned content is not a straightforward task and in order to address this challenge, we have developed the StreetLines platform, a novel information system that is able to collect, analyze and produce insights from the available tourism-related data. Its highly modular architecture allows for the continuous monitoring of varying pools of heterogeneous data sources whose contents are subsequently stored, after preprocessing, in a data repository. Following that, the aforementioned data feed a number of independent and parallel processing modules that extract useful information for all individuals involved in the tourism domain, like place recommendation for visitors and sentiment analysis and keyword extraction reports for professionals in the tourism industry. The presented platform is an outcome of the StreetLines project and apart from the contributions of its individual components, its novelty lies in the holistic approach to knowledge extraction and tourism data mining.

Construction sites are well known for the inherent risks that negatively impact the safety and well-being of workers. Identifying and minimising these hazards is critical for preventing accidents and creating a safe working environment. Traditional techniques of hazards identification in construction rely on visual assessments and professional expertise, which can be time-consuming and subjective. The goal of this research is to identify traits that indicate potential dangers in the construction industry by extracting meaningful information from accident narratives. This will be achieved through the application of a rule-based iteration approach, using the Natural Language Toolkit (NLTK) for keyword extraction and text tokenization. It is a branch of artificial intelligence and computational linguistics concerned with the interaction of computers and human language. The research methodology involves the utilization of NLTK and the application of a rule-based iteration approach to extract hazards from construction-related accident narratives. The proposed approach includes gathering accident narratives, pre-processing data, and textual analysis with NLP tool for information extraction and training the algorithm with identified attributes. The textual analysis eventually leads to the extraction of significant sources of dangers that cause accidents. The study contributes to the developing subject of construction safety management by utilizing the capabilities of NLP to enhance hazard detection, resulting in safer construction practices and lower occupational hazards. The findings emphasise the accuracy with which NLP approaches detect dangers, allowing construction professionals to proactively decrease risks and enhance overall safety on construction sites.

Keyword extraction is an essential tool for many text mining applications such as automatic indexing, summarizing, classification, clustering and automatic filtering. Automated keyword extraction is essential as the daily text data to be reached and processed have increased tremendously over the Internet, e.g. millions of news articles are published daily online. In this paper, a novel ensemble model for automatic extraction of keywords from news articles is proposed. The proposed model handles keyword extraction as a sequence labeling task. Two sub-modules representing the statistical and graphical features by their calculated scores for each input token were combined in the token classification module. The Ensemble Token Classification module was trained and tested separately with the ensemble algorithms Random Forest, XgBoost, Decision Tree and Voting Classification. For training, we collected two news datasets from Kazakh and Russian news sites published in Cyrillic alphabet. We also collected an Arabic news dataset, ArabianNews. The performance of the model was also compared with the widely used 500N-KPCrowd dataset in the literature, which consists of English news content in Latin alphabet. The proposed model achieved the best performance with an F 1 -score of 0.71 and 0.86 on the 500N-KPCrowd and Russian datasets, respectively. We attained the best F 1 -score (0.97) with the KazakhNews and ArabianNews datasets. [ABSTRACT FROM AUTHOR]

Nowadays, opinion analysis and text classification become interesting tasks in social media studies. Moreover, keyword extraction technology is important research topic and the basis of building corpora, information retrieval, text analysis and text classification...etc. Most studies carry out on the binary problem classification which has historically received more attention in the machine learning community compared to multi-class classification problem. They are often done on academic languages, leaving aside the dialects despite of their increasing use in the social networks. There is very little effort that has been dedicated in the Arabic language especially its dialects such as Algerian dialect, intended for the analysis of opinions. In this work, we aim to realize an innovative and original approach for multi-class classification task and subject detection in the field of marketing. These classes are considered as subjects of the Algerian posts. We collected dataset from Facebook and annotated them with 11 labels. We applied TF-IDF word embedding method to vectorise and extract keywords by giving a weight for each token. In the final stage, our model was trained using input vectors. We applied a deep neural network on our annotated dataset. We achieved a precision of 83%. [ABSTRACT FROM AUTHOR]

In this paper, we construct an academic literature knowledge graph based on the relationship between documents to facilitate the storage and research of academic literature data. Keywords are an important type of node in the knowledge graph. To solve the problem that there are no keywords in some documents for several reasons in the process of knowledge graph construction, an improved keyword extraction algorithm called TP-CoGlo-TextRank is proposed by using word frequency, position, word co-occurrence frequency, and a word embedding model. By combining the word frequency and position in the document, the importance of words is distinguished. By introducing the GloVe word-embedding model, which brings the external knowledge of documents into the TextRank algorithm, and combining the internal word co-occurrence frequency in the documents, the word-adjacency relationship is transferred non-uniformly. Finally, the words with the highest scores are combined into phrases if they are adjacent in the original text. The validity of the TP-CoGlo-TextRank algorithm is verified by experiments. On this basis, the Neo4j graph database is used to store and display the academic literature knowledge graph, to provide data support for research tasks such as text clustering, automatic summarization, and question-answering systems. [ABSTRACT FROM AUTHOR]

Construction sites are well known for the inherent risks that negatively impact the safety and well-being of workers. Identifying and minimising these hazards is critical for preventing accidents and creating a safe working environment. Traditional techniques of hazards identification in construction rely on visual assessments and professional expertise, which can be time-consuming and subjective. The goal of this research is to identify traits that indicate potential dangers in the construction industry by extracting meaningful information from accident narratives. This will be achieved through the application of a rule-based iteration approach, using the Natural Language Toolkit (NLTK) for keyword extraction and text tokenization. It is a branch of artificial intelligence and computational linguistics concerned with the interaction of computers and human language. The research methodology involves the utilization of NLTK and the application of a rule-based iteration approach to extract hazards from construction-related accident narratives. The proposed approach includes gathering accident narratives, pre-processing data, and textual analysis with NLP tool for information extraction and training the algorithm with identified attributes. The textual analysis eventually leads to the extraction of significant sources of dangers that cause accidents. The study contributes to the developing subject of construction safety management by utilizing the capabilities of NLP to enhance hazard detection, resulting in safer construction practices and lower occupational hazards. The findings emphasise the accuracy with which NLP approaches detect dangers, allowing construction professionals to proactively decrease risks and enhance overall safety on construction sites. [ABSTRACT FROM AUTHOR]

Topic modelling is a text mining technique for identifying salient themes from a number of documents. The output is commonly a set of topics consisting of isolated tokens that often co-occur in such documents. Manual effort is often associated with interpreting a topic's description from such tokens. However, from a human's perspective, such outputs may not adequately provide enough information to infer the meaning of the topics; thus, their interpretability is often inaccurately understood. Although several studies have attempted to automatically extend topic descriptions as a means of enhancing the interpretation of topic models, they rely on external language sources that may become unavailable, must be kept up to date to generate relevant results, and present privacy issues when training on or processing data. This paper presents a novel approach towards extending the output of traditional topic modelling methods beyond a list of isolated tokens. This approach removes the dependence on external sources by using the textual data themselves by extracting high-scoring keywords and mapping them to the topic model's token outputs. To compare how the proposed method benchmarks against the state of the art, a comparative analysis against results produced by Large Language Models (LLMs) is presented. Such results report that the proposed method resonates with the thematic coverage found in LLMs and often surpasses such models by bridging the gap between broad thematic elements and granular details. In addition, to demonstrate and reinforce the generalisation of the proposed method, the approach was further evaluated using two other topic modelling methods as the underlying models and when using a heterogeneous unseen dataset. To measure the interpretability of the proposed outputs against those of the traditional topic modelling approach, independent annotators manually scored each output based on their quality and usefulness as well as the efficiency of the annotation task. The proposed approach demonstrated higher quality and usefulness, as well as higher efficiency in the annotation task, in comparison to the outputs of a traditional topic modelling method, demonstrating an increase in their interpretability. [ABSTRACT FROM AUTHOR]

In today’s information-driven world, open data is crucial in making valuable structured data freely accessible to the public. However, the absence of quality metadata often hinders the findability and representation of this data. In this study we specifically focus on keywords, proposing a strategy for their automatic generation. In particular, we employed five existing keyword extraction methodologies (BERT, RAKE, YAKE, TEXTRANK, and ChatGPT) and proposed a novel hybrid methodology, named BRYT (read as bright). Our evaluation of these algorithms was conducted using Gestalt String Matching and Jaccard Similarity techniques. We validated our study using a selection of datasets from the EU data portal, specifically choosing those that exhibited potentially high-quality metadata. This included datasets that contained a substantial number of keywords and had comprehensive, relevant metadata. The results showed that 69.1% of the dataset keywords majorly matched (more than 50% or 5 keywords), 24.7% minorly matched (up to 50% or 5 keywords), and 6.2% did not match. The proposed hybrid model, BRYT, outperformed other algorithms in the major matches, while ChatGPT was a close second. YAKE outperformed the others in minor matches, and ChatGPT was again a close second. The evaluations concluded that BRYT consistently extracted more representative keywords in major matches, highlighting its effectiveness in improving findability. This study sets up a favorable field for further representative metadata extraction and population, making the data more findable, discoverable, and accessible.

Summary generation is an important research direction in natural language processing. Aimed at the problems of redundant information processing difficulties and an inability to generate high-quality summaries from long text in existing summary generation models, BART is the backbone model, an N + 1 coarse–fine-grained multistage summary generation framework is constructed, and a multistage mixed-attention unsupervised keyword extraction summary generation model is proposed (multistage mixed-attention unsupervised keyword extraction for summary generation, MSMAUKE-SummN). In the N-coarse-grained summary generation stages, a sentence filtering layer (PureText) is constructed to remove redundant information in long text. A mixed-attention unsupervised approach is used to iteratively extract keywords, assisting summary inference and enriching the global semantic information of coarse-grained summaries. In the 1-fine-grained summary generation stage, a self-attentive keyword selection module (KeywordSelect) is designed to obtain keywords with higher weights and enhance the local semantic representation of fine-grained summaries. Tandem N-coarse-grained and 1-fine-grained summary generation stages are used to obtain long text summaries through a multistage generation approach. The experimental results show that the model improves the ROUGE-1, ROUGE-2, and ROUGE-L metrics by a minimum of 0.75%, 1.48%, and 1.25% over the HMNET, TextRank, HAT-BART, DDAMS, and SummN models on summarized datasets such as AMI, ICSI, and QMSum. [ABSTRACT FROM AUTHOR]

In today's world, where data and information are increasingly proliferating, text summarization and technologies play a critical role in making large amounts of text data more accessible and meaningful. In business, the news industry, academic research, and many other fields, text summarization helps make quick decisions, access information faster, and manage resources more effectively. Additionally, text summarization research is conducted to further improve these technologies and develop new methods and algorithms to provide better summarization of texts. Therefore, text summarization and research in this field are of great importance in the information age. In this study, a new operating model for text summarization that can be applied to different algorithms is proposed and evaluated. Sixteen summarization algorithms covering six approaches (statistical, graph-based, content-based, pointer-based, position-based, and user-oriented) were implemented and tested on 50 different full-text article datasets. Four evaluation criteria (BLEU, Rouge-N, Rouge-L, METEOR) were used to assess the similarity between the generated summaries and the original summaries. The performance of the algorithms within each approach was averaged and the overall best-performing algorithm was selected. This best algorithm was subjected to further analysis through Topic Modelling and Keyword Extraction to identify key topics and keywords within the summarised text. The proposed model provides a standardized workflow for developing and thoroughly testing summarization algorithms across datasets and evaluation metrics to determine the most appropriate summarization approach. This study demonstrates the effectiveness of the model on a variety of algorithm types and text sources. [ABSTRACT FROM AUTHOR]

Expanding traditional video metadata and recommendation systems encompasses challenges that are difficult to address with conventional methodologies. Limitations in utilizing diverse information when extracting video metadata, along with persistent issues like bias, cold start problems, and the filter bubble effect in recommendation systems, are primary causes of performance degradation. Therefore, a new recommendation system that integrates high-quality video metadata extraction with existing recommendation systems is necessary. This research proposes the “Extraction of Meta-Data for Recommendation using keyword mapping,” which involves constructing contextualized data through object detection models and STT (Speech-to-Text) models, extracting keywords, mapping with the public dataset MovieLens, and applying a Hybrid recommendation system. The process of building contextualized data utilizes YOLO and Google’s Speech-to-Text API. Following this, keywords are extracted using the TextRank algorithm and mapped to the MovieLens dataset. Finally, it is applied to a Hybrid Recommendation System. This paper validates the superiority of this approach by comparing it with the performance of the MovieLens recommendation system that does not expand metadata. Additionally, the effectiveness of metadata expansion is demonstrated through performance comparisons with existing deep learning-based keyword extraction models. Ultimately, this research resolves the cold start and long-tail problems of existing recommendation systems through the construction of video metadata and keyword extraction.

Expert profiling aims to discover the expertise of an author. This task is useful for identifying the research groups existing within an organization as well as measuring the similarities between authors’ expertise. Thus, identifying areas of expertise becomes a critical part of this task, especially in cases where the publications are unannotated. Commonly used topic modeling methods such as Latent Dirichlet Allocation still fall short in determining the number of topics automatically and discovering the hierarchical relationships between topics. To solve these issues, we adopted a graph-based approach which constructs a graph from publication features such as authors and keywords (Silva et al., 2018). We applied the Louvain algorithm repeatedly to discover the topics with hierarchical order automatically. We utilize keyword extraction methods to generate keywords for each respective publication to handle the missing values. We perform experiments to determine the optimum HPMI value. Results showed that graphs constructed from default and SIFRank keywords with transformation weights of $\alpha =0.5$ and $\beta =1.0$ produce topics with the best HPMI score. We evaluate the profiles from this method (CDT) with ATM as the baseline. It is shown that CDT produces better MAP, MRR, and nDCG scores than ATM. The work in this manuscript shows how community detection and keyword extraction could be utilized in expert profiling tasks. Our observation shows that the Louvain algorithm used only cluster publications into one topic, and thus still has limitations in classifying multidisciplinary publications. Further development could be done to handle such publications and increase the quality of keywords.

Keyword extraction is a key step in document indexing. Keywords are semantic and content-based descriptors of a document, which can be used in document retrieval and representation. In databases containing scientific documents, such as Ganj in Irannian Research Institue for Information Science and Technology (IranDoc), it is even more critical to assign meaningful keywords for documents, since the documents are from different academic disciplines and contain technical terms.As the number of scientific documents grows exponentially, having an automatic and intelligent keyword extraction technique is getting more critical. There are various keyword extraction techniques that are either based on statistical features of the text or machine learning approaches, and sometimes a combination of both. In this research, we propose a new keyword extraction method for Persian scientific documents based on recommender systems and case-based reasoning. The proposed method is designed based on case-based reasoning in which the main assumption is that similar documents share similar keywords. There are two main steps in the proposed approach: first, similar documents to a given new document are retrieved based on TFIDF and word2vec model, second, the candidate keywords are extracted from retrieved documents and ranked based on a new scoring scheme, and a set of keyword are selected from the candidate keywords based on their score. The proposed method is tested and avaluated on a set of documents of Ganj database in three different subject areas (Art, Humanities and Engineering), based on precision, recall and expert panel

With the continuous development of the social economy, multimedia has been valued more and more as a computer technology with strong professional technology and high application level. The quality of multimedia professionals is predicted and analyzed for multimedia professional talent through the Internet according to a Chinese keyword extraction algorithm. It realizes the extraction of keywords through Internet intelligence information acquisition for solving the problem of Internet information explosion, aiming to solve the talent quality prediction analysis. The prediction and analysis of multimedia professional talent quality play a crucial role in talent recruitment and development in the ever-evolving multimedia industry. This paper constructed a Fuzzy Secured Hybrid Search (FSHS) for keyword extraction in the Chinese Language. The proposed FSHS model computes the features in the text for the computation of the talent quality prediction for the extraction of the keywords. Through the utilization of the fuzzy logic model, the features in the text are computed and classification is performed classification and extraction of the features. The simulation results show that the Chinese keyword extraction algorithm has a high recall rate and precision rate, and can effectively predict the quality of professional talents. [ABSTRACT FROM AUTHOR]

Attempts to express information from various documents in graph form are rapidly increasing. The speed and volume in which these documents are being generated call for an automated process, based on machine learning techniques, for cost-effective and timely analysis. Past studies responded to such needs by building knowledge graphs or technology trees from the bibliographic information of documents, or by relying on text mining techniques in order to extract keywords and/or phrases. While these approaches provide an intuitive glance into the technological hotspots or the key features of the select field, there still is room for improvement, especially in terms of recognizing the same entities appearing in different forms so as to interconnect closely related technological concepts properly. In this paper, we propose to build a patent knowledge network using the United States Patent and Trademark Office (USPTO) patent filings for the semiconductor device sector by fine-tuning Huggingface's named entity recognition (NER) model with our novel edge weight updating neural network. For the named entity normalization, we employ edge weight updating neural network with positive and negative candidates that are chosen by substring matching techniques. Experiment results show that our proposed approach performs very competitively against the conventional keyword extraction models frequently employed in patent analysis, especially for the named entity normalization (NEN) and document retrieval tasks. By grouping entities with named entity normalization model, the resulting knowledge graph achieves higher scores in retrieval tasks. We also show that our model is robust to the out-of-vocabulary problem by employing the fine-tuned BERT NER model. [ABSTRACT FROM AUTHOR]

Keyword extraction is a key step in document indexing. Keywords are semantic and content-based descriptors of a document, which can be used in document retrieval and representation. In databases containing scientific documents, such as Ganj in Irannian Research Institue for Information Science and Technology (IranDoc), it is even more critical to assign meaningful keywords for documents, since the documents are from different academic disciplines and contain technical terms. As the number of scientific documents grows exponentially, having an automatic and intelligent keyword extraction technique is getting more critical. There are various keyword extraction techniques that are either based on statistical features of the text or machine learning approaches, and sometimes a combination of both. In this research, we propose a new keyword extraction method for Persian scientific documents based on recommender systems and case-based reasoning. The proposed method is designed based on case-based reasoning in which the main assumption is that similar documents share similar keywords. There are two main steps in the proposed approach: first, similar documents to a given new document are retrieved based on TFIDF and word2vec model, and second, the candidate keywords are extracted from retrieved documents and ranked based on a new scoring scheme, and a set of keywords are selected from the candidate keywords based on their score. The proposed method is tested and evaluated on a set of documents of Ganj database in three different subject areas (Art, Humanities and Engineering), based on precision, recall and expert panel. [ABSTRACT FROM AUTHOR]

The present study extract, map and compare the lexical and semantic similarity of terms from author-provided keywords with machine extracted terms and topics from titles and abstracts of an inter-disciplinary field like 'digital humanities'. Author-provided terms (keywords) were first extracted and mapped through visualization software like Gephi and then these extracted terms were compared with terms extracted from title and abstract of the research articles through NLP based statistical modules. Also, the interdisciplinary of significant topics were measured through the Brillouin index. A set of 7483 articles downloaded from Scopus database on the domain of digital humanities and its associated fields were used for the purpose. We observed the researches on digital humanities are spread over a considerable number of concepts like 'Industry 4.0', 'topic modelling, 'open science'. Further, the machine algorithm-based extraction compared and identified a larger lexical similarity between these author-provided keywords and title-extracted keywords, rather than abstract-extracted keywords. Jaccard similarity of all author-keywords with machine extracted title keywords came 0.83 and SBERT BiEncoder_score was 0.7374. The top research areas extracted from titles, through unsupervised approach of term extraction resulted in topics like digital humanities approach, digital humanities visualization, indicating a strong connection to the discipline of digital humanities. The average interdisciplinarity index of top significant topics came between 1.217 and 1.284, with the highest index value for 'computational digital humanities'. As this study is based on real-world data, it is highly useful to understand how far machine algorithm-based text extraction can be helpful for information retrieval process. [ABSTRACT FROM AUTHOR]

Copyright of HMD: Praxis der Wirtschaftsinformatik is the property of Springer Nature and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Keyword extraction is the task of identifying essential words in a lengthy document. This process is primarily executed through supervised keyword extraction. In instances where the dataset is limited in size, a classification-based approach is typically employed. Therefore, this paper introduces a novel keyword extractor based on a classification approach. The proposed keyword extractor comprises three key components: RoBERTa, a keyword estimator, and a decision rule. RoBERTa encodes an input document, the keyword estimator calculates the probability of each token in the document becoming a keyword, and the decision rule ultimately determines whether each token is a keyword based on these probabilities. However, training the proposed model with a small dataset presents two challenges. One problem is the case that all tokens in the documents are not a keyword, and the other problem is that a single word can be composed of keyword tokens and non-keyword tokens. Two novel heuristics are thus proposed to tackle these problems. To address these issues, two novel heuristics are proposed. These heuristics have been extensively tested through experiments, demonstrating that the proposed keyword extractor surpasses both the generation-based approach and the vanilla RoBERTa in environments with limited data. The efficacy of the heuristics is further validated through an ablation study. In summary, the proposed heuristics have proven to be effective in developing a supervised keyword extractor with a small dataset. [ABSTRACT FROM AUTHOR]

The research of Chinese and Japanese machine translation technology is of great practical significance for promoting and inheriting Chinese excellent culture and advancing the development of economy, education, and culture in China and Japan. In this paper, a cross-lingual keyword extraction machine translation model is investigated. The traditional model is improved on the basis of recurrent neural networks RNN and attention mechanism, and the problem of low information memory capacity of LSTM and GRU algorithms is optimized. By introducing the self-attention mechanism, important information can be filtered from a large amount of information, and a keyword recognition system can be constructed. The system covers three levels: word embedding layer, encoding layer, and CRF layer, and the model effectively utilizes the feature extraction mechanism of the Transformer model and introduces the bidirectional mechanism to obtain the richer semantic representation of word vectors. After the above strategies, the scores on the task of extracting the keywords of the discourse system with Chinese characteristics reach up to 89 in word inference and 83 in cultural adaptation, which effectively improves the performance of machine translation compared with other neural machine translators. The keyword “community of destiny” appears frequently in Japanese media reports, and the content involves the excellent cultural essence of the discourse system with Chinese characteristics, which has a benign influence on the dissemination process in Japanese society.