Search

Background: Applying graph convolutional networks (GCN) to the classification of free-form natural language texts leveraged by graph-of-words features (TextGCN) was studied and confirmed to be an effective means of describing complex natural language texts. However, the text classification models based on the TextGCN possess weaknesses in terms of memory consumption and model dissemination and distribution. In this paper, we present a fast message passing network (FastMPN), implementing a GCN with message passing architecture that provides versatility and flexibility by allowing trainable node embedding and edge weights, helping the GCN model find the better solution. We applied the FastMPN model to the task of clinical information extraction from cancer pathology reports, extracting the following six properties: main site, subsite, laterality, histology, behavior, and grade. Results: We evaluated the clinical task performance of the FastMPN models in terms of micro- and macro-averaged F1 scores. A comparison was performed with the multi-task convolutional neural network (MT-CNN) model. Results show that the FastMPN model is equivalent to or better than the MT-CNN. Conclusions: Our implementation revealed that our FastMPN model, which is based on the PyTorch platform, can train a large corpus (667,290 training samples) with 202,373 unique words in less than 3 minutes per epoch using one NVIDIA V100 hardware accelerator. Our experiments demonstrated that using this implementation, the clinical task performance scores of information extraction related to tumors from cancer pathology reports were highly competitive. [ABSTRACT FROM AUTHOR]

Deep learning‐based models have become ubiquitous across a wide range of applications, including computer vision, natural language processing, and robotics. Despite their efficacy, one of the significant challenges associated with deep neural network (DNN) models is the potential risk of copyright leakage due to the inherent vulnerability of the entire model architecture and the communication burden of the large models during publishing. So far, it is still challenging for us to safeguard the intellectual property rights of these DNN models while reducing the communication time during model publishing. To this end, this paper introduces a novel approach using knowledge distillation techniques aimed at training a surrogate model to stand in for the original DNN model. To be specific, a knowledge distillation generative adversarial network (KDGAN) model is proposed to train a student model capable of achieving remarkable performance levels while simultaneously safeguarding the copyright integrity of the original large teacher model and improving communication efficiency during model publishing. Herein, comprehensive experiments are conducted to showcase the efficacy of model copyright protection, communication‐efficient model publishing, and the superiority of the proposed KDGAN model over other copyright protection mechanisms. [ABSTRACT FROM AUTHOR]

Movie reviews provide a medium of communication for the movie fans community. Movie reviews not only help viewers and potential viewers to obtain a general opinion about a movie but also allow the fans to construct an opinion of the movie. In this work, an analysis of over 60,000 movie reviews has been implemented to find meaningful text representation via text embedding. We improved the text embedding by proposing an attention-based Bidirectional Long-Short Term Memory (Bi-LSTM) network by using over 60,000 movie review text data as the training set and over 20,000 movie review text data as the testing set. Based on the data features, we performed a probabilistic analysis of the information related to words and phrases, combined the analysis results with text embedding, spatialized the text embedding, and compared the performance of the proposed attention-based spatialized word embedding Bi-LSTM model with several traditional machine learning models. The attention-based spatialized word embedding Bi-LSTM model proposed in this paper achieves an F1 score of 0.91 on the movie review sentiment classification dataset, with a prediction accuracy of 91%, outperforming the results of the current state-of-the-art research. The model can effectively identify the sentimental tendencies of movie reviews and use the analyzed sentimental tendencies to guide consumers in their consumption and obtain feedback on movie content. [ABSTRACT FROM AUTHOR]

Conducting a maturity assessment allows companies to measure their readiness in implementing novel technologies. However, this task is challenging due to the multidimensional, complex, unpredictable, and non-linear nature of innovation. In this paper, we introduce an innovative approach to maturity assessment that enables both intra- and inter-company analysis. Our approach evaluates a company's absolute maturity score concerning a specific technology or area. By leveraging a Natural Language Processing pipeline applied to a semi-structured questionnaire we extract popular concepts from the answers and present them to a human expert for analysis. The expert can refine the analysis by adding or removing concepts as needed. Subsequently, we compute a similarity metric for each answer to determine a company's maturity in specific concepts. The output of our analysis is presented through human-readable plots, offering clear insights into the internal maturity level of the company and allowing for a comparison with competitors across the chosen concepts. To demonstrate the capabilities of our method, we provide a running example showcasing both quantitative and qualitative results of the analysis. Our approach demonstrates efficiency, with preprocessing completed in 1.967 ± 0.758 s, and information extraction in 0.074 ± 0.017 s on average, excluding human intervention time, and requiring low hardware resources. • We propose a novel approach to assess the maturity level in Digital Era. • We propose an embedded representation to provide inter/intra visual representation. • We measure how mature is a company profile related to a set of topic/technologies. • We design an advanced NLP approach based on deep learning to analyze questionnaires. [ABSTRACT FROM AUTHOR]

Ensuring compliance with complex industry standards and regulations during the design and implementation phases of construction projects is a significant challenge in the building information modeling (BIM) domain. Traditional manual compliance checking methods are inefficient and error-prone, failing to meet modern engineering demands. Natural language processing (NLP) and deep learning methods have improved efficiency and accuracy in rule interpretation and compliance checking. However, these methods still require extensive manual feature engineering, large, annotated datasets, and significant computational resources. Large language models (LLMs) provide robust language understanding with minimal labeled data due to their pre-training and few-shot learning capabilities. However, their application in the AEC field is still limited by the need for fine-tuning for specific tasks, handling complex texts with nested clauses and conditional statements. This study introduces an innovative automated compliance checking framework that integrates LLM, deep learning models, and ontology knowledge models. The use of LLM is motivated by its few-shot learning capability, which significantly reduces the need for large, annotated datasets required by previous methods. Deep learning is employed to preliminarily classify regulatory texts, which further enhances the accuracy of structured information extraction by the LLM compared to directly feeding raw data into the LLM. This novel combination of deep learning and LLM significantly enhances the efficiency and accuracy of compliance checks by automating the processing of regulatory texts and reducing manual intervention. This approach is crucial for architects, engineers, project managers, and regulators, providing a scalable and adaptable solution for automated compliance in the construction industry with broad application prospects. [ABSTRACT FROM AUTHOR]

The growth of social networks has provided a platform for individuals with prejudiced views, allowing them to spread hate speech and target others based on their gender, ethnicity, religion, or sexual orientation. While positive interactions within diverse communities can considerably enhance confidence, it is critical to recognize that negative comments can hurt people's reputations and well-being. This emergence emphasizes the need for more diligent monitoring and robust policies on these platforms to protect individuals from such discriminatory and harmful behavior. Hate speech is often characterized as an intentional act of aggression directed at a specific group, typically meant to harm or marginalize them based on certain aspects of their identity. Most of the research related to hate speech has been conducted in resource-aware languages like English, Spanish, and French. However, low-resource European languages, such as Irish, Norwegian, Portuguese, Polish, Slovak, and many South Asian, present challenges due to limited linguistic resources, making information extraction labor-intensive. In this study, we present deep neural networks with FastText word embeddings using regularization methods for multi-class hate speech detection in the Norwegian language, along with the implementation of multilingual transformer-based models with hyperparameter tuning and generative configuration. FastText outperformed other deep learning models when stacked with Bidirectional LSTM and GRU, resulting in the FAST-RNN model. In the concluding phase, we compare our results with the state-of-the-art and perform interpretability modeling using Local Interpretable Model-Agnostic Explanations to achieve a more comprehensive understanding of the model's decision-making mechanisms. [ABSTRACT FROM AUTHOR]