Search

Event extraction and annotation has become a significant focus of recent efforts in biological text mining and information extraction (IE). However, event extraction, event annotation methods, and resources have so far focused almost exclusively on a single domain. State-of-the-art studies on biological event extraction and annotation are typically domain-dependent and domain-restricted. In this paper, we adopt an approach aimed at extracting events and relations for two different tasks by generating a common dataset using transfer learning and structural correspondence learning (SCL). A deep learning event extraction system was developed to evaluate our results. Our approach comprises two stages: (1) generating a dataset from two independent event extraction tasks or domains, and (2) using a classifier model to learn feature patterns from the generated dataset for event and relation extraction. The classifier in the proposed model can extract events and relations irrespective of the domain of the test input. Our study shows that this approach performs competitively compared to domain specific or dependent tasks., (Copyright © 2019 Elsevier Inc. All rights reserved.)

Due to popular successes (e.g., ChatGPT) Artificial Intelligence (AI) is on everyone's lips today. When advances in biotechnology are combined with advances in AI unprecedented new potential solutions become available. This can help with many global problems and contribute to important Sustainability Development Goals. Current examples include Food Security, Health and Well-being, Clean Water, Clean Energy, Responsible Consumption and Production, Climate Action, Life below Water, or protect, restore and promote sustainable use of terrestrial ecosystems, sustainably manage forests, combat desertification, and halt and reverse land degradation and halt biodiversity loss. AI is ubiquitous in the life sciences today. Topics include a wide range from machine learning and Big Data analytics, knowledge discovery and data mining, biomedical ontologies, knowledge-based reasoning, natural language processing, decision support and reasoning under uncertainty, temporal and spatial representation and inference, and methodological aspects of explainable AI (XAI) with applications of biotechnology. In this pre-Editorial paper, we provide an overview of open research issues and challenges for each of the topics addressed in this special issue. Potential authors can directly use this as a guideline for developing their paper. • This pre-Editorial provides an overview of some hot topics in AI for life. • The benefit for the reader is to get a short, crisp, concise intro into AI. • Authors interested in this Special Issue will get a scope of topics. • The take away message is that in the future no biotechnology can do without AI. [ABSTRACT FROM AUTHOR]

Movie reviews provide a medium of communication for the movie fans community. Movie reviews not only help viewers and potential viewers to obtain a general opinion about a movie but also allow the fans to construct an opinion of the movie. In this work, an analysis of over 60,000 movie reviews has been implemented to find meaningful text representation via text embedding. We improved the text embedding by proposing an attention-based Bidirectional Long-Short Term Memory (Bi-LSTM) network by using over 60,000 movie review text data as the training set and over 20,000 movie review text data as the testing set. Based on the data features, we performed a probabilistic analysis of the information related to words and phrases, combined the analysis results with text embedding, spatialized the text embedding, and compared the performance of the proposed attention-based spatialized word embedding Bi-LSTM model with several traditional machine learning models. The attention-based spatialized word embedding Bi-LSTM model proposed in this paper achieves an F1 score of 0.91 on the movie review sentiment classification dataset, with a prediction accuracy of 91%, outperforming the results of the current state-of-the-art research. The model can effectively identify the sentimental tendencies of movie reviews and use the analyzed sentimental tendencies to guide consumers in their consumption and obtain feedback on movie content. [ABSTRACT FROM AUTHOR]

Due to the rapid expansion of online platforms, review-based recommender systems have become the standard way to determine consumers' preferences for various goods. The purpose of this study is to address three significant issues with review-based approaches. First, these techniques suffer from a class-imbalance problem, where rating levels with lower proportions are partially neglected. As a result, their rating performance for such rare levels is unsatisfactory. To address this drawback, this paper proposes a flexible dual-optimizer network that enhances performance by utilizing classification and regression losses in the first attempt to solve this problem. Second, the typical review-based approach of preprocessing review texts with word embeddings will result in inadequate contextual information extraction. Thus, the bidirectional encoder representations from the transformers (BERT) method is introduced as the preprocessing technique to fully extract semantic information. Third, existing techniques make recommendations without considering that the user preferences may change over time. We suggest a time-varying feature extraction scheme that includes a multiscale convolutional neural network and bidirectional long short-term memory to solve this problem. We then develop an interactive component that summarizes the contextual information associated with the user-item pairings. The efficiency of the proposed time-varying attention with dual-optimizer (TADO) model is demonstrated through extensive tests on 12 benchmark datasets derived from Amazon Product Reviews. Regarding the existing state-of-the-art techniques, the proposed model attains a substantial performance increase of 22.69%, 8.75%, and 17.35%, respectively, over the Aspect-aware Latent Factor Model (ALFM), Multi-Pointer Co-Attention Networks for recommendation (MPCN), and Aspect-based Neural Recommender (ANR). Moreover, an ablation study demonstrates the importance of jointly employing the components developed for TADO. [ABSTRACT FROM AUTHOR]

Natural language processing (NLP) is rapidly becoming an important topic in the medical community. The ability to automatically analyze any type of medical document could be the key factor to fully exploit the data it contains. Cutting-edge artificial intelligence (AI) architectures, particularly machine learning and deep learning, have begun to be applied to this topic and have yielded promising results. We conducted a literature search for 1,024 papers that used NLP technology in neuroscience and psychiatry from 2010 to early 2022. After a selection process, 115 papers were evaluated. Each publication was classified into one of three categories: information extraction, classification, and data inference. Automated understanding of clinical reports in electronic health records has the potential to improve healthcare delivery. Overall, the performance of NLP applications is high, with an average F1-score and AUC above 85%. We also derived a composite measure in the form of Z-scores to better compare the performance of NLP models and their different classes as a whole. No statistical differences were found in the unbiased comparison. Strong asymmetry between English and non-English models, difficulty in obtaining high-quality annotated data, and train biases causing low generalizability are the main limitations. This review suggests that NLP could be an effective tool to help clinicians gain insights from medical reports, clinical research forms, and more, making NLP an effective tool to improve the quality of healthcare services. [ABSTRACT FROM AUTHOR]

Opinion summarization recapitulates the opinions about a common topic automatically. The primary motive of summarization is to preserve the properties of the text and is shortened in a way with no loss in the semantics of the text. The need of automatic summarization efficiently resulted in increased interest among communities of Natural Language Processing and Text Mining. This paper emphasis on building an extractive summarization system combining the features of principal component analysis for dimensionality reduction and bidirectional Recurrent Neural Networks and Long Short-Term Memory (RNN-LSTM) deep learning model for short and exact synopsis using seq2seq model. It presents a paradigm shift with regard to the way extractive summaries are generated. Novel algorithms for word extraction using assertions are proposed. The semantic framework is well-grounded in this research facilitating the correct decision making process after reviewing huge amount of online reviews, considering all its important features into account. The advantages of the proposed solution provides greater computational efficiency, better inferences from social media, data understanding, robustness and handling sparse data. Experiments on the different datasets also outperforms the previous researches and the accuracy is claimed to achieve more than the baselines, showing the efficiency and the novelty in the research paper. The comparisons are done by calculating accuracy with different baselines using Rouge tool. [ABSTRACT FROM AUTHOR]

Background: Analyzing the unstructured textual data contained in electronic health records (EHRs) has always been a challenging task. Word embedding methods have become an essential foundation for neural network-based approaches in natural language processing (NLP), to learn dense and low-dimensional word representations from large unlabeled corpora that capture the implicit semantics of words. Models like Word2Vec, GloVe or FastText have been broadly applied and reviewed in the bioinformatics and healthcare fields, most often to embed clinical notes or activity and diagnostic codes. Visualization of the learned embeddings has been used in a subset of these works, whether for exploratory or evaluation purposes. However, visualization practices tend to be heterogeneous, and lack overall guidelines.Objective: This scoping review aims to describe the methods and strategies used to visualize medical concepts represented using word embedding methods. We aim to understand the objectives of the visualizations and their limits.Methods: This scoping review summarizes different methods used to visualize word embeddings in healthcare. We followed the methodology proposed by Arksey and O'Malley (Int J Soc Res Methodol 8:19-32, 2005) and by Levac et al. (Implement Sci 5:69, 2010) to better analyze the data and provide a synthesis of the literature on the matter.Results: We first obtained 471 unique articles from a search conducted in PubMed, MedRxiv and arXiv databases. 30 of these were effectively reviewed, based on our inclusion and exclusion criteria. 23 articles were excluded in the full review stage, resulting in the analysis of 7 papers that fully correspond to our inclusion criteria. Included papers pursued a variety of objectives and used distinct methods to evaluate their embeddings and to visualize them. Visualization also served heterogeneous purposes, being alternatively used as a way to explore the embeddings, to evaluate them or to merely illustrate properties otherwise formally assessed.Conclusions: Visualization helps to explore embedding results (further dimensionality reduction, synthetic representation). However, it does not exhaust the information conveyed by the embeddings nor constitute a self-sustaining evaluation method of their pertinence. [ABSTRACT FROM AUTHOR]

Geological reports are frequently used by geologists involved in geological surveys and scientific research to record the results and outcomes of geological surveys. With such a rich data source, a substantial amount of knowledge has yet to be mined and analyzed. This paper focuses on automatically information extraction from geological reports, namely, geological named entity recognition. Geological named entity recognition has an important role in data mining, knowledge discovery and Knowledge graph construction. Existing general named entity recognition models/tools are limited in the domain of geoscience due to the various language irregularities associated with geological text, such as informal sentence structures, several domain‐geoscience words, large character lengths and multiple combinations of independent words. We present Bidirectional encoder representations from transformers (BERT)‐(Bidirectional gated recurrent unit network) BiGRU‐ (Conditional random field) CRF, which is a deep learning‐based geological named entity recognition model that is designed specifically with these linguistic irregularities in mind. Based on the pretrained language model, an integrated deep learning model incorporating BERT, BiGRU and CRF is constructed to obtain character vectors rich in semantic information through the BERT pretrained language model to alleviate for the lack of specificity of static word vectors (e.g., word2vec) and to improve the extraction capability of complex geological entities. We demonstrate our proposed model by applying it to four test datasets, including a geoscience NER data set from regional geological reports, and by comparing its performance with those of five baseline models. Plain Language Summary: Geological named entity recognition has an important role in information extraction and knowledge discovery. This paper presents BERT‐BiGRU‐CRF, which is a deep learning‐based geological named entity recognition model that is designed specifically with these linguistic irregularities in mind. We hope that our approach will serve as an alternative method that deserves further study. Key Points: A named entity recognition model based on BERT‐BiGRU‐CRF is proposed and a set of detailed experimental tests on domain data setOur model is compared in detail with four other mainstream models, and experiments demonstrate that our model obtains better performanceWe share the source code of BERT‐BiGRU‐CRF and the annotated test data [ABSTRACT FROM AUTHOR]

Archaeological site text is the main carrier of archaeological data at present, which contains rich information. How to efficiently extract useful knowledge from the massive unstructured archaeological site texts is of great significance for the mining and reuse of archaeological information. According to the site information (such as name, location, cultural type, dynasty, etc.) recorded in the Chinese archaeological site text, this paper combines deep learning and natural language processing techniques to study the information extraction method for automatically obtaining the spatio-temporal information of sites. The initial construction of the corpus of Chinese archaeological site text is completed for the first time, and the corpus is input into the Bidirectional Long Short-Term Memory with Conditional Random Fields (BiLSTM-CRF) entity recognition model and Bidirectional Gated Recurrent Units with Dual Attention (BiGRU-Dual Attention) relationship extraction model for training. The F1 values of BiLSTM-CRF model and BiGRU-Dual Attention model on the test set reach 87.87% and 88.05%, respectively. The study demonstrates that the information extraction method proposed in this paper is feasible for the Chinese archaeological site texts, which promotes the establishment of knowledge graphs in archaeology and provides new methods and ideas for the development of information mining technology in archaeology. [ABSTRACT FROM AUTHOR]

Sentiment Analysis and Opinion Mining is a computational text processing technique used for extraction of emotion from the given text, for e.g. joy, sad, disgust, fear, anger which can be polarized as positive, negative and neutral. The advancement in technology and data augmentation has been an easy access for data collection for the research work. In this research paper, different machine learning algorithms such as Extreme Gradient Boost classifier, Extra tree classifier, K-Nearest Neighbour classifier and Long Short Term Memory deep learning model was computed to classify the divergent datasets such as Twitter Bengaluru traffic data, movie review, fake news classification and financial sentiments. Performance of these classifier was tabulated with respect to independent datasets in terms of accuracy, f1-score, precision, sensitivity, mean squared error and, log loss and graphically represented with the help of PyCharm. The contribution in this paper is a methodology that automates the sentiment analysis of the divergent dataset with context of apprehension of tweets. [ABSTRACT FROM AUTHOR]

Abstract Background Analyzing the unstructured textual data contained in electronic health records (EHRs) has always been a challenging task. Word embedding methods have become an essential foundation for neural network-based approaches in natural language processing (NLP), to learn dense and low-dimensional word representations from large unlabeled corpora that capture the implicit semantics of words. Models like Word2Vec, GloVe or FastText have been broadly applied and reviewed in the bioinformatics and healthcare fields, most often to embed clinical notes or activity and diagnostic codes. Visualization of the learned embeddings has been used in a subset of these works, whether for exploratory or evaluation purposes. However, visualization practices tend to be heterogeneous, and lack overall guidelines. Objective This scoping review aims to describe the methods and strategies used to visualize medical concepts represented using word embedding methods. We aim to understand the objectives of the visualizations and their limits. Methods This scoping review summarizes different methods used to visualize word embeddings in healthcare. We followed the methodology proposed by Arksey and O’Malley (Int J Soc Res Methodol 8:19–32, 2005) and by Levac et al. (Implement Sci 5:69, 2010) to better analyze the data and provide a synthesis of the literature on the matter. Results We first obtained 471 unique articles from a search conducted in PubMed, MedRxiv and arXiv databases. 30 of these were effectively reviewed, based on our inclusion and exclusion criteria. 23 articles were excluded in the full review stage, resulting in the analysis of 7 papers that fully correspond to our inclusion criteria. Included papers pursued a variety of objectives and used distinct methods to evaluate their embeddings and to visualize them. Visualization also served heterogeneous purposes, being alternatively used as a way to explore the embeddings, to evaluate them or to merely illustrate properties otherwise formally assessed. Conclusions Visualization helps to explore embedding results (further dimensionality reduction, synthetic representation). However, it does not exhaust the information conveyed by the embeddings nor constitute a self-sustaining evaluation method of their pertinence.

Coreference Resolution is an essential task for Natural Language Processing (NLP) application, which has a paramount impact on the performance of text summarization, machine translation, text classification, and recognizing textual entailment. Mention Detection (MD) is the core component of the coreference resolution task and is additionally a process of extraction of all possible mentions from the text. Mention is referred to as a textual representation of entities in the text, such as Name, Nominal, and Pronominal mentions. The mentions appear in the text using different representations but indicating the same entity. The performance of an MD module positively affects the performance of NLP tasks such as Coreference resolution, Relation Extraction, Information retrieval, Information extraction, etc. Incorrect identification of mentions in the text severely affects the efficiency of the coreference resolution task. This paper aims to provide a comprehensive overview for the state of the art of mention detection approaches, which is utilized in the coreference resolution task and explains the importance of MD in Coreference resolution. The subsisting approaches are classified based on the underlying techniques adopted by each approach in three categories: Rule-based mention detection, Statistics-based mention detection, and Deep learning-based mention detection. The performance of deep learning is improving as more data and more powerful computing resources become available. This study endeavors to provide a comparative analysis of various mention detection approaches and help the researchers to assimilate knowledge about the mention detection approaches from sundry aspects. [ABSTRACT FROM AUTHOR]

The availability of legal judgment documents in digital form offers numerous opportunities for information extraction and application. Automatic summarization of these legal texts is a crucial and a challenging task due to the unusual structure and high complexity of these documents. Previous approaches in this direction have relied on huge labelled datasets, using hand engineered features, leveraging on domain knowledge and focussed their attention on a narrow sub-domain for increased effectiveness. In this paper, we propose simple generic techniques using neural network for the summarization task for Indian legal judgment documents. We explore two neural network architectures for this task utilizing the word and sentence embeddings for capturing the semantics. The main advantage of the proposed approaches is that they do not rely on hand crafted features, or domain specific knowledge, nor is their application restricted to a particular sub-domain thus making them suitable to be extended to other domains as well. We tackle the problem of unavailability of labelled data for the task by assigning classes/scores to sentences in the training set, based on their match with reference summary produced by humans. The experimental evaluations establish the effectiveness of our proposed approaches as compared with other baselines. [ABSTRACT FROM AUTHOR]

A variety of detailed data about geological topics and geoscience knowledge are buried in the geoscience literature and rarely used. Named entity recognition (NER) provides both opportunities and challenges to leverage this wealth of data in the geoscience literature for data analysis and further information extraction. Existing NER models and techniques are mainly based on rule‐based and supervised approaches, and developing such systems requires a costly manual effort. In this paper, we first design a generic stepwise framework for domain‐specific NER. Following this framework, domain‐specific entities and domain‐general words are collected and selected as seed terms. Normalization and grouping processes are then applied to these seed terms for further analysis. A random extraction algorithm based on a unigram language model is used to generate a large‐scale training data set consisting of probabilistically labeled pseudosentences. Each generated sentence is then used as input to the self‐training and learning algorithm. Experimental results on two constructed data sets demonstrate that the proposed model effectively recognizes and identifies geological named entities. Plain Language Summary: Existing entity recognition and classification methods are less functional for automatic geological name entity recognition. In this paper, we propose a stepwise unsupervised approach to geological name entity recognition in geological reports that requires no labeled data. This approach dynamically adapts to extract and identify unseen instances. We hope that our approach will serve as an alternative method that deserves further study. Key Points: Geological named entities are extracted from unstructured Chinese geoscience reports using deep learningThe proposed framework can be easily extended to other subject domains through fine‐tuningA training data set is constructed based on both domain‐specific entities and domain‐general words [ABSTRACT FROM AUTHOR]

The recent years have witnessed a rapid increase in the number of scientific articles in biomedical domain. These literature are mostly available and readily accessible in electronic format. The domain knowledge hidden in them is critical for biomedical research and applications, which makes biomedical literature mining (BLM) techniques highly demanding. Numerous efforts have been made on this topic from both biomedical informatics (BMI) and computer science (CS) communities. The BMI community focuses more on the concrete application problems and thus prefer more interpretable and descriptive methods, while the CS community chases more on superior performance and generalization ability, thus more sophisticated and universal models are developed. The goal of this paper is to provide a review of the recent advances in BLM from both communities and inspire new research directions. [ABSTRACT FROM AUTHOR]

Question answering over knowledge bases (KBQA) is a challenging task in natural language processing. It requires machines to answer natural language questions based on large-scale knowledge bases. Recent years have witnessed remarkable success of neural network models on many natural language processing tasks, including KBQA. In this paper, we first review the recent advances of deep learning methods on solving simple questions in two streams, the information extraction style and semantic parsing style. We then introduce how to extend the neural architectures to answer more complex questions with iteration and decomposition techniques, and summarize current research challenges. [ABSTRACT FROM AUTHOR]

Unstructured texts dominate data in construction projects. With the achievements of natural language processing (NLP) techniques, mining unstructured text data for smart construction has become increasingly significant. To understand state-of-the-art NLP for smart construction, uncover related issues, and propose potential improvements, this paper presents a comprehensive review of bottom-level techniques and mainstream applications of NLP in the industry. In total, 124 journal articles published in the last two decades are reviewed. NLP involves five core steps supported by various techniques, e.g., syntactic parsing, heuristic rules, machine learning, and deep learning. NLP has been applied for information extraction and exchanging and many downstream applications to facilitate management and decision-making. The role of NLP in smart construction and current challenges for fully reaping its benefits are discussed, and four research directions are identified, i.e., improving relation extraction, realising knowledge base auto-development, integrating multi-modal information, and achieving an accuracy-efficiency trade-off by developing an NLP application framework. It is envisioned that outcomes of this paper can assist both researchers and industrial practitioners with appreciating the research and practice frontier of NLP for smart construction and soliciting the latest NLP techniques. • An in-depth review of natural language processing (NLP) for smart construction is conducted. • 124 journal articles published in 2000–2021 are manually reviewed. • Bottom-level NLP techniques are thoroughly introduced. • NLP is used to extract and exchange information and to support downstream applications. • The role of NLP in smart construction, current challenges, and future directions are provided. [ABSTRACT FROM AUTHOR]

Educational Data Mining (EDM) is a research field that focuses on the application of data mining, machine learning, and statistical methods to detect patterns in large collections of educational data. Different machine learning techniques have been applied in this field over the years, but it has been recently that Deep Learning has gained increasing attention in the educational domain. Deep Learning is a machine learning method based on neural network architectures with multiple layers of processing units, which has been successfully applied to a broad set of problems in the areas of image recognition and natural language processing. This paper surveys the research carried out in Deep Learning techniques applied to EDM, from its origins to the present day. The main goals of this study are to identify the EDM tasks that have benefited from Deep Learning and those that are pending to be explored, to describe the main datasets used, to provide an overview of the key concepts, main architectures, and configurations of Deep Learning and its applications to EDM, and to discuss current state-of-the-art and future directions on this area of research. [ABSTRACT FROM AUTHOR]

Background: Biomedical semantic indexing is important for information retrieval and many other research fields in bioinformatics. It annotates biomedical citations with Medical Subject Headings. In face of unbalanced category distribution in the training data, sampling methods are difficult to apply for semantic indexing task. Results: In this paper, we present a novel deep serial multi-task learning model. The primary task treats the biomedical semantic indexing as a multi-label text classification issue that considers the relations of the labels. The auxiliary task is a regression task that predicts the MeSH number of the citation and provides hints for the network to make it converge faster. The experimental results on the BioASQ-Task5A open dataset show that our model outperforms the state-of-the-art solution "MTI", proposed by the US National Library of Medicine. Further, it not only achieves the highest precision among all the solutions in BioASQ-Task5A but also has faster convergence speed compared with some naive deep learning methods. Conclusions: Rather than parallel in an ordinary multi-task structure, the tasks in our model are serial and tightly coupled. It can achieve satisfied performance without any handcrafted feature. [ABSTRACT FROM AUTHOR]

[Display omitted] • We attach a top-down path to the feature Pyramid Network (FPN). • We designed a fast table parsing method (GTab). • A novel semantic similarity matching model is designed. • A translation-based graph structure-driven Q&A approach is proposed. The geological reports contain various tables, and can offer mineral element content data and stratum detailed information. Geological tabular information extraction and its semantic fusion with text is of great significance in converting and fusing geological unstructured data into structured knowledge to guide cognitive intelligence analysis in the geoscience domain. While the performance of general tools and existing table structure analysis methods is limited due to the various merged cells and diagonally split table headers. To address this issue, we propose a novel approach based on the improved Mask R-CNN model to identify and parse the forms. The geological table parsing network constructed in this paper consists of two key steps: (1) A cell feature augmentation (CFA) module to learn the contextual features for identifying cells of different sizes. (2) A table parsing method (GTab) to parse the table header cells with split lines. We compare the proposed method with commonly used table parsing methods on our constructed geological table dataset. Our models are easily integrated into a prototype system to provide joint information processing and analysis. [ABSTRACT FROM AUTHOR]

As the prevalence of social media on the Internet, opinion mining has become an essential approach to analyzing so many data. Various applications appear in a wide range of industrial domains. Meanwhile, opinions have diverse expressions which bring along research challenges. Both of the practical demands and research challenges make opinion mining an active research area in recent years. In this paper, we present a review of Natural Language Processing (NLP) techniques for opinion mining. First, we introduce general NLP techniques which are required for text preprocessing. Second, we investigate the approaches of opinion mining for different levels and situations. Then we introduce comparative opinion mining and deep learning approaches for opinion mining. Opinion summarization and advanced topics are introduced later. Finally, we discuss some challenges and open problems related to opinion mining. [ABSTRACT FROM AUTHOR]

In this paper, we propose an approach to understand situations in the real world with the sentiment analysis of Twitter data base on deep learning techniques. With the proposed method, it is possible to predict user satisfaction of a product, happiness with some particular environment or destroy situation after disasters. Recently, deep learning is able to solve problems in computer vision or voice recognition, and convolutional neural network (CNN) works good for image analysis and image classification. The biggest reason to adopt CNN in image analysis and classification is due to CNN can extract an area of features from global information, and it is able to consider the relationship among these features. The above solution can achieve a higher accuracy in analysis and classification. For natural language processing, texts data features also can be extracted piece by piece and to consider the relationship among these features, but without the consideration of context or whole sentence, the sentiment might be understood wrong. And currently, convolutional neural network is one of the most effective methods to do image classification, CNN has a convolutional layer to extract information by a larger piece of text, so we work for sentiment analysis with convolutional neural network, and we design a simple convolutional neural network model and test it on benchmark, the result shows that it achieves better accuracy performance in twitter sentiment classification than some of traditional method such as the SVM and Naive Bayes methods. [ABSTRACT FROM AUTHOR]

The general method of machine learning has encountered disadvantages in terms of the significant amount of time and effort required for feature extraction and engineering in natural language processing. However, in recent years, these disadvantages have been solved using deep learning. In this paper, we propose a mention pair (MP) model using deep learning, and a system that combines both rule-based and deep learning-based systems using a guided MP as a coreference resolution, which is an information extraction technique. Our experiment results confirm that the proposed deeplearning based coreference resolution system achieves a better level of performance than rule- and statistics-based systems applied separately. [ABSTRACT FROM AUTHOR]

[Display omitted] • A method to predict logical relationships between any pair of schedule activities with F1-Score of 88.3%. • A method to schedule a short list of input activities based on the learned logic. • Formalization of indirect logic dependencies between schedule activities. • A method for mining indirect dependencies from an input schedule. In construction, schedule mistakes causing delays beyond substantial completion dates cost contractors expensive liquidated damages. Hence, several industry guidelines, such as the DCMA's 14 point assessment, define schedule quality and offer systematic methods for ensuring it. These guidelines list "logic" as an essential control metric, and they require planners to ensure their schedules are free of missing or wrong logical dependencies. Checking the logic requires extensive construction domain knowledge, and planners perform it entirely manually as there are no available software solutions that support it. This paper offers a novel machine learning-based solution that learns construction scheduling domain knowledge from existing records completely automatically and applies it to validate the logic in input schedules achieving an F1 score of 88.3%. Furthermore, we tailor our method to use the learned knowledge to schedule a list of unordered activities. The details of the method, experimental results, benefits, and limitations are discussed. [ABSTRACT FROM AUTHOR]

Effective upkeep of aging infrastructure systems with limited funding and resources calls for efficient bridge management systems. Although data-driven models have been extensively studied in the last decade for extracting knowledge from past experience to guide future maintenance decision making, their performance and usefulness have been limited by the level of detail and accuracy of currently available bridge condition databases. This paper leverages an untapped resource for bridge condition data and proposes a new method to extract condition information from it at a high level of detail. To that end, a natural language processing approach was developed to formalize structural condition knowledge by formulating a sequence labeling task and modeling inspection narratives as a combination of words representing defects, their severity and location, while accounting for the context of each word. The proposed framework employs a deep-learning-based approach and incorporates context-aware components including a bi-directional Long Short Term Memory (LSTM) neural network architecture and a Conditional Random Field (CRF) classifier to account for the context of words when assigning labels. A dependency-based word embedding model was also used to represent the raw text while incorporating both semantic and contextual information. The sequence labeling model was trained using bridge inspection reports collected from the Virginia Department of Transportation bridge inspection database and achieved an F1 score of 94.12% during testing. The proposed model also demonstrated improvements compared with baseline sequence labeling models, and was further used to demonstrate the capability of detecting condition changes with respect to previous inspection records. Results of this study show that the proposed method can be used to extract and create a condition information database that can further assist in developing data-driven bridge management and condition forecasting models, as well as automated bridge inspection systems. [ABSTRACT FROM AUTHOR]

Construction projects face various constraints (e.g., materials and equipment). Constraint management approaches such as advanced working packaging (AWP) can remove constraints and ensure smooth work. However, due to inefficient information extraction, the prerequisite of AWP, i.e., identifying and modelling constraints, are performed manually. Efforts that integrate constraint information into project knowledge bases are also limited. This paper proposes a hybrid approach to automatically extract and integrate constraint information from texts. The approach combines a deep learning model with pre-defined rules. The model extracts constraint entities whereas rules created based on domain knowledge are used to establish relations between these entities. Extracted information is encoded into the original ontologies. The approach can extract both entities and relations with over 90% accuracy. The original ontologies can be successfully enriched and support semantic queries. The approach improves AWP by partially automating constraint identification and modelling as well as ontology development for information integration. • A deep-learning model is trained to extract constraints related entities from project documents • Domain knowledge is used to create rules to setup relations among extracted entities • Extracted entities and relations are encoded into domain ontologies to enrich its contents • The approach partially automates identification/modelling for advanced working packaging • The approach also partially automates ontology development [ABSTRACT FROM AUTHOR]

Named entity recognition (NER) is a basic but crucial task in the field of natural language processing (NLP) and big data analysis. The recognition of named entities based on Chinese is more complicated and difficult than English, which makes the task of NER in Chinese more challenging. In particular, fine-grained named entity recognition is more challenging than traditional named entity recognition tasks, mainly because fine-grained tasks have higher requirements for the ability of automatic feature extraction and information representation of deep neural models. In this paper, we propose an innovative neural network model named En2BiLSTM-CRF to improve the effect of fine-grained Chinese entity recognition tasks. This proposed model including the initial encoding layer, the enhanced encoding layer, and the decoding layer combines the advantages of pre-training model encoding, dual bidirectional long short-term memory (BiLSTM) networks, and a residual connection mechanism. Hence, it can encode information multiple times and extract contextual features hierarchically. We conducted sufficient experiments on two representative datasets using multiple important metrics and compared them with other advanced baselines. We present promising results showing that our proposed En2BiLSTM-CRF has better performance as well as better generalization ability in both fine-grained and coarse-grained Chinese entity recognition tasks. [ABSTRACT FROM AUTHOR]

Construction procedural constraints are critical in facilitating effective construction procedure checking in practice and for various inspection systems. Nowadays, the manual extraction of construction procedural constraints is costly and time-consuming. The automatic extraction of construction procedural constraint knowledge (e.g., knowledge entities and interlinks/relationships between them) from regulatory documents is a key challenge. Traditionally, natural language processing is implemented using either rule-based or machine learning approaches. Limited efforts on rule-based extraction of construction regulations often rely on pre-defined vocabularies and involve heavy feature engineering. Based on characteristics of the knowledge expression of construction procedural constraints in Chinese regulations, this paper explores a hybrid deep neural network, combining the bidirectional long short-term memory (Bi-LSTM) and the conditional random field (CRF), for the automatic extraction of the qualitative construction procedural constraints. Based on the proposed deep neural network, the recognition and extraction of named entities and relations between them are realized. Unlike existing information extraction research efforts using rule-based methods, the proposed hybrid deep learning approach can be applied without complex handcrafted features engineering. Besides, the long distance dependency relationships between different entities in regulations are considered. The model implementation results demonstrate the good performance of the end-to-end deep neural network in the extraction of construction procedural constraints. This study can be considered as one of the early explorations of knowledge extraction from construction regulations. [ABSTRACT FROM AUTHOR]

Despite the high effectiveness of the Deep Learning methods, they remain a "thing in themselves", a "black box" decisions of which can not be trusted. This is critical for such areas as medicine, financial investments, military applications and others, where the price of the error is too high. In this regard, the European Union is going to demand in 2018 from companies that they give users an explanation of the solutions obtained by automatic systems. In this paper, we offer an alternative, logical-probabilistic method of Deep Learning that can explain its decisions. This is a method of hierarchical clustering, based on the original logical-probabilistic generalization of formal concepts \cite{PFC}. For comparison with deep learning based on neural networks, the work \cite {UMLS} was chosen, in which the task of processing natural language on a set of data \textit{UMLS} is solved. To apply the logical-probabilistic generalization of formal concepts, a classification algorithm based on the energy of the contradictions Energy Learning \cite{DL_Energy} is defined. Logical-probabilistic formal concepts are defined through fixed points, as well as the formal concepts themselves, only certain rules of probability are used as rules. The energy of contradictions allows us to resolve the contradictions arising at fixed points that form probabilistic formal concepts. It is shown that this clustering algorithm is not inferior in accuracy to the method of Deep Learning \cite{UMLS}; nevertheless, the solutions obtained by it are explained by the set of probabilistic fixed-point rules.