Search

The attack technique by the malware distribution form is a dangerous, difficult to detect and prevent attack method. Current malware detection studies and proposals are often based on two main methods: using sign sets and analyzing abnormal behaviors using machine learning or deep learning techniques. This paper will propose a method to detect malware on Endpoints based on Event IDs using deep learning. Event IDs are behaviors of malware tracked and collected on Endpoints' operating system kernel. The malware detection proposal based on Event IDs is a new research approach that has not been studied and proposed much. To achieve this purpose, this paper proposes to combine different data mining methods and deep learning algorithms. The data mining process is presented in detail in section 2 of the paper.

Studies in the humanities are increasingly incorporating methods from information theory, which are based on dynamics in word usage and measures of relative entropy. This paper uses windowed relative entropy to thematic text representations to generate signals that capture information novelty and resonance and, aggregated on the actor-level, shed light on the innovators of the past. Comparing early modern chronicles and twentieth-century parliamentary debate we aim to investigate several dimensions of innovation in historical language records.

A construção de redes de conhecimento é um dos grandes desafios da pesquisa científica e inovação, no tocante ao uso e processamento da informação. Produzir conhecimento em redes de pesquisa e colaboração é importante para o entendimento e internacionalização das investigações. A utilização de métodos que possam revelar áreas de conhecimento implicitamente relacionadas é uma interessante alternativa. Entretanto, a realização deste tipo de construção requer a aplicação de métodos e técnicas específicos, os quais possam, a partir de amostras de dados, gerar informação que auxilie tal tarefa. Este artigo tem como objetivo apresentar a aplicação dos métodos de mineração de dados Doc2Vec e classificação de Reinert para a inferência de redes de conhecimento com base na similaridade e agrupamento de tópicos de documentos científicos textuais. O desenvolvimento do trabalho é baseado na metodologia KDT (do inglês, Knowledge Discovery from Texts). Diante disso, foram obtidas, utilizando técnicas de Web Scraping, dados de teses e dissertações disponibilizados na Biblioteca Digital Brasileira de Teses e Dissertações do IBICT, os quais foram tratados e submetidos à rotinas de processamento de mineração textual. São apresentados resultados quanto à classificação das amostras em grupos por similaridade semântica e grafos que representam a relação entre tais grupos.

Currently, as an effect of the COVID-19 pandemic, bioinformatics, genomics, and biological computations are gaining increased attention. Genomes of viruses can be represented by character strings based on their nucleobases. Document similarity metrics can be applied to these strings to measure their similarities. Clustering algorithms can be applied to the results of their document similarities to cluster them. P systems or membrane systems are computation models inspired by the flow of information in the membrane cells. These can be used for various purposes, one of them being data clustering. This paper studies a novel and versatile clustering method for genomes and the utilization of such membrane clustering models using document similarity metrics, which is not yet a well-studied use of membrane clustering models.

The file attached to this record is the author's final peer reviewed version. The Publisher's final version can be found by following the DOI link. E-mail and SMS are the most popular communication tools used by businesses, organizations and educational institutions. Every day, people receive hundreds of messages which could be either spam or ham. Spam is any form of unsolicited, unwanted digital communication, usually sent out in bulk. Spam emails and SMS waste resources by unnecessarily flooding network lines and consuming storage space. Therefore, it is important to develop high accuracy spam detection models to effectively block spam messages, so as to optimize resources and protect users. Various word-embedding techniques such as Bag of Words (BOW), N-grams, TF-IDF, Word2Vec and Doc2Vec have been widely applied to NLP problems, however a comparative study of these techniques for SMS spam detection is currently lacking. Hence, in this paper, we provide a comparative analysis of these popular word embedding techniques for SMS spam detection by evaluating their performance on a publicly available ham and spam dataset. We investigate the performance of the word embedding techniques using 5 different machine learning classifiers i.e. Multinomial Naive Bayes (MNB), KNN, SVM, Random Forest and Extra Trees. Based on the dataset employed in the study, N-gram, BOW and TF-IDF with oversampling recorded the highest F1 scores of 0.99 for ham and 0.94 for spam.

International audience; Nowadays, job recommender systems are more useful in the fight against unemployment due to their strong presence in e-recruitment platforms that are becoming very popular. Most of the recommender systems based on machine learning that recommend jobs to profiles use a vector representation of job offers based on keywords. However, these keywords are results of vectorization which is applied on a collection of documents where each one is a job offer. In this case, each keyword discriminates one job offer from another, whereas it can be preferable that each keyword discriminates one class from another. The aim of this paper is to improve the recommendation of job offers to users, by proposing to apply vectorization on a class-oriented collection of documents in order to obtain more useful keywords for the representation of job offers. In the case of job recommendation, each class-oriented document corresponds to a user profile. Experiments are done on two datasets (Nigam and Minajobs), using TF-IDF and Doc2Vec as vectorization techniques, Naive Bayes and Decision Trees as supervised machine learning models for top-N recommendation, and Precision, MRR and MAP as evaluation metrics. Our results show that, whatever the case, the best performance is always reach by a job recommender system resulting from our contribution. Compared to classic job recommender systems, the improvement rates can go up to 13% and 24% for systems based on Naive Bayes and go up to 55% and 46% for those based on Decision tree, respectively in the Nigam and Minajobs datasets.; Aujourd'hui, les systèmes de recommandation d'emploi sont plus utiles dans la lutte contre le chômage en raison de leur forte présence dans les plateformes de recrutement en ligne qui deviennent très populaires. La plupart des systèmes de recommandation basés sur l'apprentissage automatique qui recommandent des emplois à des profils utilisent une représentation vectorielle des offres d'emploi basée sur des mots-clés. Cependant, ces mots-clés sont les résultats d'une vectorisation appliquée à une collection de documents dont chacun est une offre d'emploi. Dans ce cas, chaque mot-clé discrimine une offre d'emploi par rapport à une autre, alors qu'il peut être préférable que chaque mot-clé discrimine une classe par rapport à une autre. L'objectif de cet article est d'améliorer la recommandation d'offres d'emploi aux utilisateurs, en proposant d'appliquer la vectorisation sur une collection de documents orientée classe afin d'obtenir des mots-clés plus utiles pour la représentation des offres d'emploi. Dans le cas de la recommandation d'emploi, chaque document orienté classe correspond à un profil d'utilisateur. Les expériences sont réalisées sur deux jeux de données (Nigam et Minajobs), en utilisant TF-IDF et Doc2Vec comme techniques de vectorisation, Naive Bayes et Decision Trees comme modèles d'apprentissage automatique supervisés pour la recommandation top-N, et Precision, MRR et MAP comme métriques d'évaluation. Nos résultats montrent que, dans tous les cas, la meilleure performance est toujours atteinte par un système de recommandation d'emploi issu de notre contribution. Par rapport aux systèmes classiques de recommandation d'emploi, les taux d'amélioration peuvent aller jusqu'à 13% et 24% pour les systèmes basés sur Naive Bayes et jusqu'à 55% et 46% pour ceux basés sur Decision tree, respectivement dans les jeux de données Nigam et Minajobs.

The special nature, volume and broadness of biomedical literature pose barriers for automated classification methods. On the other hand, manually indexing is time-consuming, costly and error prone. We argue that current word embedding algorithms can be efficiently used to support the task of biomedical text classification even in a multilabel setting, with many distinct labels. The ontology representation of Medical Subject Headings provides machine-readable labels and specifies the dimensionality of the problem space. Both deep- and shallow network approaches are implemented. Predictions are determined by the similarity between extracted features from contextualized representations of abstracts and headings. The addition of a separate classifier for transfer learning is also proposed and evaluated. Large datasets of biomedical citations are harvested for their metadata and used for training and testing. These automated approaches are still far from entirely substituting human experts, yet they can be useful as a mechanism for validation and recommendation. Dataset balancing, distributed processing and training parallelization in GPUs, all play an important part regarding the effectiveness and performance of proposed methods.

Outliers that deviate from a normal distribution are typically removed during the analysis process. However, the patterns of outliers are recognized as important information in the outlier detection method. This study proposes a technology trend screening framework based on a machine learning algorithm using outliers. The proposed method is as follows: first, we split the dataset by time into training and testing sets for training the Doc2Vec model. Next, we pre-process the patent documents using the trained model. The final outlier documents are selected from the preprocessed document data, through voting for the outlier documents extracted using the IQR, the three-sigma rule, and the Isolation Forest algorithm. Finally, the technical topics of the outlier documents extracted through the topic model are identified. This study analyzes the patent data on drones to describe the proposed method. Results show that, despite cumulative research on drone-related hardware and system technology, there is a general lack of research regarding the autonomous flight field.

The Marine Corps’ 38th Commandant calls for a force that can “affect the behavior of real-world pacing threats” through its “innovation, ingenuity, and willingness to continually adapt to and initiate changes in the operating environment.” Having a healthy and inclusive environment is vital to the Marine Corps’ success in developing and maintaining a force that can rise to the higher standard. Currently, the closest snapshot of the health of Marine Corps culture is collected via command climate surveys, which contain quantitative and free-response questions. Senior levels of Marine Corps leadership do not gain value from the free-text data because 1) analysis on free-text comments is time consuming and no analysis is done on those comments for the commander, and 2) there is no trend analysis on the aggregate of surveys over time and throughout commands, so they are not commander specific. This thesis developed a tool to gain value from free-text comments; it applied natural language processing (NLP) techniques, specifically topic extraction and sentiment analysis, on free-text comments from command climate surveys to provide a quantitative analysis on the health of culture in Marine Corps units. This approach can also be replicated to other services and can inspire efforts that will positively impact talent management and retention. Captain, United States Marine Corps Approved for public release. Distribution is unlimited.

The targeted attack is one of the social engineering attacks. The detection of this type of attack is considered a challenge as it depends on semantic extraction of the intent of the attacker. However, previous research has primarily relies on the Natural Language Processing or Word Embedding techniques that lack the context of the attacker's text message. Based on Sentence Embedding and machine learning approaches, this paper introduces a model for semantic detection of targeted attacks. This model has the advantage of encoding relevant information, which helps to improve the performance of the multi-class classification process. Messages will be categorized based on the type of security rule that the attacker has violated. The suggested model was tested using a dialogue dataset taken from phone calls, which was manually categorized into four categories. The text is pre-processed using natural language processing techniques, and the semantic features are extracted as Sentence Embedding vectors that are augmented with security policy sentences. Machine Learning algorithms are applied to classify text messages. The experimental results show that sentence embeddings with doc2vec achieved high prediction accuracy 96.8%. So, it outperformed the method applied to the same dialog dataset.

For effective learning, students must set learning objectives and adopt the ad hoc cognitive behavior to achieve them. Our research work aims to ensure good scaffolding by offering tutors the opportunity to observe learners’ cognitive behaviors, especially their cognitive engagement. In this respect, we propose in the present work an automatic system for classifying learners according to their levels of cognitive engagement. To this end, we focus on the analysis of social interactions within online discussion forums. Hence, the proposed system has two main steps: 1/Learners’ vector construction and 2/SVM-based classifier. The results show the efficiency of the proposed system with an accuracy = 0.9 and a cohen’s K = 0.89.

Appropriate indexing of resources is necessary for their efficient search, discovery and utilization. Relying solely on manual effort is time-consuming, costly and error prone. On the other hand, the special nature, volume and broadness of biomedical literature pose barriers for automated methods. We argue that current word embedding algorithms can be efficiently used to support the task of biomedical text classification. Both deep- and shallow network approaches are implemented and evaluated. Large datasets of biomedical citations and full texts are harvested for their metadata and used for training and testing. The ontology representation of Medical Subject Headings provides machine-readable labels and specifies the dimensionality of the problem space. These automated approaches are still far from entirely substituting human experts, yet they can be useful as a mechanism for validation and recommendation. Dataset balancing, distributed processing and training parallelization in GPUs, all play an important part regarding the effectiveness and performance of proposed methods.

Aquest projecte planteja la creació d'eines per a poder identificar pàgines web a partir de tràfic contingut a una captura en format pcap. Per poder-ho fer, s'ha seguit la hipòtesi que el conjunt d'adreces IP de les quals es reben o s'envien paquets identifica una pàgina web en concret. Així doncs, s'han desenvolupat les tres eines necessàries per a l'obtenció de traces, creació del model d'aprenentatge automàtic i cerca de la pàgina web. This project contemplates the creation of tools with the intention of identifying webpages from traffic contained in a capture with pcap format. In order to do so, the hypothesis of the set of IP addresses to which packets are transmitted or received identifying a specific webpage has been followed. Therefore, the three needed tools to get the traces, create the machine learning model and search the website have been developed.

In the domain of law and legal systems, jurisprudence principles (JPs) are considered major sources of legislative reasoning by jurisprudence scholars. Generally accepted JPs are often used to support the reasoning for a given jurisprudence case (JC). Although eliciting the JPs associated with a specific JC is a central task of legislative reasoning, it is complex and requires expertise, knowledge of the domain, and significant and lengthy human exertion by jurisprudence scholars. This study aimed to leverage advances in language modeling to support the task of JP elicitation. We investigated neural embeddings—specifically, doc2vec architectures—as a representation model for the task of JP elicitation using Arabic legal texts. Four experiments were conducted to evaluate three different architectures for document embedding models for the JP elicitation task. In addition, we explored an approach that integrates task-oriented word embeddings (ToWE) with document embeddings (paragraph vectors). The results of the experiments showed that using neural embeddings for the JP elicitation task is a promising approach. The paragraph vector distributed bag-of-words (PV-DBOW) architecture produced the best results for this task. To evaluate how well the ToWE model performed for the JP elicitation task, a graded relevance ranking measure, discounted cumulative gain (DCG), was used. The model achieved good results with a normalized DCG of 0.9 for the majority of the JPs. The findings of this study have significant implications for the understanding of how Arabic legal texts can be modeled and how the semantics of jurisprudence principles can be elicited using neural embeddings.

In today’s modern digital world more and more emails and messengers must be sent, processed and handled. The categorizing and classification of these text pieces can take an incredibly long time and will cost the company a lot of time and money. If the classification could be done automatically by a computer dependent on the content of the text/message it would result in a major yield for the Easit AB and its customers. In order to facilitate the task of text-classification Easit needs a solution that is made out of one language model and one classifier model. The language model will convert raw text to a vector that is representative of the text and the classifier will construe what predefined labels fit for the vector. The end goal is not to create the best solution. It is simply to create a general understanding about different language and classifier models and how to build a system that will be both fast and accurate. BERT were the primary language model during evaluation but doc2Vec and One-Hot encoding was also tested. The classifier consisted out of boundary condition models or dense neural networks that were all trained without knowledge about what language model that the text vectors came from. The validation accuracy which was presented for the IMDB-comment dataset with BERT resulted between 75% to 94%, mostly dependent on the language model and not on the classifier. The knowledge from the work resulted in a recommendation to Easit for an alternativebased system solution. I dagens moderna digitala värld är det allt mer majl-ärenden och meddelanden som ska skickas och processeras. Kategorisering och klassificering av dessa kan ta otroligt lång tid och kostar företag tid samt pengar. Om klassifieringen kunde ske automatiskt beroende på text-innehållet skulle det innebära en stor vinst för Easit AB och deras kunder. För att underlätta arbetet med text-klassifiering behöver Easit en tvådelad lösning som består utav en språkmodell och en klassifierare. Språkmodellen som omvandlar text till en vektor som representerar texten och klassifieraren tolkar vilka fördefinerade ettiketter/märken som passar för vektorn. Målet är inte att skapa den bästa lösningen utan det är att skapa en generell kunskap för hur man kan utforma ett system som kan klassifiera texten på ett träffsäkert och effektivt sätt. Vid utvärdering av olika språkmodeller användes framförallt BERT-modeller men även doc2Vec och One-Hot testas också. Klassifieraren bestod utav gränsvillkors-modeller eller dense neurala nätverk som tränades helt utan vetskap om vilken språkmodell som skickat text-vektorerna. Träffsäkerheten som uppvisades vid validering för IMDB-kommentars datasetet med BERT blev mellan 75% till 94%, primärt beroende på språkmodellen. De neuralt nätverk passar bäst som klassifierare mest på grund av deras skalbarhet med flera ettiketter. Kunskapen från arbetet resulterade i en rekommendation till Easit om en alternativbaserad systemlösning.

Aquest projecte compara diferents mètriques de distància aplicades a embeddings de documents sobre diferents gràfics conceptuals. Al saber quines mètriques són efectives d'entre totes les analitzades sota uns límits establerts, facilita l'aplicació d'aquestes a altres àmbits. Per tant, coneixent els resultats del projecte tindrem la certesa que les mètriques funcionaran correctament, estalviant temps i optimitzant els recursos de qualsevol negoci quan es necessita una comparació de diferents documents per a una aplicació més professional. Aquest projecte aplica dos models NLP: Doc2Vec i Word2Vec. L'objectiu d'aquesta investigació és profunditzar en el funcionament de les mètriques estudiades sobre aquests models. S'han creat diverses pipelines per automatitzar tot el processament de text, l'entrenament de models i l'exportació de resultats. Això ha permès poder crear tres contexts per model, fent que els resultats siguin més diversos i tindre més marge per comparar els resultats. Entre tota l'experimentació realitzada, cal destacar l'estructura de proves combinatòries que ens ha permès crear resultats d'experimentació prou grans per a avaluar discriminant per Model-Context-Mètrica. Finalment, les conclusions extretes han sigut bastant clares, la mètrica amb millor comportament general ha sigut Cosine Similarity, on per grups discriminats (Blogs, Científic, Wikipedia, Doc2Vec i Word2Vec) ha tingut el millor rendiment exceptuant Wikipedia, que ha estat en segon lloc. This project compares different distance metrics applied to document embeddings above different conceptual graphics. By knowing which metrics are effective among all the ones analysed under established limits, it facilitates the application of the aforementioned in other scopes. Therefore we have the certainty that the metrics will work properly, saving time and optimising the resources of any business when a comparison of different documents is needed for a more professional application. This project applies two NLP models: Doc2Vec and Word2Vec. The goal of this research is to delve into how the studied metrics work on these models: different pipelines have been created in order to have all the text processes automated, the fitting of the model process and the result exportation. This has allowed us to create three contexts per model, making the results more diverse and having a bigger margin to compare the results. Among all the experimentation done, the combinatory test structure should be highlighted as it has allowed us to create experimentation results large enough to evaluate the discrimination by Model-Context-Metric. Finally, the conclusions extracted are clear: the metric with best general behaviour has been Cosine Similarity, where by discriminated groups (Blogs, Scientific, Wikipedia, Doc2Vec and Word2Vec) has had the best performance excepting the Wikipedia context, that has resulted in a second place.

Выпускная квалификационная работа магистра посвящена исследованию в области определения вредоносного содержания в Ñ‚ÐµÐºÑÑ‚Ð¾Ð²Ñ‹Ñ Ð´Ð°Ð½Ð½Ñ‹Ñ . Исследованы Ñ Ð°Ñ€Ð°ÐºÑ‚ÐµÑ€Ð¸ÑÑ‚Ð¸ÐºÐ¸, изучены существующие Ñ‚ÐµÑ Ð½Ð¾Ð»Ð¾Ð³Ð¸Ð¸ для определения вредоносности в Ñ‚ÐµÐºÑÑ‚Ð¾Ð²Ñ‹Ñ Ð´Ð°Ð½Ð½Ñ‹Ñ , проведен Ð¸Ñ ÑÑ€Ð°Ð²Ð½Ð¸Ñ‚ÐµÐ»ÑŒÐ½Ñ‹Ð¹ анализ, проведен обзор методов вложений слов, выявлены недостатки и Ð½ÐµÐ¾Ð±Ñ Ð¾Ð´Ð¸Ð¼Ð¾ÑÑ‚ÑŒ новый Ð¿Ð¾Ð´Ñ Ð¾Ð´ с использованием алгоритма Doc2Vec. Представлены алгоритмы классификаторов на основе алгоритмов машинного обучения.В Ñ€Ð°Ð¼ÐºÐ°Ñ Ñ€Ð°Ð±Ð¾Ñ‚Ñ‹ предложен Ð¿Ð¾Ð´Ñ Ð¾Ð´, который позволит определять вредонсное содержание в Ñ‚ÐµÐºÑÑ‚Ð¾Ð²Ñ‹Ñ Ð´Ð°Ð½Ð½Ñ‹Ñ . В основе Ð¿Ð¾Ð´Ñ Ð¾Ð´Ð° будет лежать алгоритм Doc2Vec. Рассмотрены методы предварительного обработанного набора Ñ‚ÐµÐºÑÑ‚Ð¾Ð²Ñ‹Ñ Ð´Ð°Ð½Ð½Ñ‹Ñ . Описано использование алгоритма Doc2Vec для обучения модели.Предложенный Ð¿Ð¾Ð´Ñ Ð¾Ð´ был реализован в программной системе определения вредоносного содержания Ð²Â Ñ‚ÐµÐºÑÑ‚Ð¾Ð²Ñ‹Ñ Ð´Ð°Ð½Ð½Ñ‹Ñ . По результатам реализации предложенный Ð¿Ð¾Ð´Ñ Ð¾Ð´, точнее, чем существующие использованные другими алгоритмами вложеня слов (2-27% accuracy)., The master's thesis focuses on research into detecting fraudulent content in text data. Characteristics are examined, existing technologies for determining the harmfulness of text data are examinated and compared, a review of word embedding methods is conducted, and flaws and the need for a new approach based on the Doc2Vec algorithm are identified. Classifier algorithms based on machine learnign algorithms are presented.An approach is proposed as part of the work that will allow you to detect fraudulent content in text data. The Doc2Vec algorithm will be used to guide the process. The methods for pre-processed text data are discussed. The Doc2Vec algorithm is described in detail for model training.The proposed method was used to create a software system that could detect fraudulent context in text data. According to the results of the implementation, the proposed approach is more accurate than the existing ones used by other word embedding algorithms (2-27% accuracy).

In this thesis, we investigate the use of self-training, a semi-supervised learning method, to improve binary classification of text documents. This means making use of unlabeled samples, since labeled samples can be expensive to generate. More specifically, we want to classify emails that are retrieved by Skandinaviska Enskilda Banken (SEB). The method is tested on two datasets: the first is IMDB reviews, consisting of both labeled (good or bad) and unlabeled movie reviews; the second is provided by SEB and consists of labeled and unlabeled emails. First, supervised learning was investigated. Three different vectorization methods including two bag-of-words models and one doc2vec model were included. These were tested using five different machine learning classification methods. The comparison of the F1-score showed that doc2vec vectorization and the logistic regression classification method performed well and was used in the self-training investigation. We find that self-training on the IMDB dataset only yielded improvement for low number of labeled samples. For the SEB dataset we find that by using self-training, we can achieve the same F1-score using only around 1000 labeled samples (less than 10% of the labeled dataset), as using supervised methods on the full labeled set. We conclude that self-training can improve classification performance and also be used indirectly to reduce manual labeling efforts. I denna rapport undersöker vi användningen av self-training, en metod för semi-vägledd inlärning, för att förbättra binär klassificering av textdokument. Mer specifikt vill vi klassificera e-postbrev som hämtats av Skandinaviska Enskilda Banken (SEB). Målet är att kunna nyttja stora mängder omärkt data, eftersom att märka datapunkter kan vara dyrt. Metoden prövas på två datamängder: den första är IMDB-recensioner, bestående av både märkta (bra eller dåliga) och omärkta filmrecensioner; den andra datamängden tillhandahålls av SEB och består också av märkta och omärkta e-postbrev (relevanta och irrelevanta). Först prövas supervised learning. Tre olika vektoriseringsmetoder inklusive två bag-of-words-modeller och en doc2vec-modell inkluderas. Dessa testades med hjälp av fem olika klassificeringsmetoder för maskininlärning. Jämförelsen av F1-metriken visade att doc2vec-vektorisering och den logistisk regression fungerade bra och användes för semi-vägledd inlärning. Vi finner att self-training på IMDB-datamängden endast gav förbättring för ett lågt antal märkta datapunkter. För datamängden från SEB finner vi att genom att använda self-training kan vi uppnå samma F1-metrikvärde med endast 1000 märkta datapunkter (mindre än 10% av den märkta datamängden), som att använda metoder för vägledd inlärning på den fullständiga märkta datamängden. Vi drar slutsatsen att self-training förbättrar klassificering och kan användas för att minska mängden data som behöver märkas manuellt.

In this paper we perform a comparative analysis of three models for a feature representation of text documents in the context of document classification. In particular, we consider the most often used family of bag-of-words models, the recently proposed continuous space models word2vec and doc2vec, and the model based on the representation of text documents as language networks. While the bag-of-word models have been extensively used for the document classification task, the performance of the other two models for the same task have not been well understood. This is especially true for the network-based models that have been rarely considered for the representation of text documents for classification. In this study, we measure the performance of the document classifiers trained using the method of random forests for features generated with the three models and their variants. Multi-objective rankings are proposed as the framework for multi-criteria comparative analysis of the results. Finally, the results of the empirical comparison show that the commonly used bag-of-words model has a performance comparable to the one obtained by the emerging continuous-space model of doc2vec. In particular, the low-dimensional variants of doc2vec generating up to 75 features are among the top-performing document representation models. The results finally point out that doc2vec shows a superior performance in the tasks of classifying large documents.

Large pretrained language models like BERT have shown excellent generalization properties and have advanced the state of the art on various NLP tasks. In this paper we evaluate Finnish BERT (FinBERT) model on the IPTC Subject Codes prediction task. We compare it to a simpler Doc2Vec model used as a baseline. Due to hierarchical nature of IPTC Subject Codes, we also evaluate the effect of encoding the hierarchy in the network layer topology. Contrary to our expectations, a simpler baseline Doc2Vec model clearly outperforms the more complex FinBERT model and our attempts to encode hierarchy in a prediction network do not yield systematic improvement.

The abundance and real-time availability of Twitter data have proved beneficial in detecting events in various domains such as emergency situations, crime detection, public health, place recommendations, etc. Nevertheless, two critical challenges occur while detecting events using social media data. First, the uncertainty in capturing the contextual relationship among tweets, which is the result of the limited availability of the contextual information due to the small length of tweets. Second, the high computation cost required in event detection due to massive data processing. Earlier research works, addressing these challenges, have tried to capture the contextual information by using the dense vector representations of texts leveraging deep neural word embedding generation models such as Word2Vec and GloVe. However, these models are trained on the Euclidean vector space which fails to amalgamate the directional information of the vectors with the semantic information in text, incurring high computational costs. To target both the problems simultaneously, we propose modeling Twitter data as a graph-of-sentences which retains the contextual relationships while maintaining lower computational cost. The proposed model captures contextual information using JoSE, a spherical vector representation leveraging the word-word and word-paragraph semantic co-occurrence statistics in a spherical generative model. Furthermore, the framework uses the weighted-graph model to capture all the relationships among the Twitter data efficiently. The graph is further pruned with the help of the graph component filtering approach. The graph clustering model, employed to detect the events, leverages the edge weights and the partial-k clustering approach maintaining low computation costs. The experimentation on the annotated benchmark Twitter data set and the real-world datasets show improved run-time performance up to 30% while maintaining the qualitative performance (F1-score) comparable to the state-of-the-art models.

Document-level sentiment analysis is a challenging task given the large size of the text, which leads to an abundance of words and opinions, at times contradictory, in the same document. This analysis is particularly useful in analyzing press articles and blog posts about a particular product or company, and it requires a high concentration, especially when the topic being discussed is sensitive. Nevertheless, most existing models and techniques are designed to process short text from social networks and collaborative platforms. In this paper, we propose a combination of Convolutional Neural Networks (CNN) and Bidirectional Long Short-Term Memory (BiLSTM) models, with Doc2vec embedding, suitable for opinion analysis in long texts. The CNN-BiLSTM model is compared with CNN, LSTM, BiLSTM and CNN-LSTM models with Word2vec/Doc2vec embeddings. The Doc2vec with CNN-BiLSTM model was applied on French newspapers articles and outperformed the other models with 90.66% accuracy.

Due to the lack of face to face interaction in online learning environment, this article aims essentially to give tutors the opportunity to understand and analyze learners’ cognitive behavior. In this perspective, we propose an automatic system to assess learners’ cognitive presence regarding their social interactions within synchronous online discussions. Combining Natural Language Preprocessing, Doc2Vec document embedding method and machine learning techniques; we first make some transformations and preprocessing to the given transcripts, then we apply Doc2Vec method to represent each message as a vector that will be concatenated with LIWC and context features. The vectors are input data of Naïve Bayes algorithm; a machine learning method; that aims to classify transcripts according to cognitive presence categories.

With the exponential increase in information, it has become imperative to design mechanisms that allow users to access what matters to them as quickly as possible. The recommendation system (RS) with information technology development is the solution, it is an intelligent system. Various types of data can be collected on items of interest to users and presented as recommendations. RS also play a very important role in e-commerce. The purpose of recommending a product is to designate the most appropriate designation for a specific product. The major challenge when recommending products is insufficient information about the products and the categories to which they belong. In this paper, we transform the product data using two methods of document representation: bag-of-words (BOW) and the neural network-based document combination known as vector-based (Doc2Vec). We propose three-criteria recommendation systems (product, package and health) for each document representation method to foster online grocery shopping, which depends on product characteristics such as composition, packaging, nutrition table, allergen, and so forth. For our evaluation, we conducted a user and expert survey. Finally, we compared the performance of these three criteria for each document representation method, discovering that the neural network-based (Doc2Vec) performs better and completely alters the results. Ministerio de Economía, Industria y Competitividad | Ref. TEC2017-84197-C4-2-R

Delineating urban functional use plays a key role in understanding urban dynamics, evaluating planning strategies and supporting policymaking. In recent years, Points of Interest (POI) data, with precise geolocation and detailed attributes, have become the primary data source for exploring urban functional use from a bottom-up perspective, using local, highly disaggregated, big datasets. Previous studies using POI data have given insufficient consideration to the relationship among POI classes in the spatial context, and have failed to provide a straightforward means by which to classify urban functional areas. This study proposes an approach for delineating urban functional use at the scale of the Lower Layer Super Output Area (LSOA) in Greater London by integrating the Doc2Vec model, a neural network embedding method commonly used in natural language processing for vectoring words and documents from their context. In this study, the neural network vectorises both POI classes (‘Word’) and urban areas (‘Document’) based on their functional context by learning features from the spatial distribution of POIs in the city. Specifically, we first construct POI sequences based on the distribution of POI classes, and add their LSOA IDs as ‘document’ tags. By utilising these constructed POI–LSOA sequences, the Doc2Vec model trains the vectors of 574 POI classes (word vectors) and 4836 LSOAs (document vectors). The vectors of POI classes are then used in calculating the functional similarity scores based on their cosine distance, with the vectors of LSOAs grouped into clusters (i.e., functional areas) via the k-means clustering algorithm. We also identify latent functions in each cluster of LSOAs by performing topic modelling and enrichment factor. Compared with TF–IDF, LDA and Word2Vec models, the Doc2Vec model obtains the highest accuracy when classifying functional areas. This study proposes a straightforward approach in which the model directly trains vectors for urban areas, subsequently using them to classify urban functional areas. By employing the enhanced neural network model with low-cost and ubiquitous POI datasets, this study provides a potential tool with which to monitor urban dynamics in a timely and adaptive manner, thereby providing enhanced, data-driven support to urban planning, development and management.

Natural language inference (NLI) is an essential subtask in many natural language processing applications. It is a directional relationship from premise to hypothesis. A pair of texts is defined as entailed if a text infers its meaning from the other text. The NLI is also known as textual entailment recognition, and it recognizes entailed and contradictory sentences in various NLP systems like Question Answering, Summarization and Information retrieval systems. This paper describes the NLI problem attempted for a low resource Indian language Malayalam, the regional language of Kerala. More than 30 million people speak this language. The paper is about the Malayalam NLI dataset, named MaNLI dataset, and its application of NLI in Malayalam language using different models, namely Doc2Vec (paragraph vector), fastText, BERT (Bidirectional Encoder Representation from Transformers), and LASER (Language Agnostic Sentence Representation). Our work attempts NLI in two ways, as binary classification and as multiclass classification. For both the classifications, LASER outperformed the other techniques. For multiclass classification, NLI using LASER based sentence embedding technique outperformed the other techniques by a significant margin of 12% accuracy. There was also an accuracy improvement of 9% for LASER based NLI system for binary classification over the other techniques.

1st International Conference on Artificial Intelligence and Data Analytics, CAIDA 2021 -- 6 April 2021 through 7 April 2021 -- The rapid increment in internet usage has also resulted in bulk gerenation of text data. Therefore, investigation of new techniques for automatic classification of textual content is needed as manually managing unstructured text is challenging. The main objective of text classification is to train a model such that it should place an unseen text into correct category. In this study, text classification was performed using the Doc2vec word embedding method on the Turkish Text Classification 3600 (TTC-3600) dataset consisting of Turkish news texts and the BBC-News dataset consisting of English news texts. As the classification method, deep learning-based CNN and traditional machine learning classification methods Gauss Naive Bayes (GNB), Random Forest (RF), Naive Bayes (NB) and Support Vector Machine (SVM) are used. In the proposed model, the highest result was obtained as 94.17% in the Turkish dataset and 96.41% in the English dataset in the classification made with CNN. © 2021 IEEE.

Automated terminology extraction is a crucial task in natural language processing and ontology construction. Termhood can be inferred using linguistic and statistic techniques. This thesis focuses on the statistic methods. Inspired by feature selection techniques in documents classification, we experiment with a variety of metrics including PMI (point-wise mutual information), MI (mutual information), and Chi-squared. We find that PMI is in favour of identifying top keywords in a domain, but Chi-squared can recognize more keywords overall. Based on this observation, we propose a hybrid approach, called HMI, that combines the best of PMI and Chi-squared. HMI outperforms both PMI and Chi-squared. The result is verified by comparing overlapping between the extracted keywords and the author-identified keywords in arXiv data. When the corpora are computer science and physics papers, the top-100 hit rate can reach 0.96 for HMI. We also demonstrate that terminologies can improve documents embeddings. In this experiment, we treat machine-identified multi-word terminologies with one word. Then we use the transformed text as input for the document embedding. Compared with the representations learnt from unigrams only, we observe a performance improvement over 9.41% for F1 score in arXiv data on document classification tasks.

International audience; The main problem the project addresses is reducing time and effort spent by a vendor company on product sustaining phase. This is done by development of the approach for assisting with customer request resolution based on intellectual methods and machine learning algorithms. Solution collects resolved issues from bug tracking systems, local documentation and confluence pages, creates their respective vector models and teaches the algorithm on the data. For each retrieved unresolved issue, related resolved ones and documentation pages are found as a result of using trained algorithm. System also collects user-written rules, that are based on information from the issue, and checks them on each unresolved issue to give recommendations regarding its status change or additional needed information. Based on received information, the final report is constructed to show related issues, documentation and recommendations for each unresolved issue in a user-friendly manner. System was tested on Apache Kafka project issues and compared to manual approach performed on same data. The average time to analyze unresolved issues using the automated approach was 12.2 minutes, and the average time spent with the manual approach was 18.4 minutes, which means that our solution decreases complexity of issue analysis by ~33%.

MEng (Computer and Electronic Engineering), North-West University, Potchefstroom Campus We investigate whether word embeddings using deep neural networks can assist in the analysis of text produced by a speech-recognition system. In particular, we develop algorithms to identify which words are incorrectly detected by a speech-recognition system in broadcast news. The multilingual corpus used in this investigation con-tains speech from the eleven official South African languages, as well as Hindi. Pop-ular word embedding algorithms such as word2vec and fastText are investigated and compared with context-specific embedding representations such as doc2vec and non-context specific statistical sentence embedding methods such as term frequency-inverse document frequency (TF-IDF), which is used as our baseline method. These various embedding methods are then used as fixed length input representations for a logistic regression and feedforward neural network classifier. The output is used as an addi-tional categorical input feature to a CatBoost classifier to determine whether the words were correctly recognised. Other methods are also investigated, including a method that uses the word embedding itself and cosine similarity between specific keywords to identify whether a specific keyword was correctly detected. When relying only on the speech-text data, the best result was obtained using the TF-IDF document embed-dings as input features to a feedforward neural network. Adding the output from the feedforward neural network as an additional feature to the CatBoost classifier did not enhance the classifier’s performance compared to using the non-textual information provided, although adding the output from a weaker classifier was somewhat beneficial. Masters

This thesis explores and compares various methods for producing vector representation of unstructured log data. Ericsson wanted to investigate machine learning methods to analyze logs produced by their systems to reduce the cost and effort required for manual log analysis. Four NLP methods were used to produce vector embeddings for logs: Doc2Vec, DAN, XLNet, and RoBERTa. Also, a Random forest classifier was used to classify those embeddings. The experiments were performed on three different datasets and the results showed that the performance of the models varied based on the dataset being used. The results also show that in the case of log data, fine-tuning makes the transformer models computationally heavy and the performance gain is very low. RoBERTa without fine-tuning produced optimal vector representations for the first and third datasets used whereas DAN had better performance for the second dataset. The study also concluded that the NLP models were able to better understand and classify the third dataset as it contained more plain text information as contrasted against more technical and less human readable datasets. I den här uppsatsen undersöks och jämförs olika metoder för att skapa vektorrepresentationer av ostrukturerad loggdata. Ericsson vill undersöka om det är möjligt att använda tekniker inom maskininlärning för att analysera loggdata som produceras av deras nuvarande system och på så sätt underlätta och minska kostnaderna för manuell logganalys. Fyra olika språkteknologier undersöks för att skapa vektorrepresentationer av loggdata: Doc2vec, DAN, XLNet and RoBERTa. Dessutom används en Random Forest klassificerare för att klassificera vektorrepresentationerna. Experimenten utfördes på tre olika datamängder och resultaten visade att modellernas prestanda varierade baserat på datauppsättningen som används. Resultaten visar också att finjustering av transformatormodeller gör dem beräkningskrävande och prestandavinsten är liten.. RoBERTa utan finjustering producerade optimala vektorrepresentationer för de första och tredje dataset som användes, medan DAN hade bättre prestanda för det andra datasetet. Studien visar också att språkmodellerna kunde klassificera det tredje datasetet bättre då det innehöll mer information i klartext jämfört med mer tekniska och mindre lättlästa dataseten.

Analyze performance of unsupervised embedding algorithms in sentiment analysis of knowledge-rich data sets. We apply state-of-the-art embedding algorithms Word2Vec and Doc2Vec as the learning techniques. The algorithms build word and document embeddings in an unsupervised manner. To assess the algorithms’ performance, we define sentiment metrics and use a semantic lexicon SentiWordNet (SWN) to establish the benchmark measures. Our empirical results are obtained on the Obesity data set from i2b2 clinical discharge summaries and the Reuters Science dataset. We use the Welch’s test to analyze the obtained sentiment evaluation. On the Obesity data, the Welch’s test found significant difference between the SWN evaluation of the most positive and most negative texts. On the same data, the Word2Vec results support the SWN results, whereas the Doc2Vec results partially correspond to the Word2Vec and the SWN results. On the Reuters data, the Welch’s test did not find significant difference between the SWN evaluation of the most positive and most negative texts. On the same data, Word2Vec and Doc2Vec results only in part correspond to the SWN results. In unsupervised sentiment analysis of medical and scientific texts, the Word2Vec sentiment analysis has been more consistent with the SentiWordNet sentiment assessment than the Doc2Vec sentiment analysis. The Welch’s test of the SentiWordNet results has been a strong indicator of future correspondence between Word2Vec and SentiWordNet results.

The success of Artificial Intelligence has attracted researchers from different areas to use computational techniques in tasks of extracting knowledge from unstructured data, such as textual documents, presenting itself as a possible solution for the classification of Specialized Knowledge of Physics Teachers , which consists of an analytical tool that describes the Knowledge of Physics (PK) and Didactic knowledge of Content (PCK), considered very important to assist in the identification and analysis of knowledge mobilized by teachers in their teaching practices. However, the task of identifying and classifying knowledge present in textual documents presents some challenges, such as: investigating textual documents is laborious, time-consuming and involves the labor of specialized people. In this sense, the objective of the research is to analyze the effectiveness of the algorithms used in the automatic classification of Expert Knowledge of Physics Teachers (PTSK) in texts from a previously classified database. The methodological approach is quantitative, exploratory and applied to predict the class Knowledge of Physics (PK) or Didactic Knowledge of Content (PCK) to characterize knowledge. For this, two algorithms were used: doc2vec and J48 and the results were analyzed based on the results achieved in the validation metrics. The best result was achieved with doc2vec, obtaining an 88% success rate. Based on the results achieved, it can be concluded that the strategy of using artificial intelligence for the automatic classification of knowledge of Physics teachers is a plausible solution. El éxito de la Inteligencia Artificial ha atraído a investigadores de diferentes áreas a utilizar técnicas computacionales en tareas de extracción de conocimiento a partir de datos no estructurados, como documentos textuales, presentándose como una posible solución para la clasificación de Conocimientos Especializados de Profesores de Física. , que consiste en una herramienta analítica que describe el Conocimiento de la Física (PK) y el Conocimiento Didáctico de los Contenidos (PCK), considerados muy importantes para ayudar en la identificación y análisis de los conocimientos movilizados por los docentes en sus prácticas docentes. Sin embargo, la tarea de identificar y clasificar el conocimiento presente en los documentos textuales presenta algunos desafíos, tales como: investigar documentos textuales es laborioso, requiere mucho tiempo e implica el trabajo de personas especializadas. En este sentido, el objetivo de la investigación es analizar la efectividad de los algoritmos utilizados en la clasificación automática del Conocimiento Experto de Profesores de Física (PTSK) en textos de una base de datos previamente clasificada. El enfoque metodológico es cuantitativo, exploratorio y aplicado para predecir la clase Conocimiento de la Física (PK) o Conocimiento Didáctico del Contenido (PCK) para caracterizar conocimientos. Para ello se utilizaron dos algoritmos: doc2vec y J48 y los resultados se analizaron en base a los resultados obtenidos en las métricas de validación. El mejor resultado se logró con doc2vec, obteniendo una tasa de éxito del 88%. A partir de los resultados obtenidos, se puede concluir que la estrategia de utilizar la inteligencia artificial para la clasificación automática del conocimiento de los profesores de Física es una solución plausible. O sucesso da Inteligência Artificial tem atraído pesquisadores de diversas áreas para o uso de técnicas computacionais em tarefas de extração de conhecimentos de dados não estruturados, como os documentos textuais, apresentando-se como uma solução possível para a classificação de Conhecimentos Especializado de Professores de Física, que consiste em uma ferramenta analítica que descreve os Conhecimentos da Física (PK) e os conhecimentos Didáticos do Conteúdo (PCK), considerada muito importante para auxiliar na identificação e na análise de conhecimentos mobilizados pelos professores em suas práticas de ensino. Porém, a tarefa de identificação e classificação de conhecimentos presentes em documentos textuais apresentam alguns desafios, como: as investigações em documentos textuais é trabalhosa, demorada e envolve mão de obra de pessoas especializadas. Nesse sentido, o objetivo da pesquisa é analisar a eficácia dos algoritmos utilizados na classificação automática de Conhecimentos Especializados de Professores de Física (PTSK) em textos de uma base de dados previamente classificada. O encaminhamento metodológico é de natureza quantitativa, exploratória e aplicada para prever a classe Conhecimentos da Física (PK) ou Conhecimentos Didáticos do Conteúdo (PCK) para caracterizar um conhecimento. Para isso, foram usados dois algoritmos: o doc2vec e o J48 e os resultados foram analisados com base nos resultados alcançados nas métricas de validação. O melhor resultado foi alcançado com o doc2vec, obtendo 88% de taxa de acerto. Com base nos resultados atingidos, pode-se concluir que a estratégia de usar inteligência artificial para a classificação automática de conhecimentos de professores de Física é uma solução plausível.

In the community of bibliometrics, author name ambiguity means that author's name is not a reliable identier for associating academic papers with their authors. Author name ambiguity has been the problem in bibliometrics and service providers like Google Scholar, generating a domain of study call Author Name Disambiguation (AND). Author name ambiguity is often tackled using classication techniques, where labeled papers are provided, and papers are assigned to correct authors according to the paper text and paper citations. When applying classication methods to author name disambiguation, two issues stand out: one is that a paper has multiple views (paper text and citation network). The other is the lack of training data: there are not many papers that are labeled. To cope with these two issues, we propose to use the co-training algorithm in AND. The co-training algorithm uses two views to classify papers iteratively and add the top selected papers into the training pool. We demonstrate that the co-training algorithm outperforms the baseline multi-view classication algorithm. We also experiment with hyper-parameters in the co-training algorithm. The experiment is done on the PubMed dataset, where authors are labeled with ORCID. Papers are represented by two embeddings that are learnt from paper content and paper citation network separately. Baseline classiers for comparison are logistic regression and SVM.

Pornographic and gambling websites become increasingly stubborn via disguising, misleading, blocking, and bypassing, which hinder the construction of a safe and healthy network environment. However, most traditional approaches conduct the detection process through a single aspect of these sites, which would fail to handle the more intricate and challenging situations. To alleviate this problem, this study proposed an automatic detection system for porn and gambling websites based on visual and textual content using a decision mechanism (PG-VTDM). This system can be applied to the intelligent wireless router at home or school to realize the identification, blocking, and warning of ill-suited websites. First, Doc2Vec was employed to learn the textual features that can be used to represent the textual content in the hypertext markup language (HTML) source code of the websites. In addition, the traditional bag-of-visual-words (BoVW) was improved by introducing local spatial relationships of feature points for better representing the visual features of the website screenshot. Then, based on these two types of features, a text classifier and an image classifier were both trained. In the decision mechanism, a data fusion algorithm based on logistic regression (LR) was designed to obtain the final prediction result by measuring the contribution of the two classification results to the final category prediction. The efficiency of this proposed approach was substantiated via comparison experiments using gambling and porn website datasets crawled from the Internet. The proposed approach outperformed the approach based on a single feature and some state-of-the-art approaches, with accuracy, precision, and F-measure all over 99%.

Recently, anyone can easily publish a lot of information in blogs, SNS, and so on. Then, the amount of information existing on the web continues to increase. For that reason, it is not only time consuming to find the information that is needed, it is also important to distinguish lie information, malicious information, or wrong information. In this research, by comparing similarity with past data, we discover whether incorrect information and information that can be subject to criticism exist compared with other information.

With the growth of online information and sudden expansion in the number of electronic documents provided on websites and in electronic libraries, there is difficulty in categorizing text documents. Therefore, a rule-based approach is a solution to this problem, the purpose of this study is to classify documents by using a rule-based. This paper deals with the rule-based approach with the embedding technique for a document to vector (doc2vec) files. An experiment was performed on two data sets Reuters-21578 and the 20 Newsgroups to classify the top ten categories of these data sets by using a document to vector rule-based (D2vecRule). Finally, this method provided us a good classification result according to the F-measures and implementation time metrics. In conclusion, it was observed that our algorithm document to vector rule-based (D2vecRule) was good when compared with other algorithms such as JRip, One R, and ZeroR applied to the same Reuters-21578 dataset.

Durante a fase de manutenção de software, os processos de localização de características e bugs de software exercem um relevante papel na recuperação da rastreabilidade entre descrições expressas em linguagem natural e trechos de código fonte. Entretanto, ferramentas automatizadas propostas para esta finalidade não tem apresentado acurácia que possa ser considerada como definitiva. Neste sentido, o objetivo principal deste trabalho foi o aprimoramento de vetores de parágrafos, produzidos por uma rede neural Artificial empregada no algoritmo Doc2vec, e aplicados ao processo de localização de características e de bugs de software. Assim, melhor definidos, estes vetores puderam ser usados no cálculo de similaridades entre descrições de características e métodos do código fonte. Da mesma forma, a similaridade entre bugs e classes do código fonte pôde ser aprimorada. Para atingir este objetivo, foi aplicada uma taxa de aprendizado cíclica além da alteração da função de custo da rede neural que estavam associados ao algoritmo Doc2vec. Outras abordagens utilizadas reforçaram os ganhos em acurácia para o algoritmo Doc2vec a partir da combinação de rankings obtidos em ferramentas que expressam o estado da arte em Engenharia de Software. Um conjunto de abordagens que não apresentaram resultados satisfatórios também foram experimentadas e são descritas neste trabalho. Dentre estas podem ser citadas a melhoria da qualidade dos vetores representativos de métodos e classes do código fonte a partir de outros sistemas escritos na linguagem de programação Java; uso de pesos sintáticos diferenciados para termos que participam da composição de vetores relacionados às classes e métodos; e uso do algoritmo fastText para geração de vetores. Para cada abordagem proposta, ponderações foram feitas no sentido de se avaliar sua efetividade de uso em contextos de localização de bugs e características. Em resumo, com uma combinação das melhorias propostas para a rede neural artificial, a acurácia do estado da arte para a tarefa de localização de características pôde ser superada. Além disso, também foi possível melhorar a localização de bugs realizada a partir do uso do algoritmo Doc2vec bem como determinar quais hiperparâmetros eram mais influentes para melhoria da acurácia. Throughout a software maintenance phase, the processes of feature and bug location play an important role in the retrieval of traceability between natural language and source code. However, existing automated tools, which were designed to perform these tasks, have not yet presented the desirable accuracy expected as a definitive outcome. In this sense, this work main objective seeks to attend to the improvement of paragraph vectors, produced by artificial neural network in the Doc2vec algorithm, and applied into the processes of feature and software bug location. By improving the vector quality, these were used in the calculation of similarities found between feature descriptions and source code methods. Also, improvements in identifying bugs similarity and source code classes were observed. In order to reach this result, a cyclic learning rate was applied to the task in addition to customizing neural network loss function, which were associated with the Doc2vec algorithm. Furthermore, other approaches used have also demonstrated the gains in accuracy for the Doc2vec algorithm, provided by the combination of rankings obtained in tools that express the state of the art in Software Engineering. Yet, a set of other minor approaches were applied to this work, such as: improvement in the quality of representative vectors from source code methods, departing from other systems source code; usage of word syntactic influence within paragraphs extracted from source code; and usage of fastText algorithm to generate paragraph vectors. For each proposed approach, considerations were made in order to evaluate its effectiveness in feature and bug location tasks. In sum, the improvements proposed by this work to artificial neural network allowed to improve state of the art work in feature location tasks. Moreover, improvements in bug location were made by using the Doc2vec algorithm, and most influential hyper parameters were defined to improve accuracy. Tese (Doutorado)

Investment universe means a pool of selected assets likely to be profitable. In general, assets related to a common theme or concept are selected to form an investment universe. A well organized investment universe has a strong common theme and reduces the risk by diversifying the assets included in the universe. To form or revise an investment universe, human efforts are required by using domain knowledge about the theme, but it is hard to suggest an investment universe that reflects the latest market trends. In this paper, we propose an automated investment universe selection method based on theme keywords. The theme keywords are extracted from the news articles and the business section on the companies which are included in the S&P 500 index. After that, securities are selected to form an investment universe for each theme keywords. We employ a similarity value between the security vector and the theme keyword vector to select the related securities for the certain keywords. Regarding the vector representations, word and document embeddings are carried out using both news articles and the business section on the companies. Stock price movements of the selected securities are similar which means that the investment universes are well organized to suggest the assets tightly associated with the theme. The experimental results show that the proposed method has high future returns on the investment universes with low or high historical stock price returns.

With the growing popularity of machine learning algorithms, dramatic advances have been made for code completion, and specifically method-call completion. These advances were also possible thanks to the availability of large code repositories to learn from and to the well-defined boundaries of the method-call completion problem. This is, however, not the case for design completion, where model repositories are scarce and the space of possibilities for design completion is theoretically infinite.

近年，動画ではYouTube やニコニコ動画が，小説では小説家になろうといった利用者投稿型のCGM (Consumer Generated Media) サイトが人気である． CGM サイトには毎日多数のコンテンツが投稿されており，また膨大な利用者がコンテンツを閲覧している． 現在，CGM サイトへの投稿コンテンツの画一化が懸念されている．既に見たことのあるようなコンテンツや派生コンテンツの増加が感じられる．我々は今回，「小説家になろう」を対象に，Cos 類似度の総和を調べることで小説家になろうの画一化について定量的に評価をおこなった．次に，偏りのあるジャンルを特定するために，クラスタリングによるオンライン小説の多様性動向を調査する．各小説のあらすじをDoc2Vec を用いてベクトルで表現し，最後に，小説のベクトル群をX-means でクラスタリングする． クラスタ数やクラスタの要素数を分析することで，投稿小説の多様性動向の分析を行う．, In recent years, CGM (Consumer Generated Media) site, such as YouTube.com and Nicovideo.jp for movies, syosetu.com for novels, become very popular. Many contents are posted on those CGM sites everyday, and a huge number of users are browsing the contents. Now a day, some bloggers mentioned that similar contents are well posted to the CGM site. It is felt that contents and derivative contents which have already seen have increased. We proposed contents diversity metric, that is the sum of cosine similarities of contents. We applied our metric to contents of nicovideo.jp and syosetu.com, and found that contents diversity was decrease in both CGM. To identify bias of contents, we investigate the diversity trend of online novel using clustering. We extract synopsis part from each novels, and vectorize them by Doc2Vec. After that, we clustered the vectors using X-means clustering. We report our method and results of clustering.

International audience; The software sustaining phase is one of the essential stages of the software development life cycle. At this stage, customers can contact support and software engineers to request a resolution of any problem they meet during software utilization including questions of how to operate with software, where to find the information about particular functions and other relevant questions on software product. The work is devoted to research in the field of software sustainment automation. The distinctive feature of the article is the suggested semantic documentation search approach with the Doc2Vec machine learning algorithm, which allows automating automate customer requests resolution. Proposed semantic search is performed on documentation files, like PDFs, Microsoft Office documents, wiki pages and other text files with relevant information about the product. Documentation files, including page numbers, that have the closest semantic similarity to the textual description of an unresolved customer request, help the developer resolve the incoming request more efficiently and in a shorter time. The proposed approach is implemented in a software tool to automate the analysis of unresolved customer requests and provide recommendations to help in solving each of those requests. The results show the advantages of using the tool in the process of software product support.

Innovation Management Software contains complex data with many different variables. This data is usually presented in tabular form or with isolated graphs that visualize a single independent aspect of a dataset. However, displaying this data with interconnected, interactive charts provide much more flexibility and opportunities for working with and understanding the data. Charts that show multiple aspects of the data at once can help in uncovering hidden relationships between different aspects of the data and in finding new insights that might be difficult to see with the traditional way of displaying data. The size and complexity of the available data also invites analyzing it with machine learning techniques. In this thesis it is first explored how machine learning techniques can be used to gain additional insight from the data and then the results of this investigation are used together with the original data in order to build a prototypical dashboard for exploratory visual data analysis. This dashboard is then evaluated by means of ICE-T heuristics and the results and findings are discussed.

Multimodal machine learning is a subfield of machine learning that aims to relate data from different modalities, such as texts and images. One of the many applications that could be built upon this technique is an image retrieval system that, given a text query, retrieves suitable images from a database. In this thesis, a retrieval system based on canonical correlation is used to suggest images for news articles. Different dense text representations produced by Word2vec and Doc2vec, and image representations produced by pre-trained convolutional neural networks are explored to find out how they affect the suggestions. Which part of an article is best suited as a query to the system is also studied. Also, experiments are carried out to determine if an article's date of publication can be used to improve the suggestions. The results show that Word2vec outperforms Doc2vec in the task, which indicates that the meaning of article texts are not as important as the individual words they consist of. Furthermore, the queries are improved by rewarding words that are particularly significant.

Este Trabajo Fin de Grado plantea el diseño y la implementación de un sistema de clasificación de perfiles. Dicho sistema tiene como objetivo la obtención de una lista de perfiles similares para un perfil dado de una conocida red social. Este sistema se valdrá de técnicas de procesamiento del lenguaje natural para el análisis de los perfiles, obteniendo características semánticas y estilísticas de ellos, y será desarrollado en Python. Durante el desarrollo de este TFG se explicarán en detalle los pasos seguidos para el diseño del sistema: la obtención de datos, el almacenamiento de éstos, los modelos de procesamiento del lenguaje natural utilizados, los resultados obtenidos para cada modelo y el análisis de todos los resultados

Submitted by Ana Rebelo (amsr@isep.ipp.pt) on 2021-02-08T16:20:36Z No. of bitstreams: 1 DM_JoseCardoso_2020_MEI.pdf: 4860780 bytes, checksum: d892f8dba4b71af5366060edb64aef50 (MD5) Approved for entry into archive by Ana Rebelo (amsr@isep.ipp.pt) on 2021-02-08T16:21:00Z (GMT) No. of bitstreams: 1 DM_JoseCardoso_2020_MEI.pdf: 4860780 bytes, checksum: d892f8dba4b71af5366060edb64aef50 (MD5) Made available in DSpace on 2021-02-08T16:21:00Z (GMT). No. of bitstreams: 1 DM_JoseCardoso_2020_MEI.pdf: 4860780 bytes, checksum: d892f8dba4b71af5366060edb64aef50 (MD5) Previous issue date: 2020

We investigate whether word embeddings using deep neural networks can assist in the analysis of text produced by a speechrecognition system. In particular, we develop algorithms to identify which words are incorrectly detected by a speech-recognition system in broadcast news. The multilingual corpus used in this investigation contains speech from the eleven official South African languages, as well as Hindi. Popular word embedding algorithms such as Word2Vec and fastText are investigated and compared with context-specific embedding representations such as Doc2Vec and non-context specific statistical sentence embedding methods such as term frequency-inverse document frequency (TFIDF), which is used as our baseline method. These various embeddding methods are then used as fixed length input representations for a logistic regression and feed forward neural network classifier. The output is used as an additional categorical input feature to a CatBoost classifier to determine whether the words were correctly recognised. Other methods are also investigated, including a method that uses the word embedding itself and cosine similarity between specific keywords to identify whether a specific keyword was correctly detected. When relying only on the speech-text data, the best result was obtained using the TFIDF document embeddings as input features to a feed forward neural network. Adding the output from the feed forward neural network as an additional feature to the CatBoost classifier did not enhance the classifier’s performance compared to using the non-textual information provided, although adding the output from a weaker classifier was somewhat beneficial

A recommender system is a popular tool used by companies to increase customer satisfaction and to increase revenue. Collaborative filtering and content-based filtering are the two most common approaches when implementing a recommender system, where the former provides recommendations based on user behaviour, and the latter uses the characteristics of the items that are recommended. The aim of the study was to develop and compare different recommender system approaches, for both book and author recommendations and their ability to predict user ratings of an e-book application. The evaluation of the models was done by measuring root mean square error (RMSE) and mean absolute error (MAE). Two pure models were developed, one based on collaborative filtering and one based on content-based filtering. Also, three different hybrid models using a combination of the two pure approaches were developed and compared to the pure models. The study also explored how aggregation of book data to author level could be used to implement an author recommender system. The results showed that the aggregated author data was more difficult to predict. However, it was difficult to draw any conclusions of the performance on author data due to the data aggregation. Although it was clear that it was possible to derive author recommendations based on data from books. The study also showed that the collaborative filtering model performed better than the content-based filtering model according to RMSE but not according to MAE. The lowest RMSE and MAE, however, were achieved by combining the two approaches in a hybrid model.