Search

The Marine Corps’ 38th Commandant calls for a force that can “affect the behavior of real-world pacing threats” through its “innovation, ingenuity, and willingness to continually adapt to and initiate changes in the operating environment.” Having a healthy and inclusive environment is vital to the Marine Corps’ success in developing and maintaining a force that can rise to the higher standard. Currently, the closest snapshot of the health of Marine Corps culture is collected via command climate surveys, which contain quantitative and free-response questions. Senior levels of Marine Corps leadership do not gain value from the free-text data because 1) analysis on free-text comments is time consuming and no analysis is done on those comments for the commander, and 2) there is no trend analysis on the aggregate of surveys over time and throughout commands, so they are not commander specific. This thesis developed a tool to gain value from free-text comments; it applied natural language processing (NLP) techniques, specifically topic extraction and sentiment analysis, on free-text comments from command climate surveys to provide a quantitative analysis on the health of culture in Marine Corps units. This approach can also be replicated to other services and can inspire efforts that will positively impact talent management and retention. Captain, United States Marine Corps Approved for public release. Distribution is unlimited.

Aquest projecte planteja la creació d'eines per a poder identificar pàgines web a partir de tràfic contingut a una captura en format pcap. Per poder-ho fer, s'ha seguit la hipòtesi que el conjunt d'adreces IP de les quals es reben o s'envien paquets identifica una pàgina web en concret. Així doncs, s'han desenvolupat les tres eines necessàries per a l'obtenció de traces, creació del model d'aprenentatge automàtic i cerca de la pàgina web. This project contemplates the creation of tools with the intention of identifying webpages from traffic contained in a capture with pcap format. In order to do so, the hypothesis of the set of IP addresses to which packets are transmitted or received identifying a specific webpage has been followed. Therefore, the three needed tools to get the traces, create the machine learning model and search the website have been developed.

Aquest projecte compara diferents mètriques de distància aplicades a embeddings de documents sobre diferents gràfics conceptuals. Al saber quines mètriques són efectives d'entre totes les analitzades sota uns límits establerts, facilita l'aplicació d'aquestes a altres àmbits. Per tant, coneixent els resultats del projecte tindrem la certesa que les mètriques funcionaran correctament, estalviant temps i optimitzant els recursos de qualsevol negoci quan es necessita una comparació de diferents documents per a una aplicació més professional. Aquest projecte aplica dos models NLP: Doc2Vec i Word2Vec. L'objectiu d'aquesta investigació és profunditzar en el funcionament de les mètriques estudiades sobre aquests models. S'han creat diverses pipelines per automatitzar tot el processament de text, l'entrenament de models i l'exportació de resultats. Això ha permès poder crear tres contexts per model, fent que els resultats siguin més diversos i tindre més marge per comparar els resultats. Entre tota l'experimentació realitzada, cal destacar l'estructura de proves combinatòries que ens ha permès crear resultats d'experimentació prou grans per a avaluar discriminant per Model-Context-Mètrica. Finalment, les conclusions extretes han sigut bastant clares, la mètrica amb millor comportament general ha sigut Cosine Similarity, on per grups discriminats (Blogs, Científic, Wikipedia, Doc2Vec i Word2Vec) ha tingut el millor rendiment exceptuant Wikipedia, que ha estat en segon lloc. This project compares different distance metrics applied to document embeddings above different conceptual graphics. By knowing which metrics are effective among all the ones analysed under established limits, it facilitates the application of the aforementioned in other scopes. Therefore we have the certainty that the metrics will work properly, saving time and optimising the resources of any business when a comparison of different documents is needed for a more professional application. This project applies two NLP models: Doc2Vec and Word2Vec. The goal of this research is to delve into how the studied metrics work on these models: different pipelines have been created in order to have all the text processes automated, the fitting of the model process and the result exportation. This has allowed us to create three contexts per model, making the results more diverse and having a bigger margin to compare the results. Among all the experimentation done, the combinatory test structure should be highlighted as it has allowed us to create experimentation results large enough to evaluate the discrimination by Model-Context-Metric. Finally, the conclusions extracted are clear: the metric with best general behaviour has been Cosine Similarity, where by discriminated groups (Blogs, Scientific, Wikipedia, Doc2Vec and Word2Vec) has had the best performance excepting the Wikipedia context, that has resulted in a second place.

Выпускная квалификационная работа магистра посвящена исследованию в области определения вредоносного содержания в Ñ‚ÐµÐºÑÑ‚Ð¾Ð²Ñ‹Ñ Ð´Ð°Ð½Ð½Ñ‹Ñ . Исследованы Ñ Ð°Ñ€Ð°ÐºÑ‚ÐµÑ€Ð¸ÑÑ‚Ð¸ÐºÐ¸, изучены существующие Ñ‚ÐµÑ Ð½Ð¾Ð»Ð¾Ð³Ð¸Ð¸ для определения вредоносности в Ñ‚ÐµÐºÑÑ‚Ð¾Ð²Ñ‹Ñ Ð´Ð°Ð½Ð½Ñ‹Ñ , проведен Ð¸Ñ ÑÑ€Ð°Ð²Ð½Ð¸Ñ‚ÐµÐ»ÑŒÐ½Ñ‹Ð¹ анализ, проведен обзор методов вложений слов, выявлены недостатки и Ð½ÐµÐ¾Ð±Ñ Ð¾Ð´Ð¸Ð¼Ð¾ÑÑ‚ÑŒ новый Ð¿Ð¾Ð´Ñ Ð¾Ð´ с использованием алгоритма Doc2Vec. Представлены алгоритмы классификаторов на основе алгоритмов машинного обучения.В Ñ€Ð°Ð¼ÐºÐ°Ñ Ñ€Ð°Ð±Ð¾Ñ‚Ñ‹ предложен Ð¿Ð¾Ð´Ñ Ð¾Ð´, который позволит определять вредонсное содержание в Ñ‚ÐµÐºÑÑ‚Ð¾Ð²Ñ‹Ñ Ð´Ð°Ð½Ð½Ñ‹Ñ . В основе Ð¿Ð¾Ð´Ñ Ð¾Ð´Ð° будет лежать алгоритм Doc2Vec. Рассмотрены методы предварительного обработанного набора Ñ‚ÐµÐºÑÑ‚Ð¾Ð²Ñ‹Ñ Ð´Ð°Ð½Ð½Ñ‹Ñ . Описано использование алгоритма Doc2Vec для обучения модели.Предложенный Ð¿Ð¾Ð´Ñ Ð¾Ð´ был реализован в программной системе определения вредоносного содержания Ð²Â Ñ‚ÐµÐºÑÑ‚Ð¾Ð²Ñ‹Ñ Ð´Ð°Ð½Ð½Ñ‹Ñ . По результатам реализации предложенный Ð¿Ð¾Ð´Ñ Ð¾Ð´, точнее, чем существующие использованные другими алгоритмами вложеня слов (2-27% accuracy)., The master's thesis focuses on research into detecting fraudulent content in text data. Characteristics are examined, existing technologies for determining the harmfulness of text data are examinated and compared, a review of word embedding methods is conducted, and flaws and the need for a new approach based on the Doc2Vec algorithm are identified. Classifier algorithms based on machine learnign algorithms are presented.An approach is proposed as part of the work that will allow you to detect fraudulent content in text data. The Doc2Vec algorithm will be used to guide the process. The methods for pre-processed text data are discussed. The Doc2Vec algorithm is described in detail for model training.The proposed method was used to create a software system that could detect fraudulent context in text data. According to the results of the implementation, the proposed approach is more accurate than the existing ones used by other word embedding algorithms (2-27% accuracy).

In this thesis, we investigate the use of self-training, a semi-supervised learning method, to improve binary classification of text documents. This means making use of unlabeled samples, since labeled samples can be expensive to generate. More specifically, we want to classify emails that are retrieved by Skandinaviska Enskilda Banken (SEB). The method is tested on two datasets: the first is IMDB reviews, consisting of both labeled (good or bad) and unlabeled movie reviews; the second is provided by SEB and consists of labeled and unlabeled emails. First, supervised learning was investigated. Three different vectorization methods including two bag-of-words models and one doc2vec model were included. These were tested using five different machine learning classification methods. The comparison of the F1-score showed that doc2vec vectorization and the logistic regression classification method performed well and was used in the self-training investigation. We find that self-training on the IMDB dataset only yielded improvement for low number of labeled samples. For the SEB dataset we find that by using self-training, we can achieve the same F1-score using only around 1000 labeled samples (less than 10% of the labeled dataset), as using supervised methods on the full labeled set. We conclude that self-training can improve classification performance and also be used indirectly to reduce manual labeling efforts. I denna rapport undersöker vi användningen av self-training, en metod för semi-vägledd inlärning, för att förbättra binär klassificering av textdokument. Mer specifikt vill vi klassificera e-postbrev som hämtats av Skandinaviska Enskilda Banken (SEB). Målet är att kunna nyttja stora mängder omärkt data, eftersom att märka datapunkter kan vara dyrt. Metoden prövas på två datamängder: den första är IMDB-recensioner, bestående av både märkta (bra eller dåliga) och omärkta filmrecensioner; den andra datamängden tillhandahålls av SEB och består också av märkta och omärkta e-postbrev (relevanta och irrelevanta). Först prövas supervised learning. Tre olika vektoriseringsmetoder inklusive två bag-of-words-modeller och en doc2vec-modell inkluderas. Dessa testades med hjälp av fem olika klassificeringsmetoder för maskininlärning. Jämförelsen av F1-metriken visade att doc2vec-vektorisering och den logistisk regression fungerade bra och användes för semi-vägledd inlärning. Vi finner att self-training på IMDB-datamängden endast gav förbättring för ett lågt antal märkta datapunkter. För datamängden från SEB finner vi att genom att använda self-training kan vi uppnå samma F1-metrikvärde med endast 1000 märkta datapunkter (mindre än 10% av den märkta datamängden), som att använda metoder för vägledd inlärning på den fullständiga märkta datamängden. Vi drar slutsatsen att self-training förbättrar klassificering och kan användas för att minska mängden data som behöver märkas manuellt.

In this paper we perform a comparative analysis of three models for a feature representation of text documents in the context of document classification. In particular, we consider the most often used family of bag-of-words models, the recently proposed continuous space models word2vec and doc2vec, and the model based on the representation of text documents as language networks. While the bag-of-word models have been extensively used for the document classification task, the performance of the other two models for the same task have not been well understood. This is especially true for the network-based models that have been rarely considered for the representation of text documents for classification. In this study, we measure the performance of the document classifiers trained using the method of random forests for features generated with the three models and their variants. Multi-objective rankings are proposed as the framework for multi-criteria comparative analysis of the results. Finally, the results of the empirical comparison show that the commonly used bag-of-words model has a performance comparable to the one obtained by the emerging continuous-space model of doc2vec. In particular, the low-dimensional variants of doc2vec generating up to 75 features are among the top-performing document representation models. The results finally point out that doc2vec shows a superior performance in the tasks of classifying large documents.

Large pretrained language models like BERT have shown excellent generalization properties and have advanced the state of the art on various NLP tasks. In this paper we evaluate Finnish BERT (FinBERT) model on the IPTC Subject Codes prediction task. We compare it to a simpler Doc2Vec model used as a baseline. Due to hierarchical nature of IPTC Subject Codes, we also evaluate the effect of encoding the hierarchy in the network layer topology. Contrary to our expectations, a simpler baseline Doc2Vec model clearly outperforms the more complex FinBERT model and our attempts to encode hierarchy in a prediction network do not yield systematic improvement.

The abundance and real-time availability of Twitter data have proved beneficial in detecting events in various domains such as emergency situations, crime detection, public health, place recommendations, etc. Nevertheless, two critical challenges occur while detecting events using social media data. First, the uncertainty in capturing the contextual relationship among tweets, which is the result of the limited availability of the contextual information due to the small length of tweets. Second, the high computation cost required in event detection due to massive data processing. Earlier research works, addressing these challenges, have tried to capture the contextual information by using the dense vector representations of texts leveraging deep neural word embedding generation models such as Word2Vec and GloVe. However, these models are trained on the Euclidean vector space which fails to amalgamate the directional information of the vectors with the semantic information in text, incurring high computational costs. To target both the problems simultaneously, we propose modeling Twitter data as a graph-of-sentences which retains the contextual relationships while maintaining lower computational cost. The proposed model captures contextual information using JoSE, a spherical vector representation leveraging the word-word and word-paragraph semantic co-occurrence statistics in a spherical generative model. Furthermore, the framework uses the weighted-graph model to capture all the relationships among the Twitter data efficiently. The graph is further pruned with the help of the graph component filtering approach. The graph clustering model, employed to detect the events, leverages the edge weights and the partial-k clustering approach maintaining low computation costs. The experimentation on the annotated benchmark Twitter data set and the real-world datasets show improved run-time performance up to 30% while maintaining the qualitative performance (F1-score) comparable to the state-of-the-art models.

Document-level sentiment analysis is a challenging task given the large size of the text, which leads to an abundance of words and opinions, at times contradictory, in the same document. This analysis is particularly useful in analyzing press articles and blog posts about a particular product or company, and it requires a high concentration, especially when the topic being discussed is sensitive. Nevertheless, most existing models and techniques are designed to process short text from social networks and collaborative platforms. In this paper, we propose a combination of Convolutional Neural Networks (CNN) and Bidirectional Long Short-Term Memory (BiLSTM) models, with Doc2vec embedding, suitable for opinion analysis in long texts. The CNN-BiLSTM model is compared with CNN, LSTM, BiLSTM and CNN-LSTM models with Word2vec/Doc2vec embeddings. The Doc2vec with CNN-BiLSTM model was applied on French newspapers articles and outperformed the other models with 90.66% accuracy.

Due to the lack of face to face interaction in online learning environment, this article aims essentially to give tutors the opportunity to understand and analyze learners’ cognitive behavior. In this perspective, we propose an automatic system to assess learners’ cognitive presence regarding their social interactions within synchronous online discussions. Combining Natural Language Preprocessing, Doc2Vec document embedding method and machine learning techniques; we first make some transformations and preprocessing to the given transcripts, then we apply Doc2Vec method to represent each message as a vector that will be concatenated with LIWC and context features. The vectors are input data of Naïve Bayes algorithm; a machine learning method; that aims to classify transcripts according to cognitive presence categories.

Delineating urban functional use plays a key role in understanding urban dynamics, evaluating planning strategies and supporting policymaking. In recent years, Points of Interest (POI) data, with precise geolocation and detailed attributes, have become the primary data source for exploring urban functional use from a bottom-up perspective, using local, highly disaggregated, big datasets. Previous studies using POI data have given insufficient consideration to the relationship among POI classes in the spatial context, and have failed to provide a straightforward means by which to classify urban functional areas. This study proposes an approach for delineating urban functional use at the scale of the Lower Layer Super Output Area (LSOA) in Greater London by integrating the Doc2Vec model, a neural network embedding method commonly used in natural language processing for vectoring words and documents from their context. In this study, the neural network vectorises both POI classes (‘Word’) and urban areas (‘Document’) based on their functional context by learning features from the spatial distribution of POIs in the city. Specifically, we first construct POI sequences based on the distribution of POI classes, and add their LSOA IDs as ‘document’ tags. By utilising these constructed POI–LSOA sequences, the Doc2Vec model trains the vectors of 574 POI classes (word vectors) and 4836 LSOAs (document vectors). The vectors of POI classes are then used in calculating the functional similarity scores based on their cosine distance, with the vectors of LSOAs grouped into clusters (i.e., functional areas) via the k-means clustering algorithm. We also identify latent functions in each cluster of LSOAs by performing topic modelling and enrichment factor. Compared with TF–IDF, LDA and Word2Vec models, the Doc2Vec model obtains the highest accuracy when classifying functional areas. This study proposes a straightforward approach in which the model directly trains vectors for urban areas, subsequently using them to classify urban functional areas. By employing the enhanced neural network model with low-cost and ubiquitous POI datasets, this study provides a potential tool with which to monitor urban dynamics in a timely and adaptive manner, thereby providing enhanced, data-driven support to urban planning, development and management.

1st International Conference on Artificial Intelligence and Data Analytics, CAIDA 2021 -- 6 April 2021 through 7 April 2021 -- The rapid increment in internet usage has also resulted in bulk gerenation of text data. Therefore, investigation of new techniques for automatic classification of textual content is needed as manually managing unstructured text is challenging. The main objective of text classification is to train a model such that it should place an unseen text into correct category. In this study, text classification was performed using the Doc2vec word embedding method on the Turkish Text Classification 3600 (TTC-3600) dataset consisting of Turkish news texts and the BBC-News dataset consisting of English news texts. As the classification method, deep learning-based CNN and traditional machine learning classification methods Gauss Naive Bayes (GNB), Random Forest (RF), Naive Bayes (NB) and Support Vector Machine (SVM) are used. In the proposed model, the highest result was obtained as 94.17% in the Turkish dataset and 96.41% in the English dataset in the classification made with CNN. © 2021 IEEE.

Automated terminology extraction is a crucial task in natural language processing and ontology construction. Termhood can be inferred using linguistic and statistic techniques. This thesis focuses on the statistic methods. Inspired by feature selection techniques in documents classification, we experiment with a variety of metrics including PMI (point-wise mutual information), MI (mutual information), and Chi-squared. We find that PMI is in favour of identifying top keywords in a domain, but Chi-squared can recognize more keywords overall. Based on this observation, we propose a hybrid approach, called HMI, that combines the best of PMI and Chi-squared. HMI outperforms both PMI and Chi-squared. The result is verified by comparing overlapping between the extracted keywords and the author-identified keywords in arXiv data. When the corpora are computer science and physics papers, the top-100 hit rate can reach 0.96 for HMI. We also demonstrate that terminologies can improve documents embeddings. In this experiment, we treat machine-identified multi-word terminologies with one word. Then we use the transformed text as input for the document embedding. Compared with the representations learnt from unigrams only, we observe a performance improvement over 9.41% for F1 score in arXiv data on document classification tasks.

International audience; The main problem the project addresses is reducing time and effort spent by a vendor company on product sustaining phase. This is done by development of the approach for assisting with customer request resolution based on intellectual methods and machine learning algorithms. Solution collects resolved issues from bug tracking systems, local documentation and confluence pages, creates their respective vector models and teaches the algorithm on the data. For each retrieved unresolved issue, related resolved ones and documentation pages are found as a result of using trained algorithm. System also collects user-written rules, that are based on information from the issue, and checks them on each unresolved issue to give recommendations regarding its status change or additional needed information. Based on received information, the final report is constructed to show related issues, documentation and recommendations for each unresolved issue in a user-friendly manner. System was tested on Apache Kafka project issues and compared to manual approach performed on same data. The average time to analyze unresolved issues using the automated approach was 12.2 minutes, and the average time spent with the manual approach was 18.4 minutes, which means that our solution decreases complexity of issue analysis by ~33%.

MEng (Computer and Electronic Engineering), North-West University, Potchefstroom Campus We investigate whether word embeddings using deep neural networks can assist in the analysis of text produced by a speech-recognition system. In particular, we develop algorithms to identify which words are incorrectly detected by a speech-recognition system in broadcast news. The multilingual corpus used in this investigation con-tains speech from the eleven official South African languages, as well as Hindi. Pop-ular word embedding algorithms such as word2vec and fastText are investigated and compared with context-specific embedding representations such as doc2vec and non-context specific statistical sentence embedding methods such as term frequency-inverse document frequency (TF-IDF), which is used as our baseline method. These various embedding methods are then used as fixed length input representations for a logistic regression and feedforward neural network classifier. The output is used as an addi-tional categorical input feature to a CatBoost classifier to determine whether the words were correctly recognised. Other methods are also investigated, including a method that uses the word embedding itself and cosine similarity between specific keywords to identify whether a specific keyword was correctly detected. When relying only on the speech-text data, the best result was obtained using the TF-IDF document embed-dings as input features to a feedforward neural network. Adding the output from the feedforward neural network as an additional feature to the CatBoost classifier did not enhance the classifier’s performance compared to using the non-textual information provided, although adding the output from a weaker classifier was somewhat beneficial. Masters

This thesis explores and compares various methods for producing vector representation of unstructured log data. Ericsson wanted to investigate machine learning methods to analyze logs produced by their systems to reduce the cost and effort required for manual log analysis. Four NLP methods were used to produce vector embeddings for logs: Doc2Vec, DAN, XLNet, and RoBERTa. Also, a Random forest classifier was used to classify those embeddings. The experiments were performed on three different datasets and the results showed that the performance of the models varied based on the dataset being used. The results also show that in the case of log data, fine-tuning makes the transformer models computationally heavy and the performance gain is very low. RoBERTa without fine-tuning produced optimal vector representations for the first and third datasets used whereas DAN had better performance for the second dataset. The study also concluded that the NLP models were able to better understand and classify the third dataset as it contained more plain text information as contrasted against more technical and less human readable datasets. I den här uppsatsen undersöks och jämförs olika metoder för att skapa vektorrepresentationer av ostrukturerad loggdata. Ericsson vill undersöka om det är möjligt att använda tekniker inom maskininlärning för att analysera loggdata som produceras av deras nuvarande system och på så sätt underlätta och minska kostnaderna för manuell logganalys. Fyra olika språkteknologier undersöks för att skapa vektorrepresentationer av loggdata: Doc2vec, DAN, XLNet and RoBERTa. Dessutom används en Random Forest klassificerare för att klassificera vektorrepresentationerna. Experimenten utfördes på tre olika datamängder och resultaten visade att modellernas prestanda varierade baserat på datauppsättningen som används. Resultaten visar också att finjustering av transformatormodeller gör dem beräkningskrävande och prestandavinsten är liten.. RoBERTa utan finjustering producerade optimala vektorrepresentationer för de första och tredje dataset som användes, medan DAN hade bättre prestanda för det andra datasetet. Studien visar också att språkmodellerna kunde klassificera det tredje datasetet bättre då det innehöll mer information i klartext jämfört med mer tekniska och mindre lättlästa dataseten.

Analyze performance of unsupervised embedding algorithms in sentiment analysis of knowledge-rich data sets. We apply state-of-the-art embedding algorithms Word2Vec and Doc2Vec as the learning techniques. The algorithms build word and document embeddings in an unsupervised manner. To assess the algorithms’ performance, we define sentiment metrics and use a semantic lexicon SentiWordNet (SWN) to establish the benchmark measures. Our empirical results are obtained on the Obesity data set from i2b2 clinical discharge summaries and the Reuters Science dataset. We use the Welch’s test to analyze the obtained sentiment evaluation. On the Obesity data, the Welch’s test found significant difference between the SWN evaluation of the most positive and most negative texts. On the same data, the Word2Vec results support the SWN results, whereas the Doc2Vec results partially correspond to the Word2Vec and the SWN results. On the Reuters data, the Welch’s test did not find significant difference between the SWN evaluation of the most positive and most negative texts. On the same data, Word2Vec and Doc2Vec results only in part correspond to the SWN results. In unsupervised sentiment analysis of medical and scientific texts, the Word2Vec sentiment analysis has been more consistent with the SentiWordNet sentiment assessment than the Doc2Vec sentiment analysis. The Welch’s test of the SentiWordNet results has been a strong indicator of future correspondence between Word2Vec and SentiWordNet results.

The success of Artificial Intelligence has attracted researchers from different areas to use computational techniques in tasks of extracting knowledge from unstructured data, such as textual documents, presenting itself as a possible solution for the classification of Specialized Knowledge of Physics Teachers , which consists of an analytical tool that describes the Knowledge of Physics (PK) and Didactic knowledge of Content (PCK), considered very important to assist in the identification and analysis of knowledge mobilized by teachers in their teaching practices. However, the task of identifying and classifying knowledge present in textual documents presents some challenges, such as: investigating textual documents is laborious, time-consuming and involves the labor of specialized people. In this sense, the objective of the research is to analyze the effectiveness of the algorithms used in the automatic classification of Expert Knowledge of Physics Teachers (PTSK) in texts from a previously classified database. The methodological approach is quantitative, exploratory and applied to predict the class Knowledge of Physics (PK) or Didactic Knowledge of Content (PCK) to characterize knowledge. For this, two algorithms were used: doc2vec and J48 and the results were analyzed based on the results achieved in the validation metrics. The best result was achieved with doc2vec, obtaining an 88% success rate. Based on the results achieved, it can be concluded that the strategy of using artificial intelligence for the automatic classification of knowledge of Physics teachers is a plausible solution. El éxito de la Inteligencia Artificial ha atraído a investigadores de diferentes áreas a utilizar técnicas computacionales en tareas de extracción de conocimiento a partir de datos no estructurados, como documentos textuales, presentándose como una posible solución para la clasificación de Conocimientos Especializados de Profesores de Física. , que consiste en una herramienta analítica que describe el Conocimiento de la Física (PK) y el Conocimiento Didáctico de los Contenidos (PCK), considerados muy importantes para ayudar en la identificación y análisis de los conocimientos movilizados por los docentes en sus prácticas docentes. Sin embargo, la tarea de identificar y clasificar el conocimiento presente en los documentos textuales presenta algunos desafíos, tales como: investigar documentos textuales es laborioso, requiere mucho tiempo e implica el trabajo de personas especializadas. En este sentido, el objetivo de la investigación es analizar la efectividad de los algoritmos utilizados en la clasificación automática del Conocimiento Experto de Profesores de Física (PTSK) en textos de una base de datos previamente clasificada. El enfoque metodológico es cuantitativo, exploratorio y aplicado para predecir la clase Conocimiento de la Física (PK) o Conocimiento Didáctico del Contenido (PCK) para caracterizar conocimientos. Para ello se utilizaron dos algoritmos: doc2vec y J48 y los resultados se analizaron en base a los resultados obtenidos en las métricas de validación. El mejor resultado se logró con doc2vec, obteniendo una tasa de éxito del 88%. A partir de los resultados obtenidos, se puede concluir que la estrategia de utilizar la inteligencia artificial para la clasificación automática del conocimiento de los profesores de Física es una solución plausible. O sucesso da Inteligência Artificial tem atraído pesquisadores de diversas áreas para o uso de técnicas computacionais em tarefas de extração de conhecimentos de dados não estruturados, como os documentos textuais, apresentando-se como uma solução possível para a classificação de Conhecimentos Especializado de Professores de Física, que consiste em uma ferramenta analítica que descreve os Conhecimentos da Física (PK) e os conhecimentos Didáticos do Conteúdo (PCK), considerada muito importante para auxiliar na identificação e na análise de conhecimentos mobilizados pelos professores em suas práticas de ensino. Porém, a tarefa de identificação e classificação de conhecimentos presentes em documentos textuais apresentam alguns desafios, como: as investigações em documentos textuais é trabalhosa, demorada e envolve mão de obra de pessoas especializadas. Nesse sentido, o objetivo da pesquisa é analisar a eficácia dos algoritmos utilizados na classificação automática de Conhecimentos Especializados de Professores de Física (PTSK) em textos de uma base de dados previamente classificada. O encaminhamento metodológico é de natureza quantitativa, exploratória e aplicada para prever a classe Conhecimentos da Física (PK) ou Conhecimentos Didáticos do Conteúdo (PCK) para caracterizar um conhecimento. Para isso, foram usados dois algoritmos: o doc2vec e o J48 e os resultados foram analisados com base nos resultados alcançados nas métricas de validação. O melhor resultado foi alcançado com o doc2vec, obtendo 88% de taxa de acerto. Com base nos resultados atingidos, pode-se concluir que a estratégia de usar inteligência artificial para a classificação automática de conhecimentos de professores de Física é uma solução plausível.

In the community of bibliometrics, author name ambiguity means that author's name is not a reliable identier for associating academic papers with their authors. Author name ambiguity has been the problem in bibliometrics and service providers like Google Scholar, generating a domain of study call Author Name Disambiguation (AND). Author name ambiguity is often tackled using classication techniques, where labeled papers are provided, and papers are assigned to correct authors according to the paper text and paper citations. When applying classication methods to author name disambiguation, two issues stand out: one is that a paper has multiple views (paper text and citation network). The other is the lack of training data: there are not many papers that are labeled. To cope with these two issues, we propose to use the co-training algorithm in AND. The co-training algorithm uses two views to classify papers iteratively and add the top selected papers into the training pool. We demonstrate that the co-training algorithm outperforms the baseline multi-view classication algorithm. We also experiment with hyper-parameters in the co-training algorithm. The experiment is done on the PubMed dataset, where authors are labeled with ORCID. Papers are represented by two embeddings that are learnt from paper content and paper citation network separately. Baseline classiers for comparison are logistic regression and SVM.

Recently, anyone can easily publish a lot of information in blogs, SNS, and so on. Then, the amount of information existing on the web continues to increase. For that reason, it is not only time consuming to find the information that is needed, it is also important to distinguish lie information, malicious information, or wrong information. In this research, by comparing similarity with past data, we discover whether incorrect information and information that can be subject to criticism exist compared with other information.

With the growth of online information and sudden expansion in the number of electronic documents provided on websites and in electronic libraries, there is difficulty in categorizing text documents. Therefore, a rule-based approach is a solution to this problem, the purpose of this study is to classify documents by using a rule-based. This paper deals with the rule-based approach with the embedding technique for a document to vector (doc2vec) files. An experiment was performed on two data sets Reuters-21578 and the 20 Newsgroups to classify the top ten categories of these data sets by using a document to vector rule-based (D2vecRule). Finally, this method provided us a good classification result according to the F-measures and implementation time metrics. In conclusion, it was observed that our algorithm document to vector rule-based (D2vecRule) was good when compared with other algorithms such as JRip, One R, and ZeroR applied to the same Reuters-21578 dataset.

Durante a fase de manutenção de software, os processos de localização de características e bugs de software exercem um relevante papel na recuperação da rastreabilidade entre descrições expressas em linguagem natural e trechos de código fonte. Entretanto, ferramentas automatizadas propostas para esta finalidade não tem apresentado acurácia que possa ser considerada como definitiva. Neste sentido, o objetivo principal deste trabalho foi o aprimoramento de vetores de parágrafos, produzidos por uma rede neural Artificial empregada no algoritmo Doc2vec, e aplicados ao processo de localização de características e de bugs de software. Assim, melhor definidos, estes vetores puderam ser usados no cálculo de similaridades entre descrições de características e métodos do código fonte. Da mesma forma, a similaridade entre bugs e classes do código fonte pôde ser aprimorada. Para atingir este objetivo, foi aplicada uma taxa de aprendizado cíclica além da alteração da função de custo da rede neural que estavam associados ao algoritmo Doc2vec. Outras abordagens utilizadas reforçaram os ganhos em acurácia para o algoritmo Doc2vec a partir da combinação de rankings obtidos em ferramentas que expressam o estado da arte em Engenharia de Software. Um conjunto de abordagens que não apresentaram resultados satisfatórios também foram experimentadas e são descritas neste trabalho. Dentre estas podem ser citadas a melhoria da qualidade dos vetores representativos de métodos e classes do código fonte a partir de outros sistemas escritos na linguagem de programação Java; uso de pesos sintáticos diferenciados para termos que participam da composição de vetores relacionados às classes e métodos; e uso do algoritmo fastText para geração de vetores. Para cada abordagem proposta, ponderações foram feitas no sentido de se avaliar sua efetividade de uso em contextos de localização de bugs e características. Em resumo, com uma combinação das melhorias propostas para a rede neural artificial, a acurácia do estado da arte para a tarefa de localização de características pôde ser superada. Além disso, também foi possível melhorar a localização de bugs realizada a partir do uso do algoritmo Doc2vec bem como determinar quais hiperparâmetros eram mais influentes para melhoria da acurácia. Throughout a software maintenance phase, the processes of feature and bug location play an important role in the retrieval of traceability between natural language and source code. However, existing automated tools, which were designed to perform these tasks, have not yet presented the desirable accuracy expected as a definitive outcome. In this sense, this work main objective seeks to attend to the improvement of paragraph vectors, produced by artificial neural network in the Doc2vec algorithm, and applied into the processes of feature and software bug location. By improving the vector quality, these were used in the calculation of similarities found between feature descriptions and source code methods. Also, improvements in identifying bugs similarity and source code classes were observed. In order to reach this result, a cyclic learning rate was applied to the task in addition to customizing neural network loss function, which were associated with the Doc2vec algorithm. Furthermore, other approaches used have also demonstrated the gains in accuracy for the Doc2vec algorithm, provided by the combination of rankings obtained in tools that express the state of the art in Software Engineering. Yet, a set of other minor approaches were applied to this work, such as: improvement in the quality of representative vectors from source code methods, departing from other systems source code; usage of word syntactic influence within paragraphs extracted from source code; and usage of fastText algorithm to generate paragraph vectors. For each proposed approach, considerations were made in order to evaluate its effectiveness in feature and bug location tasks. In sum, the improvements proposed by this work to artificial neural network allowed to improve state of the art work in feature location tasks. Moreover, improvements in bug location were made by using the Doc2vec algorithm, and most influential hyper parameters were defined to improve accuracy. Tese (Doutorado)

With the growing popularity of machine learning algorithms, dramatic advances have been made for code completion, and specifically method-call completion. These advances were also possible thanks to the availability of large code repositories to learn from and to the well-defined boundaries of the method-call completion problem. This is, however, not the case for design completion, where model repositories are scarce and the space of possibilities for design completion is theoretically infinite.

近年，動画ではYouTube やニコニコ動画が，小説では小説家になろうといった利用者投稿型のCGM (Consumer Generated Media) サイトが人気である． CGM サイトには毎日多数のコンテンツが投稿されており，また膨大な利用者がコンテンツを閲覧している． 現在，CGM サイトへの投稿コンテンツの画一化が懸念されている．既に見たことのあるようなコンテンツや派生コンテンツの増加が感じられる．我々は今回，「小説家になろう」を対象に，Cos 類似度の総和を調べることで小説家になろうの画一化について定量的に評価をおこなった．次に，偏りのあるジャンルを特定するために，クラスタリングによるオンライン小説の多様性動向を調査する．各小説のあらすじをDoc2Vec を用いてベクトルで表現し，最後に，小説のベクトル群をX-means でクラスタリングする． クラスタ数やクラスタの要素数を分析することで，投稿小説の多様性動向の分析を行う．, In recent years, CGM (Consumer Generated Media) site, such as YouTube.com and Nicovideo.jp for movies, syosetu.com for novels, become very popular. Many contents are posted on those CGM sites everyday, and a huge number of users are browsing the contents. Now a day, some bloggers mentioned that similar contents are well posted to the CGM site. It is felt that contents and derivative contents which have already seen have increased. We proposed contents diversity metric, that is the sum of cosine similarities of contents. We applied our metric to contents of nicovideo.jp and syosetu.com, and found that contents diversity was decrease in both CGM. To identify bias of contents, we investigate the diversity trend of online novel using clustering. We extract synopsis part from each novels, and vectorize them by Doc2Vec. After that, we clustered the vectors using X-means clustering. We report our method and results of clustering.

International audience; The software sustaining phase is one of the essential stages of the software development life cycle. At this stage, customers can contact support and software engineers to request a resolution of any problem they meet during software utilization including questions of how to operate with software, where to find the information about particular functions and other relevant questions on software product. The work is devoted to research in the field of software sustainment automation. The distinctive feature of the article is the suggested semantic documentation search approach with the Doc2Vec machine learning algorithm, which allows automating automate customer requests resolution. Proposed semantic search is performed on documentation files, like PDFs, Microsoft Office documents, wiki pages and other text files with relevant information about the product. Documentation files, including page numbers, that have the closest semantic similarity to the textual description of an unresolved customer request, help the developer resolve the incoming request more efficiently and in a shorter time. The proposed approach is implemented in a software tool to automate the analysis of unresolved customer requests and provide recommendations to help in solving each of those requests. The results show the advantages of using the tool in the process of software product support.

Multimodal machine learning is a subfield of machine learning that aims to relate data from different modalities, such as texts and images. One of the many applications that could be built upon this technique is an image retrieval system that, given a text query, retrieves suitable images from a database. In this thesis, a retrieval system based on canonical correlation is used to suggest images for news articles. Different dense text representations produced by Word2vec and Doc2vec, and image representations produced by pre-trained convolutional neural networks are explored to find out how they affect the suggestions. Which part of an article is best suited as a query to the system is also studied. Also, experiments are carried out to determine if an article's date of publication can be used to improve the suggestions. The results show that Word2vec outperforms Doc2vec in the task, which indicates that the meaning of article texts are not as important as the individual words they consist of. Furthermore, the queries are improved by rewarding words that are particularly significant.

Este Trabajo Fin de Grado plantea el diseño y la implementación de un sistema de clasificación de perfiles. Dicho sistema tiene como objetivo la obtención de una lista de perfiles similares para un perfil dado de una conocida red social. Este sistema se valdrá de técnicas de procesamiento del lenguaje natural para el análisis de los perfiles, obteniendo características semánticas y estilísticas de ellos, y será desarrollado en Python. Durante el desarrollo de este TFG se explicarán en detalle los pasos seguidos para el diseño del sistema: la obtención de datos, el almacenamiento de éstos, los modelos de procesamiento del lenguaje natural utilizados, los resultados obtenidos para cada modelo y el análisis de todos los resultados