1. No Longer Lost in Translation: Evidence that Google Translate Works for Comparative Bag-of-Words Text Applications
- Author
-
de Vries, Erik, Schoonvelde, Martijn, Schumacher, Gijs, Biotechnology, Research Centre Arts in Society (AiS), Political Economy and Transnational Governance (PETGOV, AISSR, FMG), Challenges to Democratic Representation (AISSR, FMG), Multi-layered governance in EUrope and beyond (MLG), and Political Science and Public Administration
- Subjects
Topic model ,Text corpus ,Sociology and Political Science ,Machine translation ,Computer science ,LDA ,Political Science ,FOS: Political science ,bag-of-words models ,0211 other engineering and technologies ,Document-term matrix ,Models and Methods ,050801 communication & media studies ,02 engineering and technology ,Social and Behavioral Sciences ,computer.software_genre ,0508 media and communications ,Text mining ,Comparative Politics ,050602 political science & public administration ,Set (psychology) ,021110 strategic, defence & security studies ,business.industry ,05 social sciences ,Comparative politics ,statistical analysis of texts ,Google Translate ,0506 political science ,Bag-of-words model ,automated content analysis ,Political Science and International Relations ,Artificial intelligence ,business ,computer ,Natural language processing - Abstract
Automated text analysis allows researchers to analyze large quantities of text. Yet, comparative researchers are presented with a big challenge: across countries people speak different languages. To address this issue, some analysts have suggested using Google Translate to convert all texts into English before starting the analysis (Lucas et al. 2015). But in doing so, do we get lost in translation? This paper evaluates the usefulness of machine translation for bag-of-words models—such as topic models. We use the europarl dataset and compare term-document matrices (TDMs) as well as topic model results from gold standard translated text and machine-translated text. We evaluate results at both the document and the corpus level. We first find TDMs for both text corpora to be highly similar, with minor differences across languages. What is more, we find considerable overlap in the set of features generated from human-translated and machine-translated texts. With regard to LDA topic models, we find topical prevalence and topical content to be highly similar with again only small differences across languages. We conclude that Google Translate is a useful tool for comparative researchers when using bag-of-words text models.
- Published
- 2018
- Full Text
- View/download PDF