Search

As researchers use computational methods to study complex social behaviors at scale, the validity of this computational social science depends on the integrity of the data. On July 2, 2015, Jason Baumgartner published a dataset advertised to include “every publicly available Reddit comment” which was quickly shared on Bittorrent and the Internet Archive. This data quickly became the basis of many academic papers on topics including machine learning, social behavior, politics, breaking news, and hate speech. We have discovered substantial gaps and limitations in this dataset which may contribute to bias in the findings of that research. In this paper, we document the dataset, substantial missing observations in the dataset, and the risks to research validity from those gaps. In summary, we identify strong risks to research that considers user histories or network analysis, moderate risks to research that compares counts of participation, and lesser risk to machine learning research that avoids making representative claims about behavior and participation on Reddit. [ABSTRACT FROM AUTHOR]

This study compares six metrics commonly used to identify influential players in two of Canada’s largest political Twitter communities based on the users, and ranking order of users, identified by each metric. All tweets containing the hashtag #CPC, representing the Conservative Party of Canada (government), and #NDP, representing the New Democratic Party of Canada (official opposition), were collected over a 2-week period in March 2013 and a follower network graph was created. Social network analysis and content analysis were employed to identify influentials. Kendall’s τ was the primary quantitative measure for comparison. Categorization of Twitter profiles of users found within the top 20 most influential lists, according to each metric of influence, made up the qualitative portion of analysis. The authors find that measures of centrality—indegree and eigenvector centrality—identify the traditional political elite (media outlets, journalists, politicians) as influential, whereas measures considering the quality of messages and interactions provide a different group of influencers, including political commentators and bloggers. Finally, the authors investigate the possibility of using the local clustering coefficient of nodes to identify those who are both aware of the traditional elite and embedded in tightly knit communities, similar to the “opinion leader,” described in the Two-Step Flow Hypothesis. [ABSTRACT FROM PUBLISHER]

The movements of ideas and content between locations and languages are unquestionably crucial concerns to researchers of the information age, and Twitter has emerged as a central, global platform on which hundreds of millions of people share knowledge and information. A variety of research has attempted to harvest locational and linguistic metadata from tweets to understand important questions related to the 300 million tweets that flow through the platform each day. Much of this work is carried out with only limited understandings of how best to work with the spatial and linguistic contexts in which the information was produced, however. Furthermore, standard, well-accepted practices have yet to emerge. As such, this article studies the reliability of key methods used to determine language and location of content in Twitter. It compares three automated language identification packages to Twitter's user interface language setting and to a human coding of languages to identify common sources of disagreement. The article also demonstrates that in many cases user-entered profile locations differ from the physical locations from which users are actually tweeting. As such, these open-ended, user-generated profile locations cannot be used as useful proxies for the physical locations from which information is published to Twitter. [ABSTRACT FROM AUTHOR]