Search

Augmenting survey data with digital traces is a promising direction for combining the advantages of active and passive data collection. However, extracting interpretable measurements from digital traces for social science research is challenging. In this study, we demonstrate how to obtain measurements of news media consumption from survey respondents’ web browsing data using Bidirectional Encoder Representations from Transformers, a powerful natural language processing algorithm that estimates contextual word embeddings from text data. Our approach is particularly relevant for political scientists and communication researchers studying exposure to online news content but can easily be adapted to projects in other disciplines working with similar data sets.

Online panel surveys are often criticized for their inability to cover the offline population, potentially resulting in coverage error. Previous research has demonstrated that non-internet users in fact differ from online individuals on several sociodemographic characteristics. In attempts to reduce coverage error due to missing the offline population, several probability-based online panels equip offline households with an internet connection and a simple computer or tablet. However, the question remains whether the recruitment of offline individuals for an online panel leads to substantial changes in survey estimates. That is, it is unclear whether estimates derived from the survey data are affected by the differences between the groups of online and offline individuals. Against this background, we investigate how the inclusion of the previously offline population into the German Internet Panel affects various survey estimates such as voting behavior and social engagement. Overall, we find little evidence for the claim that equipping otherwise offline individuals with online access affects the estimates derived from previously online individuals only.

The U.S. Consumer Expenditure Interview Survey asks many filter questions to identify the items that households purchase. Each reported purchase triggers follow-up questions about the amount spent and other details. We test the hypothesis that respondents learn how the questionnaire is structured and underreport purchases in later waves to reduce the length of the interview. We analyze data from 10,416 four-wave respondents over two years of data collection. We find no evidence of decreasing data quality over time; instead, panel respondents tend to give higher quality responses in later waves. The results also hold for a larger set of two-wave respondents.

Filter questions are used to administer follow-up questions to eligible respondents while allowing respondents who are not eligible to skip those questions. Filter questions can be asked in either the interleafed or the grouped formats. In the interleafed format, the follow-ups are asked immediately after the filter question; in the grouped format, follow-ups are asked after the filter question block. Underreporting can occur in the interleafed format due to respondents’ desire to reduce the burden of the survey. This phenomenon is called motivated misreporting. Because smartphone surveys are more burdensome than web surveys completed on a computer or laptop, due to the smaller screen size, longer page loading times, and more distraction, we expect that motivated misreporting is more pronounced on smartphones. Furthermore, we expect that misreporting occurs not only in the filter questions themselves but also extends to data quality in the follow-up questions. We randomly assigned 3,517 respondents of a German online access panel to either the PC or the smartphone. Our results show that while both PC and smartphone respondents trigger fewer filter questions in the interleafed format than the grouped format, we did not find differences between PC and smartphone respondents regarding the number of triggered filter questions. However, smartphone respondents provide lower data quality in the follow-up questions, especially in the grouped format. We conclude with recommendations for web survey designers who intend to incorporate smartphone respondents in their surveys.

Social media are becoming more popular as a source of data for social science researchers. These data are plentiful and offer the potential to answer new research questions at smaller geographies and for rarer subpopulations. When deciding whether to use data from social media, it is useful to learn as much as possible about the data and its source. Social media data have properties quite different from those with which many social scientists are used to working, so the assumptions often used to plan and manage a project may no longer hold. For example, social media data are so large that they may not be able to be processed on a single machine; they are in file formats with which many researchers are unfamiliar, and they require a level of data transformation and processing that has rarely been required when using more traditional data sources (e.g., survey data). Unfortunately, this type of information is often not obvious ahead of time as much of this knowledge is gained through word-of-mouth and experience. In this article, we attempt to document several challenges and opportunities encountered when working with Reddit, the self-proclaimed “front page of the Internet” and popular social media site. Specifically, we provide descriptive information about the Reddit site and its users, tips for using organic data from Reddit for social science research, some ideas for conducting a survey on Reddit, and lessons learned in merging survey responses with Reddit posts. While this article is specific to Reddit, researchers may also view it as a list of the type of information one may seek to acquire prior to conducting a project that uses any type of social media data.

Human perceptions of fairness in (semi-)automated decision-making (ADM) constitute a crucial building block toward developing human-centered ADM solutions. However, measuring fairness perceptions is challenging because various context and design characteristics of ADM systems need to be disentangled. Particularly, ADM applications need to use the right degree of automation and granularity of data input to achieve efficiency and public acceptance. We present results from a large-scale vignette experiment that assessed fairness perceptions and the acceptability of ADM systems. The experiment varied context and design dimensions, with an emphasis on who makes the final decision. We show that automated recommendations in combination with a final human decider are perceived as fair as decisions made by a dominant human decider and as fairer than decisions made only by an algorithm. Our results shed light on the context dependence of fairness assessments and show that semi-automation of decision-making processes is often desirable.

A major concern arising from ubiquitous tracking of individuals’ online activity is that algorithms may be trained to predict personal sensitive information, even for users who do not wish to reveal such information. Although previous research has shown that digital trace data can accurately predict sociodemographic characteristics, little is known about the potentials of such data to predict sensitive outcomes. Against this background, we investigate in this article whether we can accurately predict voting behavior, which is considered personal sensitive information in Germany and subject to strict privacy regulations. Using records of web browsing and mobile device usage of about 2,000 online users eligible to vote in the 2017 German federal election combined with survey data from the same individuals, we find that online activities do not predict (self-reported) voting well in this population. These findings add to the debate about users’ limited control over (inaccurate) personal information flows.

Many surveys aim to achieve high response rates to keep bias due to nonresponse low. However, research has shown that the relationship between the nonresponse rate and nonresponse bias is small. In fact, high response rates may lead to measurement error, if respondents with low response propensities provide survey responses of low quality. In this paper, we explore the relationship between response propensity and measurement error, specifically, motivated misreporting, the tendency to give inaccurate answers to speed through an interview. Using data from four surveys conducted in several countries and modes, we analyze whether motivated misreporting is worse among those respondents who were the least likely to respond to the survey. Contrary to the prediction of our theoretical model, we find only limited evidence that reluctant respondents are more likely to misreport.

In times of increasing digitization, the protection of individual data privacy becomes more important than ever before. To craft privacy policies that do not only meet legal requirements, but also address the public’s concerns, understanding individual privacy attitudes is key. Previous research suggests that privacy attitudes depend on a set of parameters related to the data type, data collector and other situational characteristics. However, the importance of single situational characteristics may possibly be altered by changes in the environment. This circumstance becomes apparent and even more important with the outbreak of the COVID-19 pandemic. The pandemic constitutes an exceptional situation in which individuals may be willing to transmit more personal data than usual for the sake of public health and safety. In this study, we analyze how attitudes towards acceptable data use shift in times of crisis. In July 2019, long before the pandemic, we conducted a survey in Germany in which we measured respondents’ acceptance of the collection and use of health data for public health purposes including preventing the spread of a virus. As the pandemic set in, we replicated this survey in the spring of 2020 to investigate changes in respondents’ willingness to share data for public health purposes in response to the crisis. Using data from 3,502 respondents, we demonstrate and quantify the shift in privacy attitudes with situational characteristics. Public acceptance of the use of personal health data to combat an infectious disease outbreak increased notably, while acceptance of personal data use in several other scenarios barely changed over time. We conclude that policymakers need to carefully consider the intended purpose of and appropriate limitations on data use for public health and argue that the design of data collection tools should meet both public health and privacy concerns.

Panel conditioning refers to the phenomenon whereby respondents’ attitudes, behaviour, reporting of behaviour and/or knowledge are changed by repeated participation in a panel survey. Uncovering such effects, however, is difficult due to three major methodological challenges. First, researchers need to disentangle changes in behaviour and attitudes from changes in the reporting of behaviour and attitudes as panel conditioning may result in both, even at the same time and in opposite directions. Second, the identification of the causal effect of panel participation on the various forms of change mentioned above is complicated as it requires comparisons of panel respondents with control groups of people who have not been interviewed before. Third, other sources of error in (panel) surveys may easily be mistaken for panel conditioning if not properly accounted for. Such error sources are panel attrition, mode effects, and interviewer effects. In this chapter the challenges mentioned above are reviewed in detail and a methodological framework for the analysis of panel conditioning effects is provided by identifying the strengths and weaknesses of the various designs that researchers have developed to address the challenges. The chapter concludes with a discussion of a future research agenda on panel conditioning effects in longitudinal surveys.

Background The COVID-19 pandemic is the greatest public health crisis of the last 100 years. Countries have responded with various levels of lockdown to save lives and stop health systems from being overwhelmed. At the same time, lockdowns entail large socioeconomic costs. One exit strategy under consideration is a mobile phone app that traces the close contacts of those infected with COVID-19. Recent research has demonstrated the theoretical effectiveness of this solution in different disease settings. However, concerns have been raised about such apps because of the potential privacy implications. This could limit the acceptability of app-based contact tracing in the general population. As the effectiveness of this approach increases strongly with app uptake, it is crucial to understand public support for this intervention. Objective The objective of this study is to investigate the user acceptability of a contact-tracing app in five countries hit by the pandemic. Methods We conducted a largescale, multicountry study (N=5995) to measure public support for the digital contact tracing of COVID-19 infections. We ran anonymous online surveys in France, Germany, Italy, the United Kingdom, and the United States. We measured intentions to use a contact-tracing app across different installation regimes (voluntary installation vs automatic installation by mobile phone providers) and studied how these intentions vary across individuals and countries. Results We found strong support for the app under both regimes, in all countries, across all subgroups of the population, and irrespective of regional-level COVID-19 mortality rates. We investigated the main factors that may hinder or facilitate uptake and found that concerns about cybersecurity and privacy, together with a lack of trust in the government, are the main barriers to adoption. Conclusions Epidemiological evidence shows that app-based contact tracing can suppress the spread of COVID-19 if a high enough proportion of the population uses the app and that it can still reduce the number of infections if uptake is moderate. Our findings show that the willingness to install the app is very high. The available evidence suggests that app-based contact tracing may be a viable approach to control the diffusion of COVID-19.

Summary Panel survey participation can bring about unintended changes in respondents’ behaviour and/or their reporting of behaviour. Using administrative data linked to a large panel survey, we analyse whether the survey brings about changes in respondents’ labour market behaviour. We estimate the causal effect of panel participation on the take-up of federal labour market programmes by using instrumental variables. Results show that panel survey participation leads to an increase in respondents’ take-up of these measures. These results suggest that panel survey participation not only affects the reporting of behaviour, as previous studies have demonstrated, but can also alter respondents’ actual behaviour.

This study explores voters’ populist alternative news use during (different types of) democratic elections and investigates starting points for preventing potentially harmful effects. We draw from two combined data sets of web-tracking and survey data which were collected during the 2017 German Bundestag campaign (1523 participants) and the 2019 European Parliamentary election campaign in Germany (1009 participants). Results indicate that while populist alternative news outlets drew more interest during the first-order election campaign, they reached only 16.5% of users even then. Moreover, most users visited their websites rather seldom. Nonetheless, our data suggest that alternative news exposure is strongly linked to voting for (right-wing) populist parties. Regarding the origins of exposure, our analyses punctuate the role of platforms in referring users to populist alternative news. About 40% of website visits originated from Facebook alone in both data sets and another third of visits from search engines. This raises questions about algorithmic accountability.

We find rotation group bias in reports of spending in the U.S. Consumer Expenditure Survey. Contrary to our expectations, the more waves respondents complete, the higher the quality of their responses. Respondents become more likely to report the amount of money spent on purchases and less likely to report rounded amounts. There is no change over waves in the number of purchases reported or the average amount of money spent on the purchases.