Search

Paradata are a valuable source of information to better understand the response process and to determine how the data collection processes can be improved. Currently, a debate is ongoing in academia and in the professional associations of market, opinion, and social research concerning the need to inform respondents about the collection and use of their web paradata. Unfortunately, research is lacking on the design of informed consent for web paradata use and its possible consequences for respondents’ willingness to share their paradata, completion of a survey, and response behavior. The aim of this study is to provide guidance to the survey practitioner on the optimal design of informed consent for web paradata use based on experimental evidence.

Knowledge questions frequently are used in survey research to measure respondents’ topic-related cognitive ability and memory. However, in self-administered surveys, respondents can search external sources for additional information to answer a knowledge question correctly. In this case, the knowledge question measures accessible and procedural memory. Depending on what the knowledge question aims at, the validity of this measure is limited. Thus, in this study, we conducted three experiments using a web survey to investigate the effects of task difficulty, respondents’ ability, and respondents’ motivation on the likelihood of searching external sources for additional information as a form of over-optimizing response behavior when answering knowledge questions. We found that the respondents who are highly educated and more interested in a survey are more likely to invest additional efforts to answer knowledge questions correctly. Most importantly, our data showed that for these respondents, a more difficult question design further increases the likelihood of over-optimizing response behavior.

The collection and use of paradata is gaining in importance, especially in web surveys. From a research ethics’ perspective, respondents should be asked for their consent to the collection and use of web paradata. In this context, a positive attitude toward paradata use has been deemed to be a prerequisite for respondents’ willingness to share their paradata. The present study aimed to identify factors affecting respondents’ attitudes toward paradata use. Our findings revealed that adequately informing survey respondents about what paradata are and why they are used was an important determinant of their attitudes toward paradata use. Moreover, we found that respondents with a positive attitude toward the survey were more likely to have a favorable opinion of paradata use. Our findings suggest that a thorough understanding of the factors that contribute to a positive attitude toward paradata use provides the basis for improved paradata consent procedures, which in turn will increase rates of consent to paradata use and help attenuate the risk of consent bias in web surveys.

Identifying inattentive respondents in self-administered surveys is a challenging goal for survey researchers. Instructed response items (IRIs) provide a measure for inattentiveness in grid questions that is easy to implement. The present article adds to the sparse research on the use and implementation of attention checks by addressing three research objectives. In a first study, we provide evidence that IRIs identify respondents who show an elevated use of straightlining, speeding, item nonresponse, inconsistent answers, and implausible statements throughout a survey. Excluding inattentive respondents, however, did not alter the results of substantive analyses. Our second study suggests that respondents’ inattentiveness partially changes as the context in which they complete the survey changes. In a third study, we present experimental evidence that a mere exposure to an IRI does not negatively or positively affect response behavior within a survey. A critical discussion on using IRI attention checks concludes this article.

Survey researchers are often confronted with the question of how long to set the length of the field period. Longer fielding time might lead to greater participation yet requires survey managers to devote more of their time to data collection efforts. With the aim of facilitating the decision about the length of the field period, we investigated whether a longer fielding time reduces the risk of nonresponse bias to judge whether field periods can be ended earlier without endangering the performance of the survey. By using data from six waves of a probability-based mixed-mode (online and mail) panel of the German population, we analyzed whether the risk of nonresponse bias decreases over the field period by investigating how day-by-day coefficients of variation develop during the field period. We then determined the optimal cut-off points for each mode after which data collection can be terminated without increasing the risk of nonresponse bias and found that the optimal cut-off points differ by mode. Our study complements prior research by shifting the perspective in the investigation of the risk of nonresponse bias to panel data as well as to mixed-mode surveys, in particular. Our proposed method of using coefficients of variation to assess whether the risk of nonresponse bias decreases significantly with each additional day of fieldwork can aid survey practitioners in finding the optimal field period for their mixed-mode surveys.

The European Values Study (EVS) was first conducted in 1981 and then repeated in 1990, 1999, 2008, and 2017, with the aim of providing researchers with data to investigate whether European individual and social values are changing and to what degree. The EVS is traditionally carried out as a probability-based face-to-face survey that takes around 1 hour to complete. In recent years, large-scale population surveys such as the EVS have been challenged by decreasing response rates and increasing survey costs. In the light of these challenges, six countries that participated in the last wave of the EVS tested the application of self-administered mixed-modes (Denmark, Finland, Germany, Iceland, the Netherlands, and Switzerland). With the present data brief, we will introduce researchers to the latest wave of the EVS, the implemented mode experiments, and the EVS data releases. In our view, it is pivotal for data use in substantive research to make the reasoning behind design changes and country-specific implementations transparent as well as to highlight new research opportunities.

Propensity score weighting often is used to correct for attrition biases in panel surveys. While methodological literature exists on the logic of propensity score weighting and its practical applic...

Mobile coverage recently has reached an all-time high, and in most countries, high-speed Internet connections are widely available. Due to technological development, smartphones and tablets have become increasingly popular. Accordingly, we have observed an increasing use of mobile devices to complete web surveys and, hence, survey methodologists have shifted their attention to the challenges that stem from this development. The present study investigated whether the growing use of smartphones has decreased how systematically this choice of device varies between groups of respondents (i.e., how selective smartphone usage for completing web surveys is). We collected a data set of 18,520 respondents from 18 web surveys that were fielded in Germany between 2012 and 2016. Based on these data, we show that while the use of smartphones to complete web surveys has considerably increased over time, selectivity with respect to using this device has remained stable.

To reduce nonresponse bias in surveys, it has been suggested that researchers allocate additional fieldwork efforts to cases with low estimated response propensity. If these efforts are successful, nonresponse bias may be reduced by changing the variance of response propensities and hence the covariance between response propensities and variables of interest. The present study provides experimental evidence about how to intervene on low-propensity cases through interviewer selection and allocation in telephone surveys. The results suggest that this intervention can successfully increase cooperation rates for the low-propensity group and reduce nonresponse bias without increasing field costs.

In this article, we present the results of three question design experiments on inter-item correlations, which tested a grid design against a single-item design. The first and second experiments examined the inter-item correlations of a set with five and seven items, respectively, and the third experiment examined the impact of the question design on inter-item correlations of follow-up questions. We found that the inter-item correlations in grid designs were higher for both the set of five and seven items than for questions in a single-item design. Additionally, the inter-item correlations of the set of follow-up questions were higher if the preceding set of questions was presented in a grid. According to the findings of the present research, respondents seem to apply the interpretative near means related heuristic when answering grid questions, and also when answering follow-up questions preceeded by grid questions.

Survey research is still confronted by a trend of increasing nonresponse rates. In this context, several methodological advances have been made to stimulate participation and avoid bias. Yet, despite the growing number of tools and methods to deal with nonresponse, little is known about whether nonresponse biases show similar trends as nonresponse rates and what mechanisms (if any) drive changes in bias. Our article focuses on biases in cohort distributions in the U.S. and German general social surveys from 1980 to 2012 as one of the key variables in the social sciences. To supplement our cross-national comparison of these trends, we decompose changes into within-cohort change (WCC) and between-cohort change. We find that biases in cohort distributions have remained relatively stable and at a relatively low level in both countries. Furthermore, WCC (i.e., survey climate) accounts for the major part of the change in nonresponse bias.

Our study explores the adoption of Facebook and Twitter by candidates in the 2013 German Federal elections. Utilizing data from the German Longitudinal Election Study candidate survey fused with data gathered on the Twitter and Facebook use of candidates, we draw a clear distinction between Facebook and Twitter. We show that adoption of both channels is primarily driven by two factors: party and money. But the impact of each plays out differently for Facebook and Twitter. While the influence of money is homogenous for Facebook and Twitter with the more resources candidates have, the more likely they are to adopt, the effect is stronger for Facebook. Conversely, a party's impact on adoption is heterogeneous across channels, a pattern we suggest is driven by the different audiences Facebook and Twitter attract. We also find candidates' personality traits only correlate with Twitter adoption, but their impact is minimal. Our findings demonstrate that social media adoption by politicians is far from homogenous, and that there is a need to differentiate social media channels from one another when exploring motivations for their use.

This article addresses the questions of whether paradata can help us to improve the models of panel attrition and whether paradata can improve the effectiveness of propensity score weights with respect to reducing attrition biases. The main advantage of paradata is that it is collected as a by-product of the survey process. However, it is still an open question which paradata can be used to model attrition and to what extent these paradata are correlated with the variables of interest. Our analysis used data from a seven-wave web-based panel survey that had been supplemented by three cross-sectional surveys. This split panel design allowed us to assess the magnitude of attrition bias for a large number of substantive variables. Furthermore, this design enabled us to analyze in detail the effectiveness of propensity score weights. Our results showed that some paradata (e.g., response times and participation history) improved the prediction of panel attrition, whereas others did not. In addition, not all the paradata that increased the model fit resulted in weights that effectively reduced bias. These findings highlight the importance of selecting paradata that are linked to both the survey response process and the variables of interest. This article provides a first contribution to this challenge.