Search

Numerous surveys link interview data to administrative records, conditional on respondent consent, in order to explore new and innovative research questions. Optimizing the linkage consent rate is a critical step toward realizing the scientific advantages of record linkage and minimizing the risk of linkage consent bias. Linkage consent rates have been shown to be particularly sensitive to certain design features, such as where the consent question is placed in the questionnaire and how the question is framed. However, the interaction of these design features and their relative contributions to the linkage consent rate have never been jointly studied, raising the practical question of which design feature (or combination of features) should be prioritized from a consent rate perspective. We address this knowledge gap by reporting the results of a placement and framing experiment embedded within separate telephone and Web surveys. We find a significant interaction between placement and framing of the linkage consent question on the consent rate. The effect of placement was larger than the effect of framing in both surveys, and the effect of framing was only evident in the Web survey when the consent question was placed at the end of the questionnaire. Both design features had negligible impact on linkage consent bias for a series of administrative variables available for consenters and non-consenters. We conclude this research note with guidance on the optimal administration of the linkage consent question.

Recent years have seen an increase in the amount of statistics describing different phenomena based on “Big Data.” This term includes data characterized not only by their large volume, but also by their variety and velocity, the organic way in which they are created, and the new types of processes needed to analyze them and make inference from them. The change in the nature of the new types of data, their availability, and the way in which they are collected and disseminated is fundamental. This change constitutes a paradigm shift for survey research. There is great potential in Big Data, but there are some fundamental challenges that have to be resolved before its full potential can be realized. This report provides examples of different types of Big Data and their potential for survey research; it also describes the Big Data process, discusses its main challenges, and considers solutions and research needs.

Survey methodologists are searching for covariates to use in nonresponse adjustment models, ultimately hoping to find variables that are highly correlated with both the outcomes of interest and the propensity to respond. These covariates can come from auxiliary data that provide information on both respondents and nonrespondents. Two such types of auxiliary data are interviewer observations (a form of paradata) and commercially available data on small areas or households. Interviewer observations intended for use in nonresponse adjustment can be specifically designed to match the outcome variables of interest, while commercial data provide a broad set of small area descriptors that may be correlated with multiple outcomes. This analysis examines these two data sources to determine which is more predictive of the outcomes of interest for a particular survey, thereby fulfilling one of the criteria for a good adjustment variable. The outcomes of interest in this analysis are self-reports of household income and receipt of unemployment benefits from a survey of labor market participation. The findings suggest that at this point in time, compared to commercial data, interviewer observations are better at predicting these outcomes, particularly in the subpopulation that the survey targets. Therefore, the observations share more (accurate) information with the true value, making them better for adjustment on this dimension. The results will inform the work of both researchers wishing to improve their nonresponse adjustments and survey managers looking to make better use of their survey budget.

Survey data-collection costs are steadily increasing, forcing government agencies and survey organizations to consider more costeffective methods of acquiring high-quality information from respondents. Web surveys and administrative data linkage are two approaches often considered to help offset rising costs; however, these two approaches are rarely used jointly in the same survey. Consequently, little is known about how one should ask for linkage consent in a web survey in order to maximize consent rates and minimize the risk of linkage bias. The lack of an interviewer, who is often responsible for explaining the rationale and potential benefits associated with the proposed linkage, poses challenges for phrasing the consent request in a way that successfully describes to web survey respondents the advantages of linkage. To investigate this issue, we conducted a wording experiment that mentioned a particular benefit of linkage for web survey respondents: reduced questionnaire length and overall time savings. We found that respondents who received the benefit wording were more likely to provide linkage consent than respondents who received a more neutral wording.

Survey researchers are increasingly interested in obtaining as much auxiliary information on all sample units as possible, given trends of declining response rates in surveys of all formats worldwide. Surveys conducted using in-person (or “face-to-face”) interviews therefore often request that interviewers collect observations on key features of selected households, given that interviewers are the eyes and ears of the survey organization out in the field. Unfortunately, recent studies have suggested that these observations are prone to error, and that the errors may decrease the effectiveness of nonresponse adjustments based in part on the observations. In addition, no studies have investigated the strategies being used by interviewers in the field when they are making these observations, or addressed the question of whether certain observational strategies improve the accuracy of the observations. This paper examines the associations of observational strategies used by field interviewers collecting face-to-face interviews from a large area probability sample with the accuracy of observations collected by those interviewers. A qualitative analysis shows that certain strategies are in fact associated with increased accuracy of the observations, and recommendations for future practice are presented.

As survey researchers have begun exploiting paradata—for example, for the correction of nonresponse bias—the quality of these data has come into question. Inaccurate information is likely to affect the resulting statistics and conclusions drawn from such data. This paper focuses on one type of paradata, observations made by interviewers during the data-collection process, and assesses the quality of these observations by examining their measurement error properties. The analysis uses the UK Census Nonresponse Link Study, which links interviewers’ observations collected on six major UK surveys with Census data. Comparing five interviewer observations with self-reports from the Census, the accuracy of the observations for both respondents and nonrespondents to the surveys is evaluated. A multilevel modeling approach is used to explore under which conditions the interviewers’ observations match the reports on the Census forms, accounting for the clustering of sample members within interviewers and areas. The analysis finds that the overall percent agreement between the observations and the Census is generally high, ranging from 87 to 98 percent. The type of housing structure and the final result code are significantly associated with measurement error. For four of the five observations, there is evidence that the interviewer significantly influences the level of measurement error, even after controlling for household, interviewer, and area characteristics. The results presented here will inform future analyses assessing the quality of interviewers’ observations.

Linking paradata from a survey on labor market participation to administrative data, this article has a unique opportunity to examine nonresponse and measurement error jointly and provide estimates for bias, variance, and mean square error. We find that increased contact attempts resulted in significant reductions in nonresponse bias. Measure- ment error increased somewhat with increased effort, though total bias was nevertheless reduced. One of the key survey indicators revealed an interesting case of counteracting effects of nonresponse and measure- ment error whereby increased effort to recruit nonrespondents led to an increase in mean square error with increased level of effort despite a reduction in nonresponse bias and constant measurement error.

Although it is well established that self-administered ques- tionnaires tend to yield fewer reports in the socially desirable direction than do interviewer-administered questionnaires, less is known about whether different modes of self-administration vary in their effects on socially desirable responding. In addition, most mode comparison stud- ies lack validation data and thus cannot separate the effects of differ- ential nonresponse bias from the effects of differences in measurement error. This paper uses survey and record data to examine mode effects on the reporting of potentially sensitive information by a sample of re- cent university graduates. Respondents were randomly assigned to one of three modes of data collection—conventional computer-assisted tele- phone interviewing (CATI), interactive voice recognition (IVR), and the Web—and were asked about both desirable and undesirable attributes of their academic experiences. University records were used to evaluate the accuracy of the answers and to examine differences in nonresponse bias by mode. Web administration increased the level of reporting of sensi- tive information and reporting accuracy relative to conventional CATI, with IVR intermediate between the other two modes. Both mode of data collection and the actual status of the respondent influenced whether respondents found an item sensitive.