Search

Survey researchers are often confronted with the question of how long to set the length of the field period. Longer fielding time might lead to greater participation yet requires survey managers to devote more of their time to data collection efforts. With the aim of facilitating the decision about the length of the field period, we investigated whether a longer fielding time reduces the risk of nonresponse bias to judge whether field periods can be ended earlier without endangering the performance of the survey. By using data from six waves of a probability-based mixed-mode (online and mail) panel of the German population, we analyzed whether the risk of nonresponse bias decreases over the field period by investigating how day-by-day coefficients of variation develop during the field period. We then determined the optimal cut-off points for each mode after which data collection can be terminated without increasing the risk of nonresponse bias and found that the optimal cut-off points differ by mode. Our study complements prior research by shifting the perspective in the investigation of the risk of nonresponse bias to panel data as well as to mixed-mode surveys, in particular. Our proposed method of using coefficients of variation to assess whether the risk of nonresponse bias decreases significantly with each additional day of fieldwork can aid survey practitioners in finding the optimal field period for their mixed-mode surveys.

This article reviews unit nonresponse in cross-sectional household surveys, the consequences of the nonresponse on the bias of the estimates, and methods of adjusting for it. We describe the development of models for nonresponse bias and their utility, with particular emphasis on the role of response propensity modeling and its assumptions. The article explores the close connection between data collection protocols, estimation strategies, and the resulting nonresponse bias in the estimates. We conclude with some comments on the current state of the art and the need for future developments that expand our understanding of the response phenomenon.

Nonresponse is a major source of estimation error in sample surveys. The response rate is widely used to measure survey quality associated with nonresponse, but is inadequate as an indicator because of its limited relation with nonresponse bias. Schouten et al. (2009) proposed an alternative indicator, which they refer to as an indicator of representativeness or R-indicator. This indicator measures the variability of the probabilities of response for units in the population. This paper develops methods for the estimation of this R-indicator assuming that values of a set of auxiliary variables are observed for both respondents and nonrespondents. We propose bias adjustments to the point estimator proposed by Schouten et al. (2009) and demonstrate the effectiveness of this adjustment in a simulation study where it is shown that the method is valid, especially for smaller sample sizes. We also propose linearization variance estimators which avoid the need for computer-intensive replication methods and show good coverage in the simulation study even when models are not fully specified. The use of the proposed procedures is also illustrated in an application to two business surveys at Statistics Netherlands.

La no respuesta es un problema creciente en encuestas poblacionales que puede ser causa de sesgo de no-respuesta cuando respondedores y no respondedores difieren sistemáticamente. Objetivo: Comparar los resultados obtenidos mediante dos técnicas de corrección para el sesgo de no-respuesta: sustitución muestral y pesos de no respuesta obtenidos mediante propensión a responder. Métodos: Se comparan los efectos de la sustitución muestral semicontrolada y el uso de pesos de ajuste obtenidos mediante la propensión a responder sobre seis resultados de una encuesta de salud. Resultados: A pesar de las diferencias significativas entre respondedores y no respondedores, mediante la corrección las prevalencias estimadas sólo cambian levemente, dando ambas técnicas de ajuste resultados similares. Sólo en el caso del tabaquismo, la sustitución muestral parece haber aumentado el sesgo de la estimación. Conclusiones: Nuestros resultados sugieren que tanto mediante un procedimiento de sustitución muestral semicontrolada, como a través del ajuste estadístico de la no respuesta mediante la propensión a responder, se obtienen estimaciones de prevalencias corregidas similares. Unit non-response is a growing problem in sample surveys that can bias survey estimates if respondents and non-respondents differ systematically. Objetives: To compare the results of two nonresponse adjustment methods: field substitution and weighting nonresponse adjustment based on response propensity. Methods: Field substitution and response propensity weights are used to adjust for non-response and their effect on the prevalence of six survey outcomes is compared. Results Although significant differences are found between respondents and non-respondents, only slight changes on prevalence estimates are observed after adjustment, with both techniques showing similar results. In the sole case of smoking, substitution seems to have further biased survey estimates. Conclusions: Our results suggest that when there is information available for both respondents and non-respondents, or if a careful sample substitution process is performed, weighting adjustments based on response propensity and field substitution produce comparable results on prevalence estimates.

ResumenLa no respuesta es un problema creciente en encuestas poblacionales que puede ser causa de sesgo de no-respuesta cuando respondedores y no respondedores difieren sistemáticamente.ObjetivoComparar los resultados obtenidos mediante dos técnicas de corrección para el sesgo de no-respuesta: sustitución muestral y pesos de no respuesta obtenidos mediante propensión a responder.MétodosSe comparan los efectos de la sustitución muestral semicontrolada y el uso de pesos de ajuste obtenidos mediante la propensión a responder sobre seis resultados de una encuesta de salud.ResultadosA pesar de las diferencias significativas entre respondedores y no respondedores, mediante la corrección las prevalencias estimadas sólo cambian levemente, dando ambas técnicas de ajuste resultados similares. Sólo en el caso del tabaquismo, la sustitución muestral parece haber aumentado el sesgo de la estimación.ConclusionesNuestros resultados sugieren que tanto mediante un procedimiento de sustitución muestral semicontrolada, como a través del ajuste estadístico de la no respuesta mediante la propensión a responder, se obtienen estimaciones de prevalencias corregidas similares.AbstractUnit non-response is a growing problem in sample surveys that can bias survey estimates if respondents and non-respondents differ systematically.ObjetivesTo compare the results of two nonresponse adjustment methods: field substitution and weighting nonresponse adjustment based on response propensity.MethodsField substitution and response propensity weights are used to adjust for non-response and their effect on the prevalence of six survey outcomes is compared.ResultsAlthough significant differences are found between respondents and non-respondents, only slight changes on prevalence estimates are observed after adjustment, with both techniques showing similar results. In the sole case of smoking, substitution seems to have further biased survey estimates.ConclusionsOur results suggest that when there is information available for both respondents and non-respondents, or if a careful sample substitution process is performed, weighting adjustments based on response propensity and field substitution produce comparable results on prevalence estimates.