Search

Data collection staff involved in sampling designs, monitoring and analysis of surveys often have a good sense of the response rate that can be expected in a survey, even when this survey is new or done at a relatively low frequency. They make expectations of response rates, and, subsequently, costs on an almost continuous basis. Rarely, however, are these expectations formally structured. Furthermore, the expectations usually are point estimates without any assessment of precision or uncertainty. In recent years, the interest in adaptive survey designs has increased. These designs lean heavily on accurate estimates of response rates and costs. In order to account for inaccurate estimates, a Bayesian analysis of survey design parameters is very sensible. The combination of strong intrinsic knowledge of data collection staff and a Bayesian analysis is a natural next step. In this article, prior elicitation is developed for design parameters with the help of data collection staff. The elicitation is applied to two case studies in which surveys underwent a major redesign and direct historic survey data was unavailable. [ABSTRACT FROM AUTHOR]

The paper deals with the issue of solving nonresponse problems in a realized census. The purpose is to discuss the statistical methods we explain. To test the new approach, the data from the survey at one University are used. The suggested approach offers more accurate estimates because of compensation for nonresponse and the possibility to formulate broader conclusions based on the census data. The approach is advised in all surveys in which the costs of realization in the survey by census are practically the same as for sample survey, and the list of all units of the population is available. [ABSTRACT FROM AUTHOR]

Response probabilities are used in adaptive and responsive survey designs to guide data collection efforts, often with the goal of diversifying the sample composition. However, if response probabilities are also correlated with measurement error, this approach could introduce bias into survey data. This study analyzes the relationship between response probabilities and data quality in grid questions. Drawing on data from the probability-based GESIS panel, we found low propensity cases to more frequently produce item nonresponse and nondifferentiated answers than high propensity cases. However, this effect was observed only among long-time respondents, not among those who joined more recently. We caution that using adaptive or responsive techniques may increase measurement error while reducing the risk of nonresponse bias.

Sample surveys are often affected by nonresponse. These surveys have in common that their outcomes depend at least partly on a human decision whether or not to participate. If it would be completely clear how this decision mechanism works, estimates could be corrected. An often used approach is to introduce the concept of the response probability. Of course, these probabilities are a theoretical concept and therefore unknown. The idea is to estimate them by using the available data. If it is possible to obtain good estimates of the response probabilities, they can be used to improve estimators of population characteristics. Estimating response probabilities relies heavily on the use of models. An often used model is the logit model. In the article, this model is compared with the simple linear model. Estimation of response probabilities models requires the individual values of the auxiliary variables to be available for both the respondents and the nonrespondents of the survey. Unfortunately, this is often not the case. This article explores some approaches for estimating response probabilities that have less heavy data requirements. The estimated response probabilities were also used to measure possible deviations from representativity of the survey response. The indicator used is the coefficient of variation (CV) of the response probabilities. [ABSTRACT FROM AUTHOR]

Latent variable models are widely used in social sciences for measuring constructs (latent variables) such as ability, attitude, behavior, and wellbeing. Those unobserved constructs are measured through a number of observed items (variables). The observed variables are often subject to item nonresponse that may be nonignorable. Incorporating a missingness mechanism within the model used to analyze data with nonresponse is crucial to obtain valid estimates for parameters, especially when the missingness is nonignorable. In this paper, we propose a latent class model (LCM) where a categorical latent variable is used to capture a latent phenomenon, and another categorical latent variable is used to summarize response propensity. The proposed model incorporates a missingness mechanism. Bayesian estimation using Markov Chain Monte Carlo (MCMC) methods are used for fitting this LCM. Real data with binary items from the 2014 Egyptian Demographic and Health Survey (EDHS14) are used. Different levels of missingness are artificially created in order to study results of the model under low, moderate and high levels of missingness. [ABSTRACT FROM AUTHOR]

Analysis of sample survey data often requires adjustments for missing val- ues in the variables of interest. Standard adjustments based on item imputation or on propensity weighting factors rely on the availability of auxiliary variables for both responding and non-responding units. Their application can be challenging when the auxiliary variables are numerous and are themselves subject to incomplete-data problems. This paper shows how classification and regression trees and forests can overcome these difficulties and compares them with likelihood methods in terms of bias and mean squared error. The development centers on a component of income data from the U.S. Consumer Expenditure Survey, which has a relatively high rate of item missingness. Classification trees and forests are used to model the unit- level propensity for item missingness in the income component. Regression trees and forests are used to model the conditional mean of the income component. The methods are then used to estimate the mean of the income component, adjusted for item nonresponse. Thirteen methods for estimating a population mean are compared in simulation experiments. The results show that if the number of auxiliary variables with missing values is not small, or if they have substantial missingness rates, likelihood methods can be impracticable or inapplicable. Tree and forest methods are always applicable, are relatively fast, and have higher efficiency than likelihood methods under real-data situations with incomplete-data patterns similar to that in the abovementioned survey. Their efficiency loss under parametric conditions most favorable to likelihood methods is observed to be between 10-25%. [ABSTRACT FROM AUTHOR]

Summary: When missing data are produced by a non‐ignorable non‐response mechanism, analysis of the observed data should include a model for the probabilities of responding. We propose such models for non‐response in survey questions which are treated as measures of latent constructs and analysed by using latent variable models. The non‐response models that we describe include additional latent variables (latent response propensities) which determine the response probabilities. We argue that this model should be specified as flexibly as possible, and we propose models where the response propensity is a categorical variable (a latent response class). This can be combined with any latent variable model for the survey items, and an association between the latent variables measured by the items and the latent response propensities then implies a model with non‐ignorable non‐response. We consider in particular such models for the analysis of data from cross‐national surveys, where the non‐response model may also vary across the countries. The models are applied to data on welfare attitudes in 29 countries in the European Social Survey. [ABSTRACT FROM AUTHOR]

Using paradata from a prior survey that is linked to a new survey can help a survey organization develop more effective sampling strategies. One example of this type of linkage or subsampling is between the National Health Interview Survey (NHIS) and the Medical Expenditure Panel Survey (MEPS). MEPS is a nationally representative sample of the U.S. civilian, noninstitutionalized population based on a complex multi-stage sample design. Each year a new sample is drawn as a subsample of households from the prior year's NHIS. The main objective of this article is to examine how paradata from a prior survey can be used in developing a sampling scheme in a subsequent survey. A framework for optimal allocation of the sample in substrata formed for this purpose is presented and evaluated for the relative effectiveness of alternative substratification schemes. The framework is applied, using real MEPS data, to illustrate how utilizing paradata from the linked survey offers the possibility of making improvements to the sampling scheme for the subsequent survey. The improvements aim to reduce the data collection costs while maintaining or increasing effective responding sample sizes and response rates for a harder to reach population. [ABSTRACT FROM AUTHOR]

We consider four methods for estimating response propensities: three traditional ones (linear, logistic, probit) and one more recent, a decision tree method. We show that some but not all the methods produce estimates that calibrate sample totals of auxiliary variables down to the response set totals. The downward calibration property reveals interesting relationships between estimated propensities, auxiliary variables, and true response probabilities. However, the property itself does not guarantee more accurate propensity estimation. Our simulation study shows that the accuracy of the estimation method depends primarily on the relationship nature between true response probabilities and auxiliary variables. [ABSTRACT FROM AUTHOR]

It is well established that interviewers learn behaviors both during training and on the job. How this learning occurs has received surprisingly little empirical attention: Is it driven by the interviewer herself or by the respondents she interviews? There are two competing hypotheses about what happens during field data collection: (1) interviewers learn behaviors from their previous interviews, and thus change their behavior in reaction to the behaviors previously encountered; and (2) interviewers encounter different types of and, especially, less cooperative respondents (i.e., nonresponse propensity affecting the measurement error situation), leading to changes in interview behaviors over the course of the field period. We refer to these hypotheses as the experience and response propensity hypotheses, respectively. This paper examines the relationship between proxy indicators for the experience and response propensity hypotheses on interview length using data and paradata from two telephone surveys. Our results indicate that both interviewer-driven experience and respondent-driven response propensity are associated with the length of interview. While general interviewing experience is nonsignificant, within-study experience decreases interview length significantly, even when accounting for changes in sample composition. Interviewers with higher cooperation rates have significantly shorter interviews in study one; however, this effect is mediated by the number of words spoken by the interviewer. We find that older respondents and male respondents have longer interviews despite controlling for the number of words spoken, as do respondents who complete the survey at first contact. Not surprisingly, interviews are significantly longer the more words interviewers and respondents speak. [ABSTRACT FROM AUTHOR]

The article offers the author's insights on the paper "What's the Chance? Interviewers' Expectations of Response in the 2010 SCF," by economist Arthur B. Kennickell. Topics include the concern of Kennickell regarding the Survey of Consumer Finances (SCF), task of field interviewers, and interviewer judgments of response propensity.

In the present work, we have proposed the calibrated estimator of population mean in the presence of unit non response. The non response has been adjusted using the non response adjustment factor depending on different models (which are inverse linear , logistic and exponential) based on auxiliary information, incorporated in calibration estimation. We have studied the case of unit non response through calibration technique under three sampling designs named as simple random sampling (SRS), probability proportional to size sampling (PPS) and stratified random sampling. A broad simulation study has been conducted to evaluate the performances of the calibrated estimators in each sampling scheme. [ABSTRACT FROM AUTHOR]

La no respuesta es un problema creciente en encuestas poblacionales que puede ser causa de sesgo de no-respuesta cuando respondedores y no respondedores difieren sistemáticamente. Objetivo: Comparar los resultados obtenidos mediante dos técnicas de corrección para el sesgo de no-respuesta: sustitución muestral y pesos de no respuesta obtenidos mediante propensión a responder. Métodos: Se comparan los efectos de la sustitución muestral semicontrolada y el uso de pesos de ajuste obtenidos mediante la propensión a responder sobre seis resultados de una encuesta de salud. Resultados: A pesar de las diferencias significativas entre respondedores y no respondedores, mediante la corrección las prevalencias estimadas sólo cambian levemente, dando ambas técnicas de ajuste resultados similares. Sólo en el caso del tabaquismo, la sustitución muestral parece haber aumentado el sesgo de la estimación. Conclusiones: Nuestros resultados sugieren que tanto mediante un procedimiento de sustitución muestral semicontrolada, como a través del ajuste estadístico de la no respuesta mediante la propensión a responder, se obtienen estimaciones de prevalencias corregidas similares.Unit non-response is a growing problem in sample surveys that can bias survey estimates if respondents and non-respondents differ systematically. Objetives: To compare the results of two nonresponse adjustment methods: field substitution and weighting nonresponse adjustment based on response propensity. Methods: Field substitution and response propensity weights are used to adjust for non-response and their effect on the prevalence of six survey outcomes is compared. Results Although significant differences are found between respondents and non-respondents, only slight changes on prevalence estimates are observed after adjustment, with both techniques showing similar results. In the sole case of smoking, substitution seems to have further biased survey estimates. Conclusions: Our results suggest that when there is information available for both respondents and non-respondents, or if a careful sample substitution process is performed, weighting adjustments based on response propensity and field substitution produce comparable results on prevalence estimates.

Longitudinal data are collected for studying changes across time. We consider multivariate longitudinal data where multiple observed variables, measured at each time point, are used as indicators for theoretical constructs (latent variables) of interest. A common problem in longitudinal studies is dropout, where subjects exit the study prematurely. Ignoring the dropout mechanism can lead to biased estimates, especially when the dropout isnonrandom. Our proposed approach uses latent variable models to capture the evolution of the latent phenomenon over time while also accounting for possibly nonrandom dropout. The dropout mechanism is modeled with a hazard function that depends on the latent variables and observed covariates. Different relationships among these variables and the dropout mechanism are studied via 2 model specifications. The proposed models are used to study people’s perceptions of women’s work using 3 questions from 5 waves from the British Household Panel Survey. [ABSTRACT FROM AUTHOR]

The pairfam study offers the rare opportunity of conducting dyadic analyses of partner and of parent-child relationships. Not only the randomly drawn anchor respondents are surveyed in the study, but also – with the consent of the anchors – their partners, parents and children. However, we must ask whether or in how far the participation of the secondary respondents is selective, thus introducing a nonresponse bias in the dyadic data. This article analyses what factors influence the participation of partners and parents of the anchors on the German Family Panel pairfam. We focus on the question of whether the quality of the relationship between the anchor and partner or anchor and parent influences participation. Among parents there is both an influence of the relationship quality in the narrower sense and an influence of the closeness of the relationship with regard to contact and mutual support. By contrast, relationship quality appears to have a lesser significance for the participation of partners, whereas the degree of institutionalisation in the relationship has a major influence. The article aims to sensitise pairfam users to the possibility of a nonresponse bias in dyadic analyses and provides information on suitable handling of the data.

Abstract: Handling missing data based on parametric models typically involves computing the conditional expectation of missing data given observed data for nonrespondents. Under a nonignorable missing data mechanism, the conditional distribution requires joint modeling of the study and response variables. A natural way of factoring the model is to use models for the distribution of variables under complete response and for the probability of response. Sensitivity to model specification is a serious scientific problem. Models cannot be validated, however, from missing data, because, by definition, the information needed for validation is missing. In many cases, under assumed models, a Monte Carlo (MC) method can be used to compute the conditional expectation of missing given observed variables. The issue of model specification translates into the questions, how should one generate values from the conditional distribution for nonrespondents? One way to interpret this issue is as the need to specify an imputation method for the missing data. In this paper, we consider a simulation method based on the model for the distribution of respondents together with the Sampling Importance Resampling (SIR) algorithm. The proposed method is shown to be more robust than some current approaches in the sense that assumed models can be verified from respondents. A linearized variance estimation method is also studied. Results from a limited simulation study are presented. [Copyright &y& Elsevier]

How response rates are increased can determine the remaining nonresponse bias in estimates. Studies often target sample members that are most likely to be interviewed to maximize response rates. Instead, we suggest targeting likely nonrespondents from the onset of a study with a different protocol to minimize nonresponse bias. To inform the targeting of sample members, various sources of information can be utilized: paradata collected by interviewers, demographic and substantive survey data from prior waves, and administrative data. Using these data, the likelihood of any sample member becoming a nonrespondent is estimated and on those sample cases least likely to respond, a more effective, often more costly, survey protocol can be employed to gain respondent cooperation. This paper describes the two components of this approach to reducing nonresponse bias. We demonstrate assignment of case priority based on response propensity models, and present empirical results from the use of a different protocol for prioritized cases. In a field data collection, a random half of cases with low response propensity received higher priority and increased resources. Resources for high-priority cases were allocated as interviewer incentives. We find that we were relatively successful in predicting response outcome prior to the survey and stress the need to test interventions in order to benefit from case prioritization.

This review of nonresponse in cross-sectional household surveys in the United States shows trends in nonresponse rates, the main reasons for nonresponse, and changes in the components of nonresponse. It shows that nonresponse is increasing but that existing methods for modeling response mechanisms do not adequately explain these changes. [ABSTRACT FROM PUBLISHER]

Abstract: Nonresponse is a major source of estimation error in sample surveys. The response rate is widely used to measure survey quality associated with nonresponse, but is inadequate as an indicator because of its limited relation with nonresponse bias. proposed an alternative indicator, which they refer to as an indicator of representativeness or R-indicator. This indicator measures the variability of the probabilities of response for units in the population. This paper develops methods for the estimation of this R-indicator assuming that values of a set of auxiliary variables are observed for both respondents and nonrespondents. We propose bias adjustments to the point estimator proposed by and demonstrate the effectiveness of this adjustment in a simulation study where it is shown that the method is valid, especially for smaller sample sizes. We also propose linearization variance estimators which avoid the need for computer-intensive replication methods and show good coverage in the simulation study even when models are not fully specified. The use of the proposed procedures is also illustrated in an application to two business surveys at Statistics Netherlands. [Copyright &y& Elsevier]

On building a model for estimating response propensity score for survey data adjustment, we carried out computer simulations to examine the effects of seven types of variables, each having differing associations with the sample inclusion probability, response probability and study variable, by comparing the respective cases in which the variables are included and excluded by the model. Then the following main results were obtained. The most important variable for the model is the one that is simultaneously associated with the study variable, the sample inclusion probability, and the response probability. The variables which have no association with the study variable should not be included in the response propensity model. These results support the conclusions of Brookhart et al. (2006), who examined propensity score models in their study on estimating treatment effects. Additionally, a small difference was found in comparing the effects of the variable associated with sample inclusion probability and the study variable to those of the variable associated with the response probability and the study variable. [ABSTRACT FROM AUTHOR]

Parametric model-based regression imputation is commonly applied to missing-data problems, but is sensitive to misspecification of the imputation model. Little and An (2004) proposed a semiparametric approach called penalized spline propensity prediction (PSPP), where the variable with missing values is modeled by a penalized spline (P-Spline) of the response propensity score, which is logit of the estimated probability of being missing given the observed variables. Variables other than the response propensity are included parametrically in the imputation model. However they only considered point estimation based on single imputation with PSPP. We consider here three approaches to standard errors estimation incorporating the uncertainty due to non response: (a) standard errors based on the asymptotic variance of the PSPP estimator, ignoring sampling error in estimating the response propensity; (b) standard errors based on the bootstrap method; and (c) multiple imputation-based standard errors using draws from the joint posterior predictive distribution of missing values under the PSPP model. Simulation studies suggest that the bootstrap and multiple imputation approaches yield good inferences under a range of simulation conditions, with multiple imputation showing some evidence of closer to nominal confidence interval coverage when the sample size is small. [ABSTRACT FROM AUTHOR]

Many health surveys conduct an initial household interview to obtain demographic information and then request permission to obtain detailed information on health outcomes from the respondent's health care providers. A ‘complete response’ results when both the demographic information and the detailed health outcome data are obtained. A ‘partial response’ results when the initial interview is complete but, for one reason or another, the detailed health outcome information is not obtained. If ‘complete responders’ differ from ‘partial responders’ and the proportion of partial responders in the sample is at least moderately large, statistics that use only data from complete responders may be severely biased. We refer to bias that is attributable to these differences as ‘partial non-response’ bias. In health surveys it is customary to adjust survey estimates to account for potential differences by employing adjustment cells and weighting to reduce bias from partial response. Before making these adjustments, it is important to ask whether an adjustment is expected to increase or decrease bias from partial non-response. After making these adjustments, an equally important question is ‘How well does the method of adjustment work to reduce partial non-response bias?’. The paper describes methods for answering these questions. Data from the US National Immunization Survey are used to illustrate the methods. [ABSTRACT FROM AUTHOR]

Responsive survey designs introduce protocol changes to survey operations based on accumulating paradata. Case-level predictions, including response propensity, can be used to tailor data collection features in pursuit of cost or quality goals. Unfortunately, predictions based only on partial data from the current round of data collection can be biased, leading to ineffective tailoring. Bayesian approaches can provide protection against this bias. Prior beliefs, which are generated from data external to the current survey implementation, contribute information that may be lacking from the partial current data. Those priors are then updated with the accumulating paradata. The elicitation of the prior beliefs, then, is an important characteristic of these approaches. While historical data for the same or a similar survey may be the most natural source for generating priors, eliciting prior beliefs from experienced survey managers may be a reasonable choice for new surveys, or when historical data are not available. Here, we fielded a questionnaire to survey managers, asking about expected attempt-level response rates for different subgroups of cases, and developed prior distributions for attempt-level response propensity model coefficients based on the mean and standard error of their responses. Then, using respondent data from a real survey, we compared the predictions of response propensity when the expert knowledge is incorporated into a prior to those based on a standard method that considers accumulating paradata only, as well as a method that incorporates historical survey data.