Search

Some fundamental factors important for designing multistage face-to-face household surveys in the United States have not been reconsidered since they were established in the 1940s despite vast changes in all aspects of conducting these surveys. This research examines how some changes, particularly data sources and data collection costs, might affect sample designs. Modern data collection methods and tools, address-based household lists, multimode designs, and training and management methods are critical components in the cost structure of today's surveys. We explore a sample design that uses Census tracts at the first stage rather than counties as traditionally used. The tract design then samples households directly without the additional stages of subsampling of segments in the traditional design. We present statistical and operational issues associated with the designs and discuss the cost implications. Our examination finds that, for household surveys in the United States that include face-to-face data collection, tract designs that have twice as many sampled first-stage units (but the same overall sample size) have roughly the same cost as the county design, with effective sample sizes in the range of about 25 percent higher to as much as four times higher than the county design. [ABSTRACT FROM AUTHOR]

For over a decade, address-based sampling (ABS) frames have often been used to draw samples for multistage area sample surveys in lieu of traditionally listed (or enumerated) address frames. However, it is well known that the use of ABS frames for face-to-face surveys suffer from undercoverage due to, for example, households that receive mail via a PO Box rather than being delivered to the household's street address. Undercoverage of ABS frames has typically been more prominent in rural areas but can also occur in urban areas where recent construction of households has taken place. Procedures have been developed to supplement ABS frames to address this undercoverage. In this article, we investigate a procedure called Address Coverage Enhancement (ACE) that supplements the ABS frame with addresses not found on the frame, and the resulting effects the addresses added to the sample through ACE have on estimates. Weighted estimates from two studies, the Population Assessment of Tobacco and Health Study and the 2017 US Program for the International Assessment of Adult Competencies, are calculated with and without supplemental addresses. Estimates are then calculated to assess if poststratifying analysis weights to control for urbanicity at the person level brings estimates closer to estimates from the supplemented frame. Our findings show that the noncoverage bias was likely minimal across both studies for a range of estimates. The main reason is because the Computerized Delivery Sequence file coverage rate is high, and when the coverage rate is high, only very large differences between the covered and not covered will result in meaningful bias. [ABSTRACT FROM AUTHOR]

Multimode data collection has emerged as a common approach for conducting household surveys in the United States. A number of different data collection schemes have been investigated, with an emphasis on collecting as many respondents by the Web prior to going to paper data collection to reduce costs. Despite this, little research has been conducted on the approaches to weighting data from multimode surveys. The typical approach assumes that all respondents should be treated the same regardless of mode even though it is well known that the response patterns by mode vary substantially. We examine an adaptive mode adjustment to address these differences and propose an imbalance measure to help determine the adjustment factor using ideas from responsive design. We then compare the effects of the alternative weighting method in two recent sequential mixed-mode surveys and show it appears to reduce bias while only slightly increasing variances of the estimates. [ABSTRACT FROM AUTHOR]

Research on nonresponse in face-to-face surveys in the United States has shown that nonresponse increases over time for most surveys, but there are also periods of fairly stable rates. Surprisingly, nonresponse for faceto- face surveys has not been as widely studied recently as compared to telephone surveys. The focus on telephone surveys may be due to the dramatic increase in nonresponse for these surveys or perhaps because face-to-face surveys still achieve relatively high levels of response. This paper updates nonresponse trends for face-to-face household surveys conducted in the United States since 2000. The review provides a comprehensive picture of the industry by including surveys conducted by government, private, and academic organizations. The relative role of refusals and noncontacts in the total nonresponse is also presented. We tie the trends in nonresponse with data on the level of effort for some of these surveys. Many researchers have suggested that extra effort has been needed to prevent response rates from falling even more precipitously but lacked the effort data to evaluate this hypothesis for face-toface surveys. Some data on field effort are becoming available, so this question can be addressed for the first time across more than one survey. To complete the picture, we also look at loss over time for longitudinal face-to-face surveys. [ABSTRACT FROM AUTHOR]

This paper presents several results for the optimal allocation of dual frame designs under different conditions, assuming the estimators are poststratified to domain totals. Both screener and overlap designs are discussed, and the results of the allocation for the two types of designs are illustrated by showing the efficiency of optimal designs in dual frame landline and cell phone surveys. We begin with results on optimal sample allocation to the frames assuming complete response and a linear cost function. We then compare the variance for estimates from the optimal screening and overlap designs, and extend these results to the situation where response rates differ for the landline and cell phone domains. We find that a screening design achieves a smaller variance than an overlap design with the same total cost under many cost structures. [ABSTRACT FROM AUTHOR]

The article focuses on a task force appointed by the American Association of Public Opinion Research (AAPOR) Executive Council in 2011 to examine the conditions that designs survey that do not use probability samples. Topics discussed include comparisons between probability and non-probability methods and methods for collecting data and producing estimates.

The article focuses on the comments raised related to Task Force report by the american Association for Public Opinion Research (AAPOR). Topics discussed include comparison of probability sample with a non-probability sample, inferential risk with non-probability samples and equivalence of missing data.

We review existing methods for sampling an adult from a household and highlight the strengths and weaknesses of these methods with respect to issues relevant for sampling in the Middle East and developing countries. We then develop a new, flexible within-household sampling scheme that takes full advantage of the fact that household size must be obtained to produce unbiased estimates. The proposed method randomly selects an adult conditional on the number of adults in the household. The method was applied in a national sample in Qatar and it proved it was effective in a country where large household sizes are common and the information required for other sampling methods is generally not known. [ABSTRACT FROM PUBLISHER]

This article evaluates methods to improve response rates from Spanish speakers in two-phase mail surveys of the U.S. population. Many surveys are subject to nonresponse bias and under-represent the 20 percent of households that speak a language other than English at home. In the United States, Spanish speakers comprise 60 percent of the persons who speak some language other than English at home. A set of language treatment experiments was designed to improve response rates from Spanish speakers, and these were implemented in the 2011 National Household Education Surveys Program (NHES) Field Test. The language experiments included treatments that varied the languages used in the materials (both English and Spanish, or English only), whether the materials in both languages were mailed in all mailings or just in the follow-up, the way the instruments presented the language alternatives, and whether the language efforts were targeted at certain addresses or sent to all households. The findings show that sending survey materials to all sampled households in both languages substantially increased participation from Spanish-speaking households and did not jeopardize responses from monolingual English households. [ABSTRACT FROM PUBLISHER]

The twentieth century saw a dramatic change in the way information was generated as probability sampling replaced full enumeration. We examine some key events of the past and issues being addressed today to gain perspective on the requirements that might steer changes in the direction of survey sampling in the future. The potential for online Web panels and other methods that do not use probability sampling are considered. Finally, we conclude with some thoughts on how the future of survey sampling might be shaped by unpredictable factors. [ABSTRACT FROM PUBLISHER]

In this article, we describe the use of address-based sampling (ABS) for conducting a mail survey of a subgroup. Although ABS has been investigated as an alternative household sampling frame, its use for sampling subgroups is limited. Some researchers have used ABS to select households but then recruit households by telephone to the extent possible to screen for members of the subgroup. The approach we describe uses a two-phase methodology with mail as the primary mode of recruitment and data collection for all households. Facing declining response and coverage rates in its landline RDD surveys, the National Household Education Surveys (NHES) program embarked on a redesign effort. The two-phase ABS design using mail as the primary mode of data collection was chosen for the NHES:2009 Pilot Study. In the first phase, all sampled households were mailed a screener that identified members of the eligible subgroup (children) and provided the data needed for sampling among them. A second-phase substantive questionnaire was mailed to the sampled members of the subgroup. This use of the ABS and mail as the primary mode of data collection is new and largely untested. The findings from the NHES Pilot Study revealed that this approach is a feasible alternative for surveying targeted subgroups. Although the methods must undergo more robust and rigorous tests, it is an approach that may be able to fill some of the void created by the low response and coverage rates that afflict RDD surveys. [ABSTRACT FROM PUBLISHER]

For decades, traditional listing has been the standard method for constructing address-based sampling (ABS) frames. Address lists compiled by listers generally are believed to provide good coverage of residential units in the sampled area. Nonetheless, the time and resources associated with creating these address lists have led survey practitioners to consider the use of the U.S. Postal Service (USPS) Computerized Delivery Sequence File listings as an alternative. Previous studies have yielded an abundance of information about the utility of ABS frames. In our recent investigation (Montaquila et al. 2009), we found coverage of the USPS-based address lists to be generally good in urban areas but inadequate in some rural areas, and also poorer in high-population-growth areas. Resources, time, and effort could be conserved if we are able to use the USPS list as a sampling frame in areas where it provides adequate coverage. In this article, we develop a “match rate” model that uses multiple regression to predict the match rate (the proportion of traditionally listed addresses that would have been obtained from a USPS list) based on characteristics of the geographic areas of interest. We discuss the feasibility of using USPS lists for household surveys, drawing on results from our previous work and the match rate modeling. We describe an approach for setting threshold values of predicted match rate to determine when to use USPS lists in lieu of traditional listing. Finally, we assess the model's external validity based on the application of the model to an independent dataset. [ABSTRACT FROM PUBLISHER]

The article presents a report published by the American Association for Public Opinion Research (AAPOR) which reviews empirical information related to the opt-in online discussion panels used for data collection on public opinion. The Opt-In Online Panel Task Force established by the AAPOR executive council in September 2008 is mentioned. Methods used to recruit online panelists are reviewed. The author comments on the difficulties related to gathering data online in spite of the number of adults in the U.S. that do not use the Internet on a regular basis. Methods used to reduce bias in public opinion research are also discussed.

Random digit dial (RDD) sampling methodology was developed over two decades ago when local telephone exchanges used 100-series telephone banks as physical building blocks for telephone number assignment. During the intervening decades the telecommunication industry has undergone a number of fundamental changes that have had drastic effects on the efficiency and coverage of RDD samples. Based on a new study of landline telephone numbers, which was conducted during the second quarter of 2008, this paper reexamines the assumptions that are relied upon for construction of list-assisted RDD samples; quantifies the extent of undercoverage in the corresponding sampling frames; investigates alternative methods of frame construction for RDD samples; and evaluates other sample design options that can offer greater coverage with varying degrees of efficiency. [ABSTRACT FROM PUBLISHER]

This article describes several features included in a California Health Interview Survey cell phone pilot study that differ from earlier cell phone surveys conducted in the United States. One difference is that the study used a screening design and only adults living in cell-only households were interviewed. Most of the previous studies used dual frame designs. Another difference was the development and implementation of a within-household adult sampling procedure to cover adults when cell phones were shared in the household. The study was also intended to determine if conducting a cell phone interview of the same scope and length as a regular telephone interview was feasible. Most previous surveys were focused on cell phone topics and were shorter than the comparable landline surveys. We then explore some interesting problems that arise in weighting a cell phone survey using a screening design. We conclude with a discussion of planning a new cell phone survey in 2007 that incorporates findings from this study. [ABSTRACT FROM AUTHOR]

Warren Mitofsky and Joseph Waksberg were two of the most influential researchers in telephone surveys and survey research methods. Their development of the Mitofsky-Waksberg method for random digit dialing sampling revolutionized telephone sampling and surveys. Jointly and separately, Waksberg and Mitofsky were instrumental in the development of many innovative methods and applications. This article reviews some of their contributions and discusses how their approaches to these problems may be a model for addressing issues that are being encountered currently in the United States in cell phone research. [ABSTRACT FROM AUTHOR]

In 2004, we conducted a nationwide dual flame survey of landline and cell phone numbers to evaluate the feasibility of including cell phone numbers in a random digit dial telephone survey. Households with both landline and cell phones were eligible for selection in both samples. This article describes our design and data collection methods; compares the results from the two samples (with an emphasis on operational characteristics); and presents the outcomes of two experimental manipulations designed to improve the cell phone response rate. [ABSTRACT FROM AUTHOR]

Changes in the U.S. telephone system, especially the rapid growth in the prevalence and use of cell phones, raise concerns about undercoverage error in random digit dial (RDD) telephone samples. A supplement to the Current Population Survey (CPS) was conducted in 2004 to examine telephone service and usage in U.S. households. This article explores the potential for biases in RDD surveys resulting from the increases in cell phones by presenting estimates of the percentage of households with different types of telephone service, including the percentage of cell-only households, and giving demographic profiles of households by type of telephone service. Logistic regression models examine variables that predict whether households are without a telephone or only have cell phones. These predictors may be used for weighting adjustments to reduce undercoverage biases. We address some additional issues, including the wording of questions for measuring telephone service, that are relevant if telephone-sampling methods are revised to include cell phones. The estimates from the CPS supplement are also used to help understand some of the new sampling and weighting problems associated with selecting samples from cell phone numbers. [ABSTRACT FROM AUTHOR]

We conducted a dual frame survey of landline and cell phone numbers in 2004 to evaluate the feasibility of including cell phone numbers in random digit dial telephone surveys in the United States. By sampling cell phone numbers, the coverage bias associated with households that have only cell phones is eliminated. However, we discovered two major sources of nonresponse bias in the dual frame sample. In an attempt to reduce these biases, we applied several different estimation schemes. But a comparison to the 2004 Current Population Survey Cell Phone Supplement showed that none of the estimation schemes substantially reduced the nonresponse bias of the estimates. We suggest other methods that might be used in future surveys that include cell phones and discuss the need for additional data collection and research on this issue. [ABSTRACT FROM AUTHOR]

Multiple imputation is a method of estimating the variances of estimators that are constructed with some imputed data. We give an expression for the bias of the multiple-imputation variance estimator for data that are collected with a complex sample design. The bias may be sizable for certain estimators, such as domain means, when a large fraction of the values are imputed. A bias-adjusted variance estimator is suggested. [ABSTRACT FROM AUTHOR]

This article presents information on sampling in random digit dial surveys. The study proposes a new and simple method of within-household sampling that is minimally intrusive and, thus, should not have a deleterious effect on response rates. The method results in a probability sample of eligible persons within households. The method is especially simple to implement in computer-assisted telephone interview random digit dial surveys. The study focus on sampling one adult from each household as this is the primary application, but we also describe how the method can be extended to sample more than one person per household. This paper provides an introduction to the method and results from one study. Experimental comparisons of this approach with other methods of within-household sampling would be a useful next step. The author has presented the method as a way of sampling one adult from all adults in the household, but it can be applied in any situation in which the screener respondent is eligible by first determining if the screener respondent is sampled. If the screener respondent is not eligible, a variant of the method can be used.

Abstract The method for estimating residency rates in random digit dial (RDD) telephone surveys is important for computing response rates. This article reviews existing methods of estimating residency rates and introduces a new "survival" method that takes advantage of more information to provide improved estimates. Examples of applying this to large RDD samples are given along with suggestions for use of the method in other surveys. [ABSTRACT FROM AUTHOR]

A number of researchers have suggested list-assisted sampling for the selection of telephone households to overcome some of the operational difficulties associated with the Mitofsky-Waksberg methods of random digit dialing (RDD). An advantage of a list-assisted method of RDD is that an equal probability systematic sample of telephone numbers can be selected and the variances of estimates from such a sample are usually lower than from a clustered design like the Mitofsky-Waksberg method. The main disadvantage of the list-assisted method is that it excludes some households from the sample, thus creating a coverage bias in the estimates. This article describes research on the coverage bias for a particular method of list-assisted sampling. The two key determinants of coverage bias are the proportion of households that are not eligible for the sample and the differences in the characteristics of the covered and not covered populations. The results show that about 4 percent of all households are excluded in national samples using this method of sampling. Furthermore, they show that the differences between the covered and uncovered populations are generally not large. The coverage bias resulting from these conditions may often be small. [ABSTRACT FROM AUTHOR]

An obituary for Dale Kulp, founder and president of Marketing Systems Group, is presented.