Search

This article leverages a five-treatment response mode experiment (paper-only, web-only, sequential web-mail, choice, and choice-plus [choice with a promised incentive for responding online]) that was conducted within a nationally representative survey. Because this survey's sample was drawn from respondents to another nationally representative survey, we have rich frame data that includes multiple indicators of comfort using the internet for our sample members and we can compare their response behavior across two surveys. We find that the paper-only treatment yielded a lower response rate than most of the other treatments, but there were not significant differences between the response rates for the other treatments. Among our mixed-mode treatments, the sequential web-mail treatment had the highest percentage of response by web and the lowest cost per response. When focusing on the subgroups that we expected to be the least—and the most—comfortable with the internet, we found that the paper-only treatment generally performed worse than the others, even among subgroups expected not to be comfortable with the internet. We generally did not find significant differences in the effect of response mode treatment on the response rate or percentage of response by web between the subgroups who were the most and least comfortable with the internet. In terms of the consistency of response mode choice over time, our results suggest that some people respond consistently—but also that response mode preferences are weak enough that they can be influenced by the way in which the modes are offered. We ultimately recommend using a sequential web-mail design to minimize costs while still providing people who cannot or will not respond by web with another response mode option. We also find evidence that there may be a growing lack of interest in responding by paper; more research is needed in this area. [ABSTRACT FROM AUTHOR]

When survey respondents have nothing to report for questions asking for a quantity, they should report a lack of quantity instead of skipping the question. Reports of zero can be collected by asking respondents to check a None box. However, the effectiveness of this design has not been supported and, with the growing prevalence of web surveys, new design options are available. This study on data quality and user experience tested a web survey and the impact of three designs for reporting zero: a None box, instructions to enter “0” if respondents had nothing to report, and a Yes/No filter question preceding a write-in quantity. We administered the online survey to an opt-in panel of respondents in the U.S. Respondents were randomly assigned one of the three designs. They received a four-question survey with expected high rates of reporting zero. After controlling for demographic variables, we found that the None Box design resulted in significantly more item nonresponse than the Enter 0 design and nominally more than the Filter Question design. Additionally, the None Box design was the least efficient as it took more time on average to answer, and respondents who received that design rated the survey as less easy to answer. These findings align with previous literature that the None Box design is problematic. This research supports the use of alternative question designs in web surveys such as entering “0” or using filter questions when respondents may not have anything to report. [ABSTRACT FROM AUTHOR]