Search

Web surveys are a common self-administered mode of data collection using written language to convey information. This language is usually accompanied by visual design elements, such as numbers, symbols, and graphics. As shown by previous research, such elements of survey questions can affect response behavior because respondents sometimes use interpretive heuristics, such as the " middle means typical " and the " left and top means first " heuristics when answering survey questions. In this study, we adopted the designs and survey questions of two experiments reported in Tourangeau, Couper, and Conrad (2004). One experiment varied the position of nonsubstantive response options in relation to other substantive response options and the second experiment varied the order of the response options. We implemented both experiments in an eye-tracking study. By recording respondents' eye movements, we are able to observe how they read question stems and response options and we are able to draw conclusions about the survey response process the questions initiate. This enables us to investigate the mechanisms underlying the two interpretive heuristics and to test the assumptions of Tourangeau et al. (2004) about the ways in which interpretive heuristics influence survey responding. The eye-tracking data reveal mixed results for the two interpretive heuristics. For the middle means typical heuristic, it remains somewhat unclear whether respondents seize on the conceptual or visual midpoint of a response scale when answering survey questions. For the left and top means first heuristic, we found that violations of the heuristic increase response effort in terms of eye fixations. These results are discussed in the context of the findings of the original studies. [ABSTRACT FROM AUTHOR]

Existing research shows that response options are endorsed at a higher rate when presented in forced-choice format (FC, yes-no grids) than in check-all-that-apply format (CATA). Information Processing Theory explains this contrast with differential effects on the level of cognitive reflection. The study tests this hypothesis with eye movement data collected in a randomized laboratory experiment in which 131 respondents participated in a web survey with four treatment questions. In one condition (CATA) the questions were presented in check-all format, in the other (FC) in forced-choice structure. I find higher levels of affirmative responses and longer completion times in FC compared to CATA in three of the questions. With all four questions, respondents invested more cognitive effort—measured by fixation counts and times—in FC than in CATA when considering the question in total. I find no differences when considering only the list of response options, however. This indicates that the longer fixation times did not result from a more careful evaluation of the response options. Other possible causes and practical implications are discussed. [ABSTRACT FROM AUTHOR]