Common measures of vaccination intention generate substantially different estimates that can reduce predictive validity

Abstract Surveys often estimate vaccination intentions using dichotomous ("Yes"/"No") or trichotomous ("Yes," "Unsure," "No") response options presented in different orders. Do survey results depend on these variations? This controlled experiment randomized participants to dichotomous or trichotomous measures of vaccine intentions (with “Yes” and “No” options presented in different orders). Intentions were measured separately for COVID-19, its booster, and influenza vaccines. Among a sample of U.S. adults (N = 4,764), estimates of vaccine intention varied as much as 37.5 ± 17.4 percentage points as a function of the dichotomous or trichotomous response set. Among participants who had not received the COVID-19 vaccine, the “Unsure” option was more likely to reduce the share of “No” (versus “Yes”) responses, whereas among participants who had received the COVID-19 vaccine, the “Unsure” option was more likely to reduce the share of “Yes” (versus “No”) responses. The “Unsure” category may increase doubt and decrease reliance on past vaccination behavior when forming intentions. The order of “Yes” and “No” responses had no significant effect. Future research is needed to further evaluate why the effects of including the “Unsure” option vary in direction and magnitude.