Search

Does attentiveness matter in survey responses? Do more attentive survey participants give higher quality responses? Using data from a recent online survey that identified inattentive respondents using instructed-response items, we demonstrate that ignoring attentiveness provides a biased portrait of the distribution of critical political attitudes and behavior. We show that this bias occurs in the context of both typical closed-ended questions and in list experiments. Inattentive respondents are common and are more prevalent among the young and less educated. Those who do not pass the trap questions interact with the survey instrument in distinctive ways: they take less time to respond; are more likely to report nonattitudes; and display lower consistency in their reported choices. Inattentiveness does not occur completely at random and failing to properly account for it may lead to inaccurate estimates of the prevalence of key political attitudes and behaviors, of both sensitive and more prosaic nature.

We examine the quality of two probability-based polls, one interviewer administered (telephone) and one self-administered (Internet and mail mixed mode survey). The polls use the same sampling frame (registered voters) and the same questions. First, we examine the representativeness of both surveys using information known about the population, and although we find important differences between the two in terms of sampling and nonresponse bias, we also find that both surveys represent the underlying population despite low response rates. We also test for mode effects between surveys due to social desirability and how it influences nondifferentiation or satisficing. Using a variety of methods (t-tests, multivariate regression, and genetic propensity matching), we find evidence that the presence of an interviewer alters response patterns on ego-driven questions. The implications of our work are important, due to the increasing popularity of mixed mode surveys. Researchers need to be methodologically sensitive to these differences when analyzing surveys that allow for different response modes.

Over the last decade the survey environment has been rapidly changing, creating new opportunities and new challenges for survey methodologists. These include: relatively low response among all contact modes and designs; the rapid replacement of landline telephones for mobile and smart phones; the integration of new and different types of data for sampling and new sampling methods; the use of different survey interview modes; how to study behaviors and opinions about sensitive topics; and the development of survey designs for experts and opinion leaders. All have implications for data quality and for our ability to consistently and reliably answer substantive and theoretical questions about politics and society. Obviously, a comprehensive survey of how survey methodologists are advancing new approaches to deal with this vast array of challenges and opportunities is not possible in a mini-symposium in Political Analysis. Nevertheless, in this Mini-Symposium on Advances in Survey Methodology, we include five papers that focus on new directions that survey methodologists, especially those working in political science, are developing in response to the changing environment.

Empirical researchers studying party systems often struggle with the question of how to count parties. Indexes of party system fragmentation used to address this problem (e.g., the effective number of parties) have a fundamental shortcoming: since the same index value may represent very different party systems, they are impossible to interpret and may lead to erroneous inference. We offer a novel approach to this problem: instead of focusing on index measures, we develop a model that predicts theentire distributionof party vote-shares and, thus, does not require any index measure. First, a model of party counts predicts the number of parties. Second, a set of multivariatetmodels predicts party vote-shares. Compared to the standard index-based approach, our approach helps to avoid inferential errors and, in addition, yields a much richer set of insights into the variation of party systems. For illustration, we apply the model on two data sets. Our analyses call into question the conclusions one would arrive at by the index-based approach. Software is provided to implement the proposed model.

Ordinal variables—categorical variables with a defined order to the categories, but without equal spacing between them—are frequently used in social science applications. Although a good deal of research exists on the proper modeling of ordinal response variables, there is not a clear directive as to how to model ordinal treatment variables. The usual approaches found in the literature for using ordinal treatment variables are either to use fully unconstrained, though additive, ordinal group indicators or to use a numeric predictor constrained to be continuous. Generalized additive models are a useful exception to these assumptions (Beck and Jackman 1998). In contrast to the generalized additive modeling approach, we propose the use of a Bayesian shrinkage estimator to model ordinal treatment variables. The estimator we discuss in this paper allows the model to contain both individual group level indicators and a continuous predictor. In contrast to traditionally used shrinkage models that pull the data toward a common mean, we use a linear model as the basis. Thus, each individual effect can be arbitrary, but the model “shrinks” the estimates toward a linear ordinal framework according to the data. We demonstrate the estimator on two political science examples: the impact of voter identification requirements on turnout (Alvarez, Bailey, and Katz 2007), and the impact of the frequency of religious service attendance on the liberality of abortion attitudes (e.g., Singh and Leahy 1978, Tedrow and Mahoney 1979, Combs and Welch 1982).

An important property of any party system is the set of choices it presents to the electorate. In this paper we analyze the distribution of parties relative to voters in the multidimensional issue space and introduce two measures of the dispersion of the parties in the issue space relative to the voters, which we call measures of the compactness of the parties in the issue space. We show how compactness is easily computed using standard survey items found on national election surveys. Because we study the spacing of the parties relative to the distribution of the voters, we produce metric-free measures of compactness of the party system. The measures can be used to compare party systems across issues, over time within countries, and across countries. Comparing the compactness of party systems across countries allows us to determine the relative amount of issue choice afforded voters in different polities. We examine the compactness of the issue space and test the impact it has on voter choice in four countries: the United States, the Netherlands, Canada, and Great Britain. We demonstrate that the more compact the distribution of the parties in the issue space on any given issue, the less voters weight that issue in their vote decision. Thus we provide evidence supporting theories suggesting that the greater the choice offered by the parties in an election, the more likely it is that issue voting will play a major role in that election.

This article provides a basic report about subject recruitment processes for Web-based surveys. Using data from our ongoing Internet Survey of American Opinion project, two different recruitment techniques (banner advertisement and subscription campaigns) are compared. This comparison, together with a typology of Web-based surveys, provides insight into the validity and generalizability of Internet survey data. The results from this analysis show that, although Internet survey respondents differ demographically from the American population, the relationships among variables are similar across recruitment methods and match those implied by substantive theory. Thus, our research documents the basic methodology of subject acquisition for Web-based surveys, which, as we argue in our conclusion, may soon become the survey interview mode of choice for social scientists.

Questions of causation are important issues in empirical research on political behavior. Most of the discussion of the econometric problems associated with multiequation models with reciprocal causation has focused on models with continuous dependent variables (e.g., Markus and Converse 1979; Page and Jones 1979). Yet, many models of political behavior involve discrete or dichotomous dependent variables; this paper describes two techniques which can consistently estimate reciprocal relationships between dichotomous and continuous dependent variables. The first, two-stage probit least squares (2SPLS), is very similar to two-stage instrumental variable techniques. The second, two-stage conditional maximum likelihood (2SCML), may overcome problems associated with 2SPLS, but has not been used in the political science literature. We demonstrate the potential pitfalls of ignoring the problems of reciprocal causation in nonrecursive choice models and examine the properties of both techniques using Monte Carlo simulations: we find that 2SPLS slightly outperforms 2SCML in terms of bias but that 2SCML produces more accurate standard errors. However, the 2SCML model offers an explicit statistical test for endogeneity. The results from our simulations, and the statistical test for exogeneity, lead us to advocate the use of 2SCML for estimation of this class of causal models. We then apply both of these techniques to an empirical example focusing on the relationship between voter preferences in a presidential election and the voter's uncertainty about the policy positions taken by the candidates. This example demonstrates the importance of these techniques for political science research.

 Pablo Barbera, "Birds of the Same Feather Tweet Together: Bayesian Ideal Point Estimation Using Twitter Data" (doi:10.1093/pan/mpu011).  Andrew M. Guess, "Measure for Measure: An Experimental Test of Online Political Media" (doi:10.1093/pan/mpu010).  Adam J. Berinsky, Gregory A. Huber, and Gabriel S. Lenz, "Evaluating Online Labor Markets for Experimental Research: Amazon.com's Mechanical Turk" (doi:10.1093/pan/mpr057).  Zack W. Almquist and Carter T. Butts, “Dynamic Network Logistic Regression: A Logistic Choice Analysis of Interand Intra-Group Blog Citation Dynamics in the 2004 US Presidential Election” (doi: 10.1093/pan/mpt016).  Vito D’Orazio, Steven T. Landis, Glenn Palmer and Philip Schrodt, “Separating the Wheat from the Chaff: Applications of Automated Document Classification Using Support Vector Machines” (doi: 10.1093/pan/mpt030).  Christopher Lucas, Richard A. Nielsen, Margaret E. Roberts, Brandon M. Stewart, Alex Storer, and Dustin Tingley, “Computer assisted text analysis for comparative politics” (doi: 10.1093/pan/mpu019).

Beginning in the spring of 2011, Political Analysis has published six virtual issues.1 Each of those virtual issues has focused on an important topic of political methodology: discrete choice, causal inference, Bayesian methods, election fraud, big data, and measurement. Appearing about every six months, these virtual issues have been edited by top scholars in each area. They contain important papers that have been previously published in Political Analysis, and thanks to Oxford University Press, the papers in the current virtual issue are available for free downloading. To celebrate the success of the Political Analysis virtual issue series, we take a different approach for this particular virtual issue. Instead of organizing this virtual issue around a particular methodological or substantive topic, the editors of Political Analysis have instead selected eight papers, published in the last two years, that we believe are making important contemporary contributions to political methodology. Like previous virtual issues, these papers will be available for free downloading for a limited period. The papers in this virtual issue are