Analytic flexibility in laboratory aggression paradigms: Relations with personality traits vary (slightly) by operationalization of aggression

Competitive reaction time tasks (CRTTs) have been used widely in social science research, but recent criticism has been directed at the flexible quantification strategies used with this methodology. One estimate suggests that over 150 different quantification strategies have been used, and there is evidence to suggest that different operationalizations can affect the results and interpretations of experiments using CRTTs (Elson, Mohseni, Breuer, Scharkow, & Quandt, 2014). In the current investigation, we re-analyze data from four extant samples from two different sites (total N = 600) to examine how the relations between a range of personality traits and aggression vary based on how aggression is operationalized. Our results suggest that there is a modest degree of heterogeneity in effect size and direction for these relations, and that effect size and direction were more consistent for traits more generally related to lab aggression (e.g., psychopathy, low FFM Agreeableness). Additionally, profile matching analyses suggest that different operationalizations yield strong intraclass correlations with one another. These results were consistent across site, methodology, and type of sample, suggesting that these issues are likely generalizable across most labs using CRTTs. We conclude with suggestions for future directions, particularly emphasizing the need for adequately-powered samples, and for researchers to preregister a justified plan for how CRTT data will be analyzed.