1. How many participants do I need to test an interaction? Conducting an appropriate power analysis and achieving sufficient power to detect an interaction
- Author
-
Nicolas Sommet, David Laurence Weissman, Nicolas Cheutin, and Andrew Elliot
- Abstract
Power analysis for first-order interactions poses two challenges: (i) Conducting an appropriate power analysis is difficult, because the typical expected effect size of an interaction depends on its shape; (ii) Achieving sufficient power is difficult, because interactions are often modest in size. This paper consists of three parts. PART 1 addresses the first challenge. We first use a fictional study to explain the difference between power analyses for interactions and main effects. Then, we introduce an intuitive taxonomy of 12 types of interactions based on the shape of the interaction (reversed, fully attenuated, partially attenuated) and the size of the simple slopes (median, smaller, larger), and we offer mathematically-derived sample size recommendations to detect each interaction with a power of .80/.90/.95 (for two-tailed tests in between-participant designs). PART 2 addresses the second challenge. We first describe a preregistered meta-study (159 studies from recent articles in influential psychology journals) showing that the median power to detect interactions of a typical size is .18. Then, we use simulations (≈ 900,000,000 datasets) to generate power curves for the 12 types of interactions, and test three approaches to increase power without increasing sample size: (i) preregistering one-tailed tests (+21% gain), (ii) using a mixed design (+75% gain), and (ii) preregistering contrast analysis for a fully attenuated interaction (+62% gain). PART 3 introduces INT×Power, a web-application that enables users to draw their interaction and determine the sample size needed to reach the power of their choice, with the option of using/combining these approaches: www.intxpower.com.
- Published
- 2022
- Full Text
- View/download PDF