1. Combining independent p-values in replicability analysis: a comparative study.
- Author
-
Hoang, Anh-Tuan and Dickhaus, Thorsten
- Subjects
- *
DISTRIBUTION (Probability theory) , *NULL hypothesis , *COMPARATIVE studies , *FISHER exact test - Abstract
Given a family of null hypotheses H 1 , ... , H s , we are interested in the hypothesis H s γ that at most γ − 1 of these null hypotheses are false. Assuming that the corresponding p-values are independent, we are investigating combined p-values that are valid for testing H s γ . In various settings in which H s γ is false, we determine which combined p-value works well in which setting. Via simulations, we find that the Stouffer method works well if the null p-values are uniformly distributed and the signal strength is low, and the Fisher method works better if the null p-values are conservative, i.e. stochastically larger than the uniform distribution. The minimum method works well if the evidence for the rejection of H s γ is focused on only a few non-null p-values, especially if the null p-values are conservative. Methods that incorporate the combination of e-values work well if the null hypotheses H 1 , ... , H s are simple. Finally, we also consider nonparametric permutation-based combination methods. They are useful for addressing the conservativity of parametric p-values which are computed under least favourable parameter configurations. [ABSTRACT FROM AUTHOR]
- Published
- 2022
- Full Text
- View/download PDF