Search

Statistical significance tests done with low statistical power levels can result in reports of exaggerated effect sizes. Funnel graphs can show these exaggerations for a given area of research by revealing a negative correlation between a study's sample size on the y-axis and the effect size obtain by the study on the x-axis. A recent paper by Fritz, Scherndl, and Kühberger (2013) recommended that effect sizes should not be reported when there is a negative correlation between sample size and effect size for a given research area. This recommendation fails to consider magnitudes of the negative correlations and thereby misses the opportunity to mitigate effect size exaggerations and approximate more correct effect size estimates. This comment explains both the incorrectness of the recommendation and the approach and calculations necessary to correct effect size estimates. [ABSTRACT FROM AUTHOR]

Kühberger, Scherndl, and Fritz commented on an attempt by Bradley and Brand to adjust sets of exaggerated effect sizes reported in literatures associated with underpowered Null Hypothesis Statistical Tests (NHST). Their comment highlighted two important issues: (a) the senior author, Bradley, made an error in presenting the correction formula, and (b) there is an inherent incompatibility between inferential statistics and accurate measurement. The proper formula is presented here with evidence that the formula is relatively accurate in appropriately estimating effect sizes that have been exaggerated through NHST. The term relatively accurate is used since power cannot be 100%, and thus any attempt to estimate a true effect size will be out by some relationship between the alpha level, power, and of course statistical variability of the estimates. [ABSTRACT FROM AUTHOR]