Misapplied Metrics: Variation in the h-index within and between disciplines

Scholars and university administrators have a vested interest in building equitable valuation systems of academic work for both practical (e.g., resource distribution) and more lofty purposes (e.g., what constitutes “good” research). Well-established inequalities in science pose a difficult challenge to those interested in constructing a parsimonious and fair method for valuation as stratification occurs within academic disciplines, but also between them. Despite warnings against the practice, the popular h-index has been formally used as one such metric of valuation. In this article, we use the case of the h-index to examine how within and between discipline inequalities extend from the reliance of metrics, an illustration of the risk involved in the so-called “tyranny of metrics.” Using data from over 42,000 high performing scientists across 120 disciplines, we construct multilevel models predicting the h-index. Results suggest significant within-discipline variation in several forms, including a female penalty, as well as significant between discipline variation. Conclusions include recommendations to avoid using the h-index or similar metrics for valuation purposes.