Reliability generalization meta-analysis of orthorexia nervosa using the ORTO-11/12/15/R scale in all populations and language versions

Abstract Background The ORTO scale was developed in 2004 as a self-report questionnaire to assess symptoms of orthorexia nervosa (ON). ON is an unhealthy preoccupation with eating healthy food. The scale aims to measure obsessive attitudes and behaviors related to the selection, purchase, preparation, and consumption of pure, healthy food. Since its development, the ORTO-15 has been adapted into several shorter versions. The objective was to conduct a reliability generalization meta-analysis of the ORTO scale and its variant versions in all populations and languages. Methods A systematic literature search was conducted to identify studies reporting the internal consistency of ORTO. Random-effect models were used to evaluate summary statistics of reliability coefficients, weighting the coefficients by the inverse variance using the restricted maximum likelihood method. The heterogeneity among the reliability coefficients was evaluated and assessed using numerous statistical metrics. The tau (τ), tau2 (τ2), I2, H2, R2, df, and the Q-statistic are among those obtained. Meta-regression analyses were used to examine moderators such as age and sex. Results Twenty-one studies (k = 21) involving 11,167 participants (n = 11,167) were analyzed. The overall effect estimate on internal consistency was 0.59 (95% CI 0.49–0.68), with a minimum reliability coefficient of 0.23 and a maximum reliability coefficient of 0.83. The heterogeneity statistics were found to have an I2 of 99.31%, which suggested high heterogeneity owing to a decrease in the confidence interval (95% CI) and an increase in variability. Sensitivity analysis revealed that a few studies strongly influenced the overall estimate. Egger’s test suggested possible publication bias. Neither age nor sex significantly moderated reliability via meta-regression. Conclusions The ORTO scale has a relatively low pooled reliability coefficient. Alternative ON assessment tools with enhanced psychometric properties are needed. Clinicians should not base diagnoses or treatment decisions on ORTO alone. Comprehensive psychiatric assessment is essential for accurate ON evaluation.