Daniele Mandrioli, Susan L Norris, Kathrine Sørensen, Tracey J. Woodruff, Alexis Descatha, Tom Loney, Denise Torreão Corrêa da Silva, Patrice Sutton, Lisa Bero, Emilie van Deventer, Marília Silva Paulo, Vivi Schlünssen, Paul T.J. Scheepers, Ellen K. Silbergeld, Rebecca L. Morgan, Daria Sgargi, Diana Gagliardi, Frank Pega, Lode Godderis, Daniela Vianna Pachito, Thomas Tenkate, Yuka Ujita, Claudine Backes, Alberto Modenese, Organisation Mondiale de la Santé / World Health Organization Office (OMS / WHO), Vieillissement et Maladies chroniques : approches épidémiologique et de santé publique (VIMA), Université de Versailles Saint-Quentin-en-Yvelines (UVSQ)-Institut National de la Santé et de la Recherche Médicale (INSERM), Cohortes épidémiologiques en population (CONSTANCES), Université de Versailles Saint-Quentin-en-Yvelines (UVSQ)-Institut National de la Santé et de la Recherche Médicale (INSERM)-Université Paris-Saclay-Université de Paris (UP), Department of Public Health [Copenhagen, Denmark] (Danish Ramazzini Centre), Aarhus University [Aarhus]-The National Research Center for Work Environment [Copenhagen, Denmark], Johns Hopkins University (JHU), Department of Obstetrics, Gynecology and Reproductive Sciences, University of California [San Francisco] (UCSF), University of California-University of California, 1 E11 OH0010676-02, World Health Organization, Université de Versailles Saint-Quentin-en-Yvelines (UVSQ), Institut National de la Santé et de la Recherche Médicale (INSERM)-Université de Versailles Saint-Quentin-en-Yvelines (UVSQ), UMS011 Cohortes épidémiologiques en population (CONSTANCES), Unité de pathologie professionnelle, Université de Versailles Saint-Quentin-en-Yvelines (UVSQ)-Assistance publique - Hôpitaux de Paris (AP-HP) (APHP)-Hôpital Raymond Poincaré [AP-HP], IRSET - Épidémiologie en santé au travail et ergonomie (IRSET-ESTER), Institut de recherche en santé, environnement et travail (Irset), Université d'Angers (UA)-Université de Rennes 1 (UR1), Université de Rennes (UNIV-RENNES)-Université de Rennes (UNIV-RENNES)-École des Hautes Études en Santé Publique [EHESP] (EHESP)-Institut National de la Santé et de la Recherche Médicale (INSERM)-Structure Fédérative de Recherche en Biologie et Santé de Rennes ( Biosit : Biologie - Santé - Innovation Technologique )-Université d'Angers (UA)-Université de Rennes 1 (UR1), Université de Rennes (UNIV-RENNES)-Université de Rennes (UNIV-RENNES)-École des Hautes Études en Santé Publique [EHESP] (EHESP)-Institut National de la Santé et de la Recherche Médicale (INSERM)-Structure Fédérative de Recherche en Biologie et Santé de Rennes ( Biosit : Biologie - Santé - Innovation Technologique ), and Aarhus University [Copenhagen, Denmark]-The National Research Center for Work Environment [Copenhagen, Denmark]
Background The World Health Organization (WHO) and the International Labour Organization (ILO) are developing joint estimates of the work-related burden of disease and injury (WHO/ILO Joint Estimates). For this, systematic reviews of studies estimating the prevalence of exposure to selected occupational risk factors will be conducted to provide input data for estimations of the number of exposed workers. A critical part of systematic review methods is to assess risk of bias (RoB) of individual studies. In this article, we present and describe the development of such a tool, called the Risk of Bias in Studies estimating Prevalence of Exposure to Occupational risk factors (RoB-SPEO) tool; report results from RoB-SPEO's pilot testing; note RoB-SPEO's limitations; and suggest how the tool might be tested and developed further. Methods Selected existing RoB tools used in environmental and occupational health systematic reviews were reviewed and analysed. From existing tools, we identified domains for the new tool and, if necessary, added new domains. For each domain, we then identified and integrated components from the existing tools (i.e. instructions, domains, guiding questions, considerations, ratings and rating criteria), and, if necessary, we developed new components. Finally, we elicited feedback from other systematic review methodologists and exposure scientists and agreed upon RoB-SPEO. Nine experts pilot tested RoB-SPEO, and we calculated a raw measure of inter-rater agreement (Pi) for each of its domain, rating Pi 0.80 as almost perfect agreement. Results Our review found no standard tool for assessing RoB in prevalence studies of exposure to occupational risk factors. We identified six existing tools for environmental and occupational health systematic reviews and found that their components for assessing RoB differ considerably. With the new RoB-SPEO tool, assessors judge RoB for each of eight domains: (1) bias in selection of participants into the study; (2) bias due to a lack of blinding of study personnel; (3) bias due to exposure misclassification; (4) bias due to incomplete exposure data; (5) bias due to conflict of interest; (6) bias due to selective reporting of exposures; (7) bias due to difference in numerator and denominator; and (8) other bias. The RoB-SPEO's ratings are low, probably low, probably high, high or no information. Pilot testing of the RoB-SPEO tool found substantial inter-rater agreement for six domains (range of Pi for these domains: 0.51–0.80), but poor agreement for two domains (i.e. Pi of 0.31 and 0.33 for biases due to incomplete exposure data and in selection of participants into the study, respectively). Limitations of RoB-SPEO include that it has not yet been fully performance-tested. Conclusions We developed the RoB-SPEO tool for assessing RoB in prevalence studies of exposure to occupational risk factors. The tool will be applied and its performance tested in the ongoing systematic reviews for the WHO/ILO Joint Estimates., Highlights • WHO and ILO are systematically reviewing occupational exposure prevalence studies. • Assessing risk of bias (RoB) in individual studies is a core step of systematic reviews. • We developed RoB-SPEO, a tool to assess RoB in occupational exposure prevalence studies. • The performance of RoB-SPEO will be tested in the WHO/ILO systematic reviews.