1. Curated Data In — Trustworthy In Silico Models Out: The Impact of Data Quality on the Reliability of Artificial Intelligence Models as Alternatives to Animal Testing
- Author
-
Scott S. Auerbach, Nicole Kleinstreuer, Eugene N. Muratov, John P. Rooney, Charles Schmitt, Ivan Rusyn, Alexander Tropsha, and Vinicius M. Alves
- Subjects
0303 health sciences ,Computational model ,Data curation ,Computer science ,business.industry ,media_common.quotation_subject ,In silico ,Alternatives to animal testing ,General Medicine ,010501 environmental sciences ,Toxicology ,01 natural sciences ,General Biochemistry, Genetics and Molecular Biology ,03 medical and health sciences ,Medical Laboratory Technology ,Robustness (computer science) ,Data quality ,Quality (business) ,Artificial intelligence ,business ,Reliability (statistics) ,030304 developmental biology ,0105 earth and related environmental sciences ,media_common - Abstract
New Approach Methodologies (NAMs) that employ artificial intelligence (AI) for predicting adverse effects of chemicals have generated optimistic expectations as alternatives to animal testing. However, the major underappreciated challenge in developing robust and predictive AI models is the impact of the quality of the input data on the model accuracy. Indeed, poor data reproducibility and quality have been frequently cited as factors contributing to the crisis in biomedical research, as well as similar shortcomings in the fields of toxicology and chemistry. In this article, we review the most recent efforts to improve confidence in the robustness of toxicological data and investigate the impact that data curation has on the confidence in model predictions. We also present two case studies demonstrating the effect of data curation on the performance of AI models for predicting skin sensitisation and skin irritation. We show that, whereas models generated with uncurated data had a 7–24% higher correct classification rate (CCR), the perceived performance was, in fact, inflated owing to the high number of duplicates in the training set. We assert that data curation is a critical step in building computational models, to help ensure that reliable predictions of chemical toxicity are achieved through use of the models.
- Published
- 2021
- Full Text
- View/download PDF