Back to Search
Start Over
Inconsistency in the use of the term 'validation' in studies reporting the performance of deep learning algorithms in providing diagnosis from medical imaging
- Source :
- PLoS ONE, PLoS ONE, Vol 15, Iss 9, p e0238908 (2020)
- Publication Year :
- 2020
- Publisher :
- Public Library of Science (PLoS), 2020.
-
Abstract
- BackgroundThe development of deep learning (DL) algorithms is a three-step process-training, tuning, and testing. Studies are inconsistent in the use of the term "validation", with some using it to refer to tuning and others testing, which hinders accurate delivery of information and may inadvertently exaggerate the performance of DL algorithms. We investigated the extent of inconsistency in usage of the term "validation" in studies on the accuracy of DL algorithms in providing diagnosis from medical imaging.Methods and findingsWe analyzed the full texts of research papers cited in two recent systematic reviews. The papers were categorized according to whether the term "validation" was used to refer to tuning alone, both tuning and testing, or testing alone. We analyzed whether paper characteristics (i.e., journal category, field of study, year of print publication, journal impact factor [JIF], and nature of test data) were associated with the usage of the terminology using multivariable logistic regression analysis with generalized estimating equations. Of 201 papers published in 125 journals, 118 (58.7%), 9 (4.5%), and 74 (36.8%) used the term to refer to tuning alone, both tuning and testing, and testing alone, respectively. A weak association was noted between higher JIF and using the term to refer to testing (i.e., testing alone or both tuning and testing) instead of tuning alone (vs. JIF 10: adjusted odds ratio 2.41, P = 0.089). Journal category, field of study, year of print publication, and nature of test data were not significantly associated with the terminology usage.ConclusionsExisting literature has a significant degree of inconsistency in using the term "validation" when referring to the steps in DL algorithm development. Efforts are needed to improve the accuracy and clarity in the terminology usage.
- Subjects :
- Diagnostic Imaging
Computer and Information Sciences
Medical Journals
Systematic Reviews
Computer science
Science
Validation Studies as Topic
Research and Analysis Methods
030218 nuclear medicine & medical imaging
Machine Learning
Machine Learning Algorithms
03 medical and health sciences
Deep Learning
0302 clinical medicine
Artificial Intelligence
Diagnostic Medicine
Medicine and Health Sciences
Humans
030212 general & internal medicine
Scientific Publishing
Multidisciplinary
Applied Mathematics
Simulation and Modeling
Research Assessment
Term (time)
Systematic review
Bibliometrics
Physical Sciences
Medicine
Journal Impact Factor
Periodicals as Topic
Medical Humanities
Algorithm
Mathematics
Algorithms
Research Article
Test data
Subjects
Details
- ISSN :
- 19326203
- Volume :
- 15
- Database :
- OpenAIRE
- Journal :
- PLOS ONE
- Accession number :
- edsair.doi.dedup.....4b40e8fa92de00b168c8851e133c301b