Back to Search
Start Over
The Data Representativeness Criterion: Predicting the Performance of Supervised Classification Based on Data Set Similarity
- Source :
- PLoS ONE, Vol 15, Iss 8, p e0237009 (2020), PLoS ONE, PLoS ONE, 15(8):e0237009. Public Library of Science
- Publication Year :
- 2020
-
Abstract
- In a broad range of fields it may be desirable to reuse a supervised classification algorithm and apply it to a new data set. However, generalization of such an algorithm and thus achieving a similar classification performance is only possible when the training data used to build the algorithm is similar to new unseen data one wishes to apply it to. It is often unknown in advance how an algorithm will perform on new unseen data, being a crucial reason for not deploying an algorithm at all. Therefore, tools are needed to measure the similarity of data sets. In this paper, we propose the Data Representativeness Criterion (DRC) to determine how representative a training data set is of a new unseen data set. We present a proof of principle, to see whether the DRC can quantify the similarity of data sets and whether the DRC relates to the performance of a supervised classification algorithm. We compared a number of magnetic resonance imaging (MRI) data sets, ranging from subtle to severe difference is acquisition parameters. Results indicate that, based on the similarity of data sets, the DRC is able to give an indication as to when the performance of a supervised classifier decreases. The strictness of the DRC can be set by the user, depending on what one considers to be an acceptable underperformance.<br />12 pages, 6 figures
- Subjects :
- Central Nervous System
Data Analysis
FOS: Computer and information sciences
Computer science
Computer Vision
Computer Vision and Pattern Recognition (cs.CV)
Bayesian inference
Computer Science - Computer Vision and Pattern Recognition
02 engineering and technology
computer.software_genre
Nervous System
Diagnostic Radiology
Pattern Recognition, Automated
Machine Learning
DESIGN
Medicine and Health Sciences
Image Processing, Computer-Assisted
0202 electrical engineering, electronic engineering, information engineering
BRAIN
Multidisciplinary
Training set
Radiology and Imaging
Applied Mathematics
Simulation and Modeling
Magnetic Resonance Imaging
Multidisciplinary Sciences
Data Acquisition
Data Interpretation, Statistical
Physical Sciences
Science & Technology - Other Topics
Medicine
020201 artificial intelligence & image processing
Supervised Machine Learning
Anatomy
Transfer of learning
Algorithms
Data set similarity
Research Article
Computer and Information Sciences
Imaging Techniques
Science
Research and Analysis Methods
Machine learning
Proof of Concept Study
Representativeness heuristic
Methodology (stat.ME)
Machine Learning Algorithms
Diagnostic Medicine
Artificial Intelligence
020204 information systems
Humans
Statistics - Methodology
Science & Technology
business.industry
Biology and Life Sciences
Probability Theory
Probability Distribution
Transfer learning
Data set
Artificial intelligence
business
Classifier (UML)
computer
Mathematics
Subjects
Details
- Language :
- English
- ISSN :
- 19326203
- Database :
- OpenAIRE
- Journal :
- PLoS ONE, Vol 15, Iss 8, p e0237009 (2020), PLoS ONE, PLoS ONE, 15(8):e0237009. Public Library of Science
- Accession number :
- edsair.doi.dedup.....89322265f97c89d5a13c94891a4135df