1. Evaluating the Robustness of ML Models to Out-of-Distribution Data Through Similarity Analysis
- Author
-
Lindén, Joakim, Forsberg, Håkan, Daneshtalab, Masoud, Söderquist, Ingemar, Lindén, Joakim, Forsberg, Håkan, Daneshtalab, Masoud, and Söderquist, Ingemar
- Abstract
In Machine Learning systems, several factors impact the performance of a trained model. The most important ones include model architecture, the amount of training time, the dataset size and diversity. We present a method for analyzing datasets from a use-case scenario perspective, detecting and quantifying out-of-distribution (OOD) data on dataset level. Our main contribution is the novel use of similarity metrics for the evaluation of the robustness of a model by introducing relative Fréchet Inception Distance (FID) and relative Kernel Inception Distance (KID) measures. These relative measures are relative to a baseline in-distribution dataset and are used to estimate how the model will perform on OOD data (i.e. estimate the model accuracy drop). We find a correlation between our proposed relative FID/relative KID measure and the drop in Average Precision (AP) accuracy on unseen data., Part of ISBN 9783031429408QC 20231009 more...
- Published
- 2023
- Full Text
- View/download PDF