Training with scaled logits to alleviate class-level over-fitting in few-shot learning.

Deep learning methods are criticized for the requirement of large set of labeled training data. Therefore, few-shot learning (FSL), which enables fast learning with only a few labeled examples, draws increasing attention. In FSL, the model is trained on base classes to learn the meta-knowledge, which is then applied to novel classes. The generalization from base classes to novel classes in FSL suffers from the class-level over-fitting problem. Generally, in traditional classification tasks, training samples and test samples are from the same class set, where over-fitting happens at the sample level. However, in FSL, the model is evaluated with classification tasks on novel classes. Thus, good fitting on base classes does not guarantee a good generalization to novel classes. In this paper, we reveal the class-level over-fitting problem in FSL and provide an explanation of the cause of this problem. Based on the explanation, we argue that simply scaling the logits of classifier during training can alleviate the class-level over-fitting problem, and analyze how scaling logits (SL) alleviates class-level over-fitting based on gradient back-propagation. Extensive experiments show that SL boosts the performance to the extent of 14% on four popular benchmark datasets. Further, SL also demonstrates its effectiveness on confidence calibration. [ABSTRACT FROM AUTHOR]