Output augmentation works well without any domain knowledge

Data augmentation is a method to compensate a lack of sufficient amount of training data by increasing variations of the training data. It is also used even when there is a huge amount of training data to improve a generalization performance on the test data. In this paper, we propose a new method, Output-Augmentation (OA), which we use to improve the generalization performance without data augmentation. It augments each original output (but not input) and produces an arbitrary number of outputs which average to the original output. Updating the parameters is done by using the gradient over both the original and the augmented outputs. We conclude that the proposed novel method strongly complements the existing ones by showing empirical evaluations where we see improvements of the generalization performance in the task of image classification.