UFKT: Unimportant filters knowledge transfer for CNN pruning.

• This paper proposes a filter pruning approach for CNN model compression by transferring the knowledge of unimportant filters to the filters of higher importance. • Before pruning unimportant filters, a custom regularizer is utilized for knowledge transfer, which increases the gap between L1-norms of important and unimportant filters. • The effect of the penalty imposed in the custom regularizer is analyzed to justify the need for knowledge transfer before pruning. • In order to validate the robustness of the proposed framework across different CNN architectures, we experiment with five popular CNNs, namely, LeNet-5, VGG-16, ResNet-56, ResNet-110, and ResNet-50. • Experiments are performed on three benchmark datasets MNIST, CIFAR-10, and ImageNet. • An improvement over the baseline in terms of accuracy is observed even after removing 95.15%, 62.28%, and 62.39% of the Floating Point OPerations (FLOPs) from architectures LeNet-5, ResNet-56, and ResNet-110, respectively. As the deep learning models have been widely used in recent years, there is a high demand for reducing the model size in terms of memory and computation without much compromise in the model performance. Filter pruning is a very widely adopted strategy for model compression. The existing filter pruning methods identify the unimportant filters and prune them without worrying about information loss. They try to recover the same by fine-tuning the remaining filters, limiting their performance. In this paper, we tackle this problem by utilizing the knowledge from unimportant filters before pruning to minimize information loss. First, the proposed method identifies the unimportant and important filters by exploiting the lower and higher importance, respectively, using the L 1 -norm of filters. Next, the proposed custom UFKT-Reg regularizer ( R ufkt ) transfers the knowledge from unimportant filters before pruning to remaining filters, notably to a fixed number of important filters. Hence, the proposed method minimizes information loss due to the removal of unimportant filters. The experiments are conducted using the three benchmark datasets, including MNIST, CIFAR-10, and ImageNet. The proposed filter pruning method outperforms many recent state-of-the-art filter pruning methods. An improvement over the baseline in terms of accuracy is observed even after removing 95.15%, 62.28%, and 62.39% of the Floating Point OPerations (FLOPs) from architectures LeNet-5, ResNet-56, and ResNet-110, respectively. After pruning 53.25% of FLOPS from ResNet-50, only 1.02% and 0.47% of drops are observed in top-1 and top-5 accuracies, respectively. The code used in this paper will be publicly available at (https://github.com/sarvanichinthapalli/UFKT). [ABSTRACT FROM AUTHOR]