Gated filter pruning via sample manifold relationships.

Filter pruning is an essential method for compressing and accelerating deep neural networks on computationally restricted devices. Despite recognizing the high correlation between filter redundancy and samples, existing methods primarily focus on independently searching for optimal subnetworks from individual input while ignoring the relationships among different inputs. In this paper, we propose a novel approach called Gated Filter Pruning based on Sample Manifold Relationships, which exploits and aligns the manifold relationships of all samples during training to obtain an optimal subnetwork. Firstly, we introduce a Gated Filter Normalization Module (GFNM) that excavates the manifold information of each sample, applicable to the operator level without adding many additional parameters. GFNM incorporates explainable control variables jointly optimized with convolutional weights, explicitly determining the competition and cooperation among filters during training. Subsequently, Manifold Regularized Pruning Module (MRPM) measures the manifold relationships between samples and subnetworks, efficiently regularizing the solution space of sample-network pairs. The manifold relationships between samples and subnetworks are aligned in training to derive an effective subnetwork for all input samples. Extensive experimental results validate the effectiveness of our method, demonstrating competitive performance in terms of accuracy and computational cost compared to state-of-the-art (SOTA) methods. [ABSTRACT FROM AUTHOR]