Back to Search
Start Over
Filter Sketch for Network Pruning.
- Source :
-
IEEE Transactions on Neural Networks & Learning Systems . Dec2022, Vol. 33 Issue 12, p7091-7100. 10p. - Publication Year :
- 2022
-
Abstract
- We propose a novel network pruning approach by information preserving of pretrained network weights (filters). Network pruning with the information preserving is formulated as a matrix sketch problem, which is efficiently solved by the off-the-shelf frequent direction method. Our approach, referred to as FilterSketch, encodes the second-order information of pretrained weights, which enables the representation capacity of pruned networks to be recovered with a simple fine-tuning procedure. FilterSketch requires neither training from scratch nor data-driven iterative optimization, leading to a several-orders-of-magnitude reduction of time cost in the optimization of pruning. Experiments on CIFAR-10 show that FilterSketch reduces 63.3% of floating-point operations (FLOPs) and prunes 59.9% of network parameters with negligible accuracy cost for ResNet-110. On ILSVRC-2012, it reduces 45.5% of FLOPs and removes 43.0% of parameters with only 0.69% accuracy drop for ResNet-50. Our code and pruned models can be found at https://github.com/lmbxmu/FilterSketch. [ABSTRACT FROM AUTHOR]
- Subjects :
- *COST control
*COVARIANCE matrices
*RECOMMENDER systems
*INFORMATION filtering
Subjects
Details
- Language :
- English
- ISSN :
- 2162237X
- Volume :
- 33
- Issue :
- 12
- Database :
- Academic Search Index
- Journal :
- IEEE Transactions on Neural Networks & Learning Systems
- Publication Type :
- Periodical
- Accession number :
- 160690297
- Full Text :
- https://doi.org/10.1109/TNNLS.2021.3084206