Search

We present a new PyTorch-based framework for neural network compression with fine-tuning named Neural Network Compression Framework (NNCF) (https://github.com/openvinotoolkit/nncf) . It leverages recent advances of various network compression methods and implements some of them, namely quantization, sparsity, filter pruning and binarization. These methods allow producing more hardware-friendly models that can be efficiently run on general-purpose hardware computation units (CPU, GPU) or specialized deep learning accelerators. We show that the implemented methods and their combinations can be successfully applied to a wide range of architectures and tasks to accelerate inference while preserving the original model’s accuracy. The framework can be used in conjunction with the supplied training samples or as a standalone package that can be seamlessly integrated into the existing training code with minimal adaptations.

Despite the achievements of recent binarization methods on reducing the performance degradation of Binary Neural Networks (BNNs), gradient mismatching caused by the Straight-Through-Estimator (STE) still dominates quantized networks. This paper proposes a meta-based quantizer named QuantNet, which utilizes a differentiable sub-network to directly binarize the full-precision weights without resorting to STE and any learnable gradient estimators. Our method not only solves the problem of gradient mismatching, but also reduces the impact of discretization errors, caused by the binarizing operation in the deployment, on performance. Generally, the proposed algorithm is implemented within a fully differentiable framework, and is easily extended to the general network quantization with any bits. The quantitative experiments on CIFAR-100 and ImageNet demonstrate that QuantNet achieves the significant improvements comparing with previous binarization methods, and even bridges gaps of accuracies between binarized models and full-precision models.

A sparse dynamic binary neural network is characterized by local binary connection and can exhibit various binary periodic orbits. First, we consider a condition of sparse connection parameters that guarantees storage and switching of binary periodic orbits. Second, we give two examples of the network that realize the storage and stitching of two periodic orbits related to insect gaits patterns. Presenting an FPGA-based hardware, typical phenomena are confirmed experimentally.

This paper studies hardware implementation of a simple dynamic binary neural network that can generate various periodic orbits. The network is characterized by local binary connection and signum activation function. First, using a simple feature quantity, stability of a target periodic orbit is considered. Second, using a FPGA board, a test circuit is implemented. The signum activation function is realized by a majority decision circuit and the binary connection is realized by switches and inverters. The circuit operation is confirmed experimentally.

The dynamic binary neural network is characterized by ternary connection parameters and can generate various binary periodic orbits. This paper considers two interesting problems based on typical examples. First, effect of connection sparsity on stability of target periodic orbits is considered: adding branches adequately to the most sparse network, stability of the periodic orbits can be reinforced. Second, fault tolerance of the network is considered: cutting one branch from the network, storage and stability of the periodic orbits are preserved in high probability.