Search

Computationally intensive Inference tasks of Deep neural networks have enforced revolution of new accelerator architecture to reduce power consumption as well as latency. The key figure of merit in hardware inference accelerators is the number of multiply-and-accumulation operations per watt (MACs/W), where, the state-of-the-arts MACs/W remains several hundreds Giga-MACs/W. We propose a Tera-MACS/W neural hardware inference Accelerator (TMA) with 8-bit activations and scalable integer weights less than 1-byte. The architectures main feature is configurable neural processing element for matrix-vector operations. The proposed neural processing element has Multiplier-less Massive Parallel Processor to work without any multiplications, which makes it attractive for energy efficient high-performance neural network applications. We benchmark our systems latency, power, and performance using Alexnet trained on ImageNet. Finally, we compared our accelerators throughput and power consumption to the prior works. The proposed accelerator outperforms the state of the art in terms of energy and area achieving 2.3 TMACS/W@1.0 V, 65 nm CMOS technology.

Recent progress in parallel computing machines, deep neural networks, and training techniques have contributed to the significant advances in artificial intelligence (AI) with respect to tasks such as object classification, speech recognition, and natural language processing. The development of such deep learning-based techniques has enabled AI-based networks to outperform humans in the recognition of objects in images. The graphics processing unit (GPU) has been the primary component used for parallel computing during the inference and training phases of deep neural networks. In this study, we perform training using a desktop or a server with one or more GPUs and inference using hardware accelerators on embedded devices. Performance, power consumption, and requirements of embedded system present major hindrances to the application of deep neural network-based systems using embedded controllers such as drones, AI speakers, and autonomous vehicles. In particular, power consumption of a commercial GPU commonly surpasses the power budget of a stand-alone embedded system. To reduce the power consumption of hardware accelerators, reductions in the precision of input data and hardware weight have become popular topics of research in this field. However, precision and accuracy share a trade-off relationship. Therefore, it is essential to optimize precision in a manner that does not degrade the accuracy of the inference process. In this context, the primary issues faced by hardware accelerators are loss of accuracy and high power consumption.

Advances in the architecture of inference accelerators and quantization techniques of neural networks allow effective on-device inference in embedded devices. Privacy issues for user data, as well as increasing needs of user-specific services, have led to a need for on-device training. The dot product operation required in backpropagation can be computed efficiently by multiplier–accumulators (MACs) in the inference accelerator if forward and backward propagation of the neural network have the same precision. This chapter introduces a quantization technique to enable computation by the digital neuron inference accelerator with the same precision as that using the forward path. Updating the 5-bit weights with gradients of higher precision is challenging. To address this issue, this chapter also introduces a probabilistic weight update. It also describes the hardware implementation of the probabilistic weight-update scheme. The proposed training technique achieves 98.15% recognition accuracy on the MNIST dataset.