Search

Convolution neural network (CNN) with pruning techniques has shown remarkable prospects in electrocardiogram (ECG) classification. However, efficiently deploying the existing pruned neural network to wearable devices for ECG classification is a great challenge due to the limited hardware resource and randomly distributed sparse weights. To address this issue, an efficient unstructured sparse CNN accelerator is proposed in this paper. A tile-first dataflow with compressed data storage format is presented to skip zero weight multiplications and increase the computing efficiency during inference of small-scale model with large sparsity. The two-level weight index matching structure in the dataflow exploits shifting operation to select valid data pairs and maintain the fully-pipelined calculation process. A configurable processing element (PE) array with 32-bit instruction control is proposed to increase the flexibility of the accelerator. Verified in FPGA and post-synthesis simulations in SMIC 40nm process, the proposed sparse CNN accelerator consumes $3.93~\mu $ J/classification at 2MHz clock frequency and it achieves an averaged ECG classification accuracy of 98.99%. A computing efficiency of 118.75% is realized which is improved by 48% compared to the dense baseline. In brief, the proposed efficient CNN accelerator is especially suitable for wearable ECG classification device. [ABSTRACT FROM AUTHOR]

This paper presents an advanced offline scheduling scheme to improve the accelerator efficiency, especially for the highly-pruned convolutional neural networks (HP-CNNs). Based on the existing outlier-aware accelerator design, we demonstrate the efficiency drop of HP-CNN processing for the first time, and element-wise channel merging is proposed to make a dense processing sequence even for the highly-pruned model. The dedicated hardware architecture is also presented to process the merged channels with the minimum hardware-level overheads, improving the energy efficiency for handling HP-CNNs by preserving the hardware utilization. We further investigate the optimal size of accumulator and multiplexer, in addition to the number of merged channels, exploiting the attractive energy-performance trade-offs. As a result, unlike the practical HP-CNNs for the on-device solutions, the proposed method enhances the overall efficiency by up to 33% compared to the state-of-the-art schemes. [ABSTRACT FROM AUTHOR]

This paper presents a block-based embedded convolutional neural network (CNN) for gesture classification on field-programmable gate array (FPGA) in real time. Gesture recognition is an important tool to spontaneous interact with human machine interface. Many CNN architectures using RGB images have been proposed for gesture classification. RGB based gesture classification may cause incorrect results under insufficient light or similar gestures. In addition, most of the CNN architectures cannot run in real time on edge devices due to their large number of parameters and DRAM data access. In this paper, a block-based CNN using RGB-D data is proposed for gesture classification. Adding depth images to RGB images boots the classification accuracy. A CNN architecture with block-based feature maps is built for embedded FPGA implementations. The total number of parameters of the proposed RGB-D embedded CNN (eCNN) model is only 0.17M and it achieves 99.96% and 99.88% accuracy with 32-bit floating point and 8-bit fixed point implementation for America Sign Language (ASL) data set. The RTL simulation of the proposed eCNN model has the average inference speed of 0.171 milliseconds at frequency of 250MHz for a single pair RGB-D image. Implemented on a FPGA integrated with Microsoft Kinect v2 achieve an inference time in 19.42 ms which achieves high accuracy and real-time performance. [ABSTRACT FROM AUTHOR]

Residual block is a very common component in recent state-of-the art CNNs such as EfficientNet/EfficientDet. Shortcut data accounts for nearly 40% of feature-maps access in ResNet152. Most of the previous DNN compilers/accelerators ignore the shortcut data optimization. This paper presents ShortcutFusion, an optimization tool for FPGA-based accelerator with a reuse-aware static memory allocation for shortcut data, to maximize on-chip data reuse given resource constraints. From TensorFlow DNN models, the proposed design generates instruction sets for a group of nodes which uses an optimized data reuse for each residual block. The accelerator design implemented on the Xilinx KCU1500 FPGA card $2.8\times $ faster and $9.9\times $ more power efficient than NVIDIA RTX 2080 Ti for $256\times 256$ input size. Compared to the result from baseline, in which the weights/inputs/outputs are accessed from the off-chip memory exactly once per each layer, ShortcutFusion reduces the DRAM access by 47.8-84.8% for RetinaNet, Yolov3, ResNet152, and EfficientNet. Given a similar buffer size to ShortcutMining, which also “mine” the shortcut data in hardware, the proposed work reduces off-chip access for feature-maps $5.27\times $ while accessing weight from off-chip memory exactly once. [ABSTRACT FROM AUTHOR]

In-memory computing (IMC) is an effective solution for energy-efficient artificial intelligence applications. Analog IMC amortizes the power consumption of multiple sensing amplifiers with an analog-to-digital converter (ADC) and simultaneously completes the calculation of multi-line data with a high parallelism degree. Based on a universal one-transistor one-magnetic tunnel junction (MTJ) spin transfer torque magnetic RAM (STT-MRAM) cell, this paper demonstrates a novel tunneling magnetoresistance (TMR) ratio magnifying method to realize analog IMC. Previous concerns including low TMR ratio and analog calculation nonlinearity are addressed using device-circuit interaction. The TMR is magnified $7500\times $ using a latch structure in combination with the device. Peripheral circuits are minimally modified to enable in-memory matrix-vector multiplication. A current mirror with a feedback structure is implemented to enhance analog computing linearity and calculation accuracy. The proposed design maximumly supports 1024 2-bit input and 1-bit weight multiply-and-accumulate (MAC) computations simultaneously. The proposal is simulated using the 28-nm CMOS process and MTJ compact model. The integral nonlinearity is reduced by 57.6% compared with the conventional structure. 9.47-25.4 TOPS/W is realized with 2-bit input, 1-bit weight, and 4-bit output convolution neural network (CNN). [ABSTRACT FROM AUTHOR]

Existing deep convolutional neural networks (CNNs) generate massive interlayer feature data during network inference. To maintain real-time processing in embedded systems, large on-chip memory is required to buffer the interlayer feature maps. In this paper, we propose an efficient hardware accelerator with an interlayer feature compression technique to significantly reduce the required on-chip memory size and off-chip memory access bandwidth. The accelerator compresses interlayer feature maps through transforming the stored data into frequency domain using hardware-implemented $8\times 8$ discrete cosine transform (DCT). The high-frequency components are removed after the DCT through quantization. Sparse matrix compression is utilized to further compress the interlayer feature maps. The on-chip memory allocation scheme is designed to support dynamic configuration of the feature map buffer size and scratch pad size according to different network-layer requirements. The hardware accelerator combines compression, decompression, and CNN acceleration into one computing stream, achieving minimal compressing and processing delay. A prototype accelerator is implemented on an FPGA platform and also synthesized in TSMC 28-nm COMS technology. It achieves 403GOPS peak throughput and $1.4\times \sim 3.3\times $ interlayer feature map reduction by adding light hardware area overhead, making it a promising hardware accelerator for intelligent IoT devices. [ABSTRACT FROM AUTHOR]

With the growing applications of Graph Convolutional Networks (GCN), there is also an increasing demand for its efficient hardware acceleration. Compared with CNN tasks, GCN tasks have new challenges such as randomness, sparsity, and nonuniformity, which will lead to poor performance of previous AI accelerators. In this paper, we propose DyGA, a hardware-efficient GCN accelerator, which is featured by strategies of graph partitioning, customized storage policy, traffic-aware dynamic scheduling, and out-of-order execution. Synthesized and evaluated under TSMC 28-nm, the accelerator achieves an average throughput of over 95% of its peak performance with full utilization of hardware on representative graph data sets. Having a high area-efficiency with 0.217 GOPS/K-logic-gates and 8.06 GOPS/KB-PE-buffer, and thus an energy-efficiency of 384GOPS/W, the proposed accelerator outperforms previous state-of-the-art works in the sparse data processing. [ABSTRACT FROM AUTHOR]

It remains challenging for a Convolutional Neural Network (CNN) accelerator to maintain high hardware utilization and low processing latency with restricted on-chip memory. This paper presents an In-Execution Configuration Accelerator (IECA) that realizes an efficient control scheme, exploring architectural data reuse, unified in-execution controlling, and pipelined latency hiding to minimize configuration overhead out of the computation scope. The proposed IECA achieves row-wise convolution with tiny distributed buffers and reduces the size of total on-chip memory by removing 40% of redundant memory storage with shared delay chains. By exploiting a reconfigurable Sequence Mapping Table (SMT) and Finite State Machine (FSM) control, the chip realizes cycle-accurate Processing Element (PE) control, automatic loop tiling and latency hiding without extra time slots for pre-configuration. Evaluated on AlexNet and VGG-16, the IECA retains over 97.3% PE utilization and over 95.6% memory access time hiding on average. The chip is designed and fabricated in a UMC 55-nm process running at a frequency of 250 MHz and achieves an area efficiency of 30.55 GOPS/mm2 and 0.244 GOPS/KGE (kilo-gate-equivalent), which makes an over $2.0\times $ and $2.1\times $ improvement, respectively, compared with that of previous related works. Implementation of the IEC control scheme uses only a 0.55% area of the 2.75 mm2 core. [ABSTRACT FROM AUTHOR]

Depthwise separable convolution (DSC) has become one of the essential structures for lightweight convolutional neural networks. Nevertheless, its hardware architecture has not received much attention. Several previous hardware designs incur either high off-chip memory traffic or large on-chip memory usage, and hence have deficiency in terms of hardware efficiency as well as performance. This paper proposes two efficient dynamic design techniques, i.e. adaptive row-based dataflow scheduling and adaptive computation mapping, to achieve a much better trade-off between hardware efficiency and performance for DSC-based lightweight CNN accelerator. The effectiveness and efficiency of the proposed dynamic design techniques have been extensively evaluated using six DSC-based lightweight CNNs. Compared with the reference architectures, the simulation results show the proposed architectural techniques can at least reduce on-chip buffer size by 50.4% and improve the performance of convolution calculation by 1.18 × while maintaining the minimum off-chip memory traffic. MobileNetV2 is implemented on Zynq UltraScale+ ZCU102 SoC FPGA, and the results show the proposed accelerator can achieve 381.7 frames per second (fps), which is 1.43 × of the reference design, and it can save about 36.3% on-chip buffer size compared with the reference design, while maintaining the same off-chip memory traffic. [ABSTRACT FROM AUTHOR]

In-memory computing provides unprecedented power and area efficiency for the execution of convolutional neural networks by using memory bitcells to perform dot-product (DP) operations in the analog domain. Yet, these operators suffer from analog non-idealities (ANIs) that degrade the inference accuracy. This paper proposes design guidelines inferred from a holistic simulation-based analysis of the impact of ANIs on the accuracy-efficiency trade-off that affects current-domain DP operators based on conventional 6T-SRAM bitcell arrays. We define a custom SNR metric aware of the DP operand distribution to quantify decision errors associated with various ANIs, over ranges of input/output resolution and hardware design parameters. We find out that non-linearity and local mismatch are the dominant ANIs limiting the design space, while IR drops turn out to be critical only when targeting high parallelism. We then quantify the accuracy-efficiency trade-off related to these dominant ANIs across the design space and propose optimal design choices. We notably identify that using larger operators can either improve or worsen the SNR depending on the target output resolution. Furthermore, we show that hardware calibration techniques which mitigate mismatch help to recover a fraction of the lost SNR, with greater effectiveness when scaling down the supply voltage. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNN) have been proved to be an effective method in the field of artificial intelligence (AI), and large-scale deploying CNN to embedded devices, no doubt, will greatly promote the development and application of AI into the practical industry. However, mainly due to the space-time complexity of CNN, computing power, memory bandwidth and flexibility are performance bottlenecks. In this paper, a framework containing model compression and hardware acceleration is proposed to solve the above problems. This framework consists of a mixed pruning method, data storage optimization for efficient memory utilization and an accelerator for mapping CNN on field programmable gate array (FPGA). The mixed pruning method is used to compress the model, and data bit-width is reduced to 8-bit by data quantization. Accelerator based on FPGA makes it flexible, configurable and efficient for CNN implementation. The model compression is evaluated on NVIDIA RTX2080Ti, and the results illustrate that the VGG16 is compressed by $30\times $ and the fully convolutional network (FCN) is compressed by $11\times $ within 1% accuracy loss. The compressed model is deployed and accelerated on ZCU102, which is up to $1.7\times $ and $24.5\times $ better in energy efficiency compared with RTX2080Ti and Intel i7 7700. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNNs) have been widely used in image classification and recognition due to their effectiveness; however, CNNs use a large volume of weight data that is difficult to store in on-chip memory of embedded designs. Pruning can compress the CNN model at a small accuracy loss; however, a pruned CNN model operates slower when implemented on a parallel architecture. In this paper, a hardware-oriented CNN compression strategy is proposed; a deep neural network (DNN) model is divided into “no-pruning layers ($NP$ -layers)” and “pruning layers ($P$ -layers)”. A $NP$ -layer has a regular weights distribution for parallel computing and high performance. A $P$ -layer is irregular due to pruning, but it generates a high compression ratio. Uniform and incremental quantization schemes are used to achieve a tradeoff between compression ratio and processing efficiency at a small loss in accuracy. A distributed convolutional architecture with several parallel finite impulse response (FIR) filters is further proposed for the regular model in the $NP$ -layers. A shift-accumulator based processing element with an activation-driven data flow (ADF) is proposed for the irregular sparse model in the $P$ -layers. Based on the proposed compression strategy and hardware architecture, a hardware/algorithm co-optimization (HACO) approach is proposed for implementing a $NP-P$ hybrid compressed CNN model on FPGAs. For a hardware accelerator on a single FPGA chip without the use of off-chip memory, a $27.5\times $ compression ratio is achieved with 0.44% top-5 accuracy loss for VGG-16. The implementation of the compressed VGG-16 model on a Xilinx VCU118 evaluation board processes 83.0 frames per second (FPS) for image applications, this is $1.8\times $ superior than the state-of-the-art design found in the technical literature. [ABSTRACT FROM AUTHOR]

This paper presents a field-programmable gate array (FPGA) design of a segmentation algorithm based on convolutional neural network (CNN) that can process light detection and ranging (LiDAR) data in real-time. For autonomous vehicles, drivable region segmentation is an essential step that sets up the static constraints for planning tasks. Traditional drivable region segmentation algorithms are mostly developed on camera data, so their performance is susceptible to the light conditions and the qualities of road markings. LiDAR sensors can obtain the 3D geometry information of the vehicle surroundings with high precision. However, it is a computational challenge to process a large amount of LiDAR data in real-time. In this paper, a CNN model is proposed and trained to perform semantic segmentation using data from the LiDAR sensor. An efficient hardware architecture is proposed and implemented on an FPGA that can process each LiDAR scan in 17.59 ms, which is much faster than the previous works. Evaluated using Ford and KITTI road detection benchmarks, the proposed solution achieves both high accuracy in performance and real-time processing in speed. [ABSTRACT FROM AUTHOR]

This paper demonstrates a compact mixed-signal (MS) convolutional neural network (CNN) design procedure by proposing a MS modular neuron unit that alleviates analog circuit related design issues such as noise. Through the first step of the proposed procedure, we design a CNN in software with a minimized number of channels for each layer while satisfying the network performance to the target, which creates low representational and computational cost. Then, the network is reconstructed and retrained with a single modular neuron that is recursively utilized for the entire network for the maximum hardware efficiency with a fixed number of parameters that consider signal attenuation. For the last step of the proposed procedure, the parameters of the networks are quantized to an implementable level of MS neurons. We designed the networks for MNIST and Cifar-10 and achieved compact CNNs with a single MS neuron with 97% accuracy for MNIST and 85% accuracy for Cifar-10 whose representational cost and computational cost are reduced to least two times smaller than prior works. The estimated energy per classification of the hardware network for Cifar-10 with a single MS neuron, designed with optimum noise and matching requirements, is $0.5\mu $ J, which is five times smaller than its digital counterpart. [ABSTRACT FROM AUTHOR]

Hardware failures are undesired but a common problem in circuits. Such failures are inherently due to the aging of circuitry or variation in circumstances. In critical systems, customers demand the system never to fail. Several self-healing and fault tolerance techniques have been proposed in the literature for recovering a circuitry from a fault. Such techniques come to the rescue when a fault has already occurred but they are typically uninformed about the possibility of an impending failure (i.e., fault prediction), which can be used as a pre-stage to fault tolerance and self-healing. This paper presents an approach to early fault prediction of circuits. The proposed method uses Fast Fourier Transform (FFT) to get the fault frequency signature, Principal Component Analysis (PCA) to get the most important data with reduced dimension, and Convolutional Neural Network (CNN) to learn and classify the fault. The proposed method is validated for working in different circuits by testing it using two circuits: comparator and amplifier. The comparator and amplifier are implemented using 45 nm technology on HSPICE to extract the failures dataset in terms of voltage, current, temperature, noise, and delay. This extracted data is used for training the proposed approach using Tensorflow. To the best of our knowledge, this is the first work of fault prediction at the transistor level for hardware system. The proposed approach considers aging, short-circuit, and open-circuit faults, and it provides a fault prediction accuracy of 98.93% and 98.91% for comparator and amplifier circuits, respectively. The proposed method is tested for two different circuits for its validation, and it consumes 1.08 W for Altera Arria 10 GX FPGA device. [ABSTRACT FROM AUTHOR]

Quantization is a promising technique to compress the size of Convolutional Neural Network (CNN) models. Recently, various precision-scalable designs have been presented to reduce the computational complexity in CNNs. However, most of them adopt straightforward calculation scheme to implement the CNN, which causes high bandwidth requirement and low hardware utilization efficiency. This paper proposes a new precision-scalable architecture which can fully reduce the computational complexity in CNN inference and meanwhile has a finely simplified calculation scheme. Based on the proposed scheme, a well-optimized multiplier called Compositional Processing Element (C-PE) is devised. Compared with the previous multipliers, the new C-PE requires less area and power. Furthermore, two levels of optimization are introduced to the design to relieve the bandwidth problem and increase the hardware utilization efficiency. Implemented under the TSMC 90nm CMOS technology, the whole design achieves 6-68.1 fps in various precisions on VGG16 benchmark and a 49.8TOPS/W energy efficiency at 500MHz when scaled to 28nm, which is much better than previous precision-scalable ones. [ABSTRACT FROM AUTHOR]

In this paper, we propose O4-DNN, a high-performance FPGA-based architecture for convolutional neural network (CNN) accelerators relying on operation packing and out-of-order (OoO) execution for DSP blocks augmented with LUT-based glue logic. The high-level architecture is comprised of a systolic array of processing elements (PEs), supporting output stationary dataflow. In this architecture, the computational unit of each PE is realized by using a DSP block as well as a small number of LUTs. Given the limited number of DSP blocks in FPGAs, the combination (DSP block and some LUTs) provides more computational power obtainable through each DSP block. The proposed computational unit performs eight convolutional operations on five input operands where one of them is an 8-bit weight and the others are four 8-bit input feature (IF) maps. In addition, to improve the energy efficiency of the proposed computational unit, we present an approximate form of the unit suitable for neural network applications. To reduce the memory bandwidth as well as increase the utilization of the computational units, a data reusing technique based on the weight sharing is also presented. To improve the performance of the proposed computational unit further, an addressing approach for computing the partial sums out-of-order is proposed. The efficacy of the architecture is assessed using two FPGA devices executing four state-of-the-art neural networks. Experimental results show that this architecture leads to, on average (up to), $2.5\times $ ($3.44\times$) higher throughput compared to a baseline structure. In addition, on average (maximum of), 12% (40%) energy efficiency improvement is achievable by employing the O4-DNN compared to the baseline structure. [ABSTRACT FROM AUTHOR]

To reduce multiplication operations in convolution of convolutional neural networks (CNNs), there are three widely used convolutional acceleration algorithms, i.e., Winograd, FFT and FFA. However, current accelerators based on these convolutional acceleration algorithms have issues on flexibility and efficiency. Firstly, some accelerators utilized a combination of these acceleration algorithms and employed multiple types of computational units to achieve their respective advantages. As a result, some computational units are left unused when the best-performing unit is working, which causes much area inefficiency. Secondly, current accelerators tend to choose small parameters of these convolutional acceleration algorithms to avoid unacceptable precision loss, as a result, they are hardly to support large kernel sizes and lack of flexibility. Thirdly, these acceleration algorithms are typically presented for 1-stride convolutions, consequently, few implementation considers the acceleration of large-stride convolutions, which is a major restriction to hardware flexibility. This paper proposed a stride-based convolution decomposition method (SCDM) to reform different convolution shapes (i.e., kernel sizes & strides) to an identical pattern. With the aid of SCDM, a Winograd-stretched and hardware-efficient design (WHD) is presented to utilize one uniform computational unit for the acceleration of different convolution shapes, which combines complementary performance advantages on both Winograd F(4,3)andF(4,2) units. Compared to current FFT-based or FFA-based works, WHD can stretch the use range of Winograd and simplify implementation, thereby achieving hardware flexibility and efficiency. Evaluation results show that 34.08%~55.41% operation reduction were achieved on six CNN models, while incurring a slight hardware overhead. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNNs) are widely employed in modern computer vision algorithms, where the input image is convolved iteratively by many filters to extract the knowledge behind it. However, while the depth of convolutional layers gets deeper and deeper in recent years, the enormous computational complexity makes it difficult to be deployed on embedded systems with limited hardware resources. In this paper, inspired by rate-distortion optimization in image and video coding, we propose a computation-performance optimization (CPO) method to remove the redundant convolution filters in a CNN with performance constraints. To prove the effectiveness of the proposed method, CPO is applied to the networks for image super-resolution and image classification. Under almost the same PSNR drop and accuracy drop for performance evaluation in these two tasks, we can achieve the best parameter and computation reduction when compared with previous works. [ABSTRACT FROM AUTHOR]

Emerging convolutional neural networks (CNNs) tend to be designed with varied per-layer data widths and sparse representations. However, these two features, which bring many redundant computations, have not been exploited simultaneously in existing hardware architectures for CNNs. This paper proposes an energy-quality scalable architecture, namely folded precision-adjustable processor (FPAP), to eliminate all computational redundancies by using folding techniques. On one hand, FPAP decomposes the dominant multiply-accumulate (MAC) operations into multiple adds and folds them into single arithmetic unit. Only effective adds (or part of them) are then calculated serially. Thus, FPAP can adapt to different per-layer data widths and enable precision-adjustable approximate computing. Particularly, FPAP adaptively selects either activation or weight to be decomposed in every single MAC to minimize the total number of adds and clock cycles. On the other hand, a 1-D convolution is undertaken by a multi-tap transposed finite impulse response (FIR) filter, which is folded into one tap to skip MACs with zero weights or activations. Besides, a judicious delay element remapping scheme and a novel genetic algorithm-based kernel reallocation scheme, are developed to reduce the power consumption in a folded FIR filter and mitigate the load imbalance issue caused by irregular sparsity, respectively. With all these optimizations, FPAP is able to reach comparable or even faster processing speed over the corresponding unfolded design in sparse CNNs while consuming smaller area. Experimental results on real CNN models demonstrate that FPAP can scale its energy efficiency from 4.28 to 23.63 TOP/s/W, and area efficiency from 37.79 to 164.15GOP/s/mm2, respectively, under the TSMC 28-nm HPC CMOS technology. [ABSTRACT FROM AUTHOR]

This paper presents a rescheduled dataflow of convolution and its hardware architecture that can enhance energy efficiency. For convolution involving a large amount of computations and memory accesses, previous accelerators employed parallel processing elements to meet real-time constraints. Though the previous approaches made a success in implementing complex convolution models, they load the same input features and filter weights from on-chip memories multiple times due to the iterative property of convolution operations, suffering from high energy consumption. To mitigate redundant memory accesses, a novel dataflow is proposed that computes convolution operations incrementally so as to reuse the loaded data as maximally as possible. In addition, several convolution accelerators supporting the rescheduled dataflow are investigated, and qualitative and quantitative analyses are performed to suggest a promising candidate for various convolution models. Simulation results show that the energy efficiency of the proposed accelerator outperforms that of the previous accelerator significantly. [ABSTRACT FROM AUTHOR]

In the past few years, the demand for real-time hardware implementations of deep neural networks (DNNs), especially convolutional neural networks (CNNs), has dramatically increased, thanks to their excellent performance on a wide range of recognition and classification tasks. When considering real-time action recognition and video/image classification systems, latency is of paramount importance. Therefore, applications strive to maximize the accuracy while keeping the latency under a given application-specific maximum: in most cases, this threshold cannot exceed a few hundred milliseconds. Until now, the research on DNNs has mainly focused on achieving a better classification or recognition accuracy, whereas very few works in literature take in account the computational complexity of the model. In this paper, we propose an efficient computational method, which is inspired by a computational core of fully connected neural networks, to process convolutional layers of state-of-the-art deep CNNs within strict latency requirements. To this end, we implemented our method customized for VGG and VGG-based networks which have shown state-of-the-art performance on different classification/recognition data sets. The implementation results in 65-nm CMOS technology show that the proposed accelerator can process convolutional layers of VGGNet up to 9.5 times faster than state-of-the-art accelerators reported to-date while occupying 3.5 mm2. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNNs) have achieved tremendous success in the computer vision domain recently. The pursue for better model accuracy drives the model size and the storage requirements of CNNs as well as the computational complexity. Therefore, Compression Learning by InParallel Pruning-Quantization (CLIP-Q) was proposed to reduce a vast amount of weight storage requirements by using a few quantized segments to represent all weights in a CNN layer. Among various quantization strategies, CLIP-Q is suitable for hardware accelerators because it reduces model size significantly while maintaining the full-precision model accuracy. However, the current CLIP-Q approach did not consider the hardware characteristics and it is not straightforward when mapped to a CNN hardware accelerator. In this work, we propose a software-hardware codesign platform that includes a modified version of CLIP-Q algorithm and a hardware accelerator, which consists of $5\times 5$ reconfigurable convolutional arrays with input and output channel parallelization. Additionally, the proposed CNN accelerator maintains the same accuracy of a full-precision CNN in Cifar-10 and Cifar-100 datasets. [ABSTRACT FROM AUTHOR]

Multi-bit-width convolutional neural network (CNN) maintains the balance between network accuracy and hardware efficiency, thus enlightening a promising method for accurate yet energy-efficient edge computing. In this work, we develop state-of-the-art multi-bit-width accelerator for NAS Optimized deep learning neural networks. To efficiently process the multi-bit-width network inferencing, multi-level optimizations have been proposed. Firstly, differential Neural Architecture Search (NAS) method is adopted for the high accuracy multi-bit-width network generation. Secondly, hybrid Booth based multi-bit-width multiply-add-accumulation (MAC) unit is developed for data processing. Thirdly, vector systolic array is proposed for effectively accelerating the matrix multiplications. With vector-style systolic dataflow, both the processing time and logic resources consumption can be reduced when compared with the classical systolic array. Finally, The proposed multi-bit-width CNN acceleration scheme has been practically deployed on FPGA platform of Xilinx ZCU102. Average performance on accelerating the full NAS optimized VGG16 network is 784.2 GOPS, and peek performance of the convolutional layer can reach as high as 871.26 GOPS for INT8, 1676.96 GOPS for INT4, and 2863.29 GOPS for INT2 respectively, which is among the best results in previous CNN accelerator benchmarks. [ABSTRACT FROM AUTHOR]

The appealing property of low area, low power, and high bit error tolerance has made Stochastic Computing (SC) a promising alternative to conventional binary arithmetic for many computation intensive tasks, e.g., convolutional neural networks (CNNs). However, current SC-based CNN accelerators suffer from the intrinsic computation error and exponentially growing latency. In this work, we optimize both the architecture of SC multiply-and-accumulate (MAC) unit and the overall acceleration strategy of CNN accelerator to favor SC. A low-complexity bit-stream-extending method is proposed to suppress the computation error of SC and ensure the trained fix-point model can be deployed into SC-based hardware without fine-tuning. Besides, distribution-determined partition scheme is developed to design hybrid stochastic-binary computing (SBC) MAC unit which boosts the processing of bit streams at a minimum overhead. For the overall accelerator, the SBC-based MAC array is extended to reuse hardware resources and improve throughput, since the judiciously chosen loop unrolling strategy can better benefit SC operations. The proposed CNN accelerator with extended SBC-MAC array is synthesized and validated using TSMC 28nm CMOS on several representative CNNs, targeted at ImageNet dataset. Compared with precise binary implementation, our proposed design gains 44% area reduction and 50% power saving but induces only 4% additional computation latency and 0.5% accuracy degradation. [ABSTRACT FROM AUTHOR]

Low-power keywords recognition has been a focus of acoustic signal processing for several decades. This work investigates the domain-specific speech recognition microprocessor based on optimized one-dimensional convolutional recurrent neural network (1D-CRNN). Compared to previous DNN based frameworks, the proposed 1D-CRNN can process both the feature extraction and keywords classification, and achieve high recognition accuracy with reduced computation operations under wide range background noise SNRs. An energy-efficient 1D-CRNN accelerator is implemented to dynamically reconfigure and process the different layers. This accelerator has the characteristics of “More is Less” in three aspects: 1) the hybrid network with more complex layers is much more compact and requires less computation; 2) although the weight width quantized to 8 bits requires more memory size and multiplication energy cost, the required network neurons can be reduced and hardware utilization can be improved; 3) an energy-aware self-compensation tensor multiplication unit with dual power supply based on approximation design method can be utilized for 1D-CRNN computing. Compared to the state-of-the-art architectures, the novel more-is-less architecture can achieve a much lower power consumption of $1.4~\mu \text{W}\sim 2.1~\mu \text{W}$ (over 80% reduced) under an industry 22nm technology, while maintaining higher system adaptability (support SNRs: −5dB~Clean) for 1~5 real-time keywords recognition. [ABSTRACT FROM AUTHOR]

To enable efficient deployment of convolutional neural networks (CNNs) on embedded platforms for different computer vision applications, several convolution variants have been introduced, such as depthwise convolution (DWCV), transposed convolution (TPCV), and dilated convolution (DLCV). To address the utilization degradation issue occurred in a general convolution engine for these emerging operators, a highly flexible and reconfigurable hardware accelerator is proposed to efficiently support various CNN-based vision tasks. Firstly, to avoid workload imbalance of TPCV, a zero transfer and skipping (ZTS) method is proposed to reorganize the computation process. To eliminate the redundant zero calculations of TPCV and DLCV, a sparsity-alike processing (SAP) method is proposed based on weight-oriented dataflow. Secondly, the DWCV or pooling layers are configured to be directly executed after standard convolutions without external memory accesses. Furthermore, a programmable execution schedule is introduced to gain better flexibility. Finally, the proposed accelerator is evaluated on Intel Arria 10 SoC FPGA. Experimental results show state-of-the-art performance on both large-scale and lightweight CNNs for image segmentation or classification. Specifically, the accelerator can achieve a processing speed up to 339.9 FPS and computational efficiency up to 0.58 GOPS/DSP, which is $3.3\times $ better than the prior art evaluated on the same network. [ABSTRACT FROM AUTHOR]

Temporal Convolutional Networks (TCNs) involving mono channels as input, have shown superior performance compared to state-of-the-art sequence detection recursive networks in a variety of applications. TCNs leverage the concept of dilated causal convolution for a wider receptive field coverage of input (mono) channels, which requires scaling the delay between input samples in Multiply-Accumulate (MAC) units in different layers. We demonstrate a possible data-flow transformation to convert a dilated convolution to a non-dilated convolution to remove such need for delay scaling while maintaining the same receptive field. The new data-flow transformation allows for hardware units to be shared across all layers with single-delay units between the MAC units. We demonstrate how such data-flow transformation can be easily achieved using generic Finite Impulse Response (FIR) filter modules, simplifying the deployment of TCNs. We validate the predicted savings using Cadence Stratus High-Level Synthesis (HLS). A gesture recognition case study using ultrasound is synthesized achieving 25% savings in both energy and area if the data-flow transformation is applied. [ABSTRACT FROM AUTHOR]

GPUs became the reference platform for both training and inference phases of Convolutional Neural Networks (CNN) due to their tailored architecture to the CNN operators. However, GPUs are power-hungry architectures. A path to enable the deployment of CNNs in energy-constrained devices is adopting hardware accelerators for the inference phase. The design space exploration of CNNs using standard approaches, such as RTL, is limited due to their complexity. Thus, designers need frameworks enabling design space exploration that delivers accurate hardware estimation metrics to deploy CNNs. This work proposes a framework to explore CNNs design space, providing power, performance, and area (PPA) estimations. The heart of the framework is a system simulator. The system simulator front-end is TensorFlow, and the back-end is performance estimations obtained from the physical synthesis of hardware accelerators, not only from components like multipliers and adders. The first set of results evaluate the CNN accuracy using integer quantization, the accelerators PPA after physical synthesis, and the benefits of using a system simulator. These results allow a rich design space exploration, enabling selecting the best set of CNN parameters to meet the design constraints. [ABSTRACT FROM AUTHOR]

Convolutional Neural Networks (CNNs) have proven to be extremely accurate for image recognition, even outperforming human recognition capability. When deployed on battery–powered mobile devices, efficient computer architectures are required to enable fast and energy-efficient computation of costly convolution operations. Despite recent advances in hardware accelerator design for CNNs, two major problems have not yet been addressed effectively, particularly when the convolution layers have highly diverse structures: (1) minimizing energy-hungry off-chip DRAM data movements; (2) maximizing the utilization factor of processing resources to perform convolutions. This work thus proposes an energy-efficient architecture equipped with several optimized dataflows to support the structural diversity of modern CNNs. The proposed approach is evaluated on convolutional layers of VGGNet-16 and ResNet-50. Results show that the architecture achieves a Processing Element (PE) utilization factor of 98% for the majority of 3 × 3 and 1 × 1 convolutional layers, while limiting latency to 396.9 ms and 92.7 ms when performing convolutional layers of VGGNet-16 and ResNet-50, respectively. In addition, the proposed architecture benefits from the structured sparsity in ResNet-50 to reduce the latency to 42.5 ms when half of the channels are pruned. [ABSTRACT FROM AUTHOR]

We present an approximate logarithmic multiplier with two-stage operand trimming, which prioritises area and energy consumption while retains acceptable accuracy. The multiplier trims the least significant parts of input operands in the first stage and the mantissas of the obtained operands’ approximations in the second stage. We evaluated the multiplier’s efficiency in terms of error, energy, and area utilisation using NanGate 45nm. The experimental results show that the proposed multiplier exhibits smaller area utilisation and energy consumption than the state-of-the-art designs and that it behaves well in image processing and image classification with convolutional neural networks. [ABSTRACT FROM AUTHOR]

Dynamic power is a major source of power dissipation for high speed designs. Domain isolation methodology is a recently-proposed technique for reducing dynamic power based on controlling the evaluation phase of dynamic logic (toggling control). This work demonstrates some design issues in the domain isolation methodology and explains why it is inefficient with pipelined systems. We propose fixes for its identified issues, which enables using the toggling control with pipelined systems in a more efficient way. A novel flow named “Power Reduction Flow” is proposed for reducing dynamic power of digital circuits. Our flow uses novel design analytical methods, novel “Dynamic Logic Modifier Flow”, and novel “Dynmic Logic Area Validation Flow” for reducing dynamic power with conditionally improving performance. The new design analytical methods are based on probability theory, SystemVerilog covergroups, and digital circuit modeling. A new event type perspective is also proposed to analyze designs to reduce dynamic power in them. Experimental results using TSMC 65 nm and low supply voltages show up to 59% power reduction compared to the original traditional techniques with improving circuit’s performance by 3 × of its original maximum operating frequency at the cost of an extra 12.3% increase in area. [ABSTRACT FROM AUTHOR]

The data transfer bottleneck in Von Neumann architecture owing to the separation between processor and memory hinders the development of high-performance computing. The computing in memory (CIM) concept is widely considered as a promising solution for overcoming this issue. In this article, we present a time-domain CIM (TD-CIM) scheme using spintronics, which can be applied to construct the energy-efficient convolutional neural network (CNN). Basic Boolean logic operations are implemented through recording the bit-line output at different moments. A multi-addend addition mechanism is then introduced based on the TD-CIM circuit, which can eliminate the cascaded full adders. To further optimize the compatibility of TD-CIM circuit for CNN, we also propose a quantization method that transforms floating-point parameters of pre-trained CNN models into fixed-point parameters. Finally, we build a TD-CIM architecture integrating with a highly reconfigurable array of field-free spin-orbit torque magnetic random access memory (SOT-MRAM) and evaluate its benefits for the quantized CNN. By performing digit recognition with the MNIST dataset, we find that the delay and energy are respectively reduced by 1.2-2.7 times and $2.4\times 10 ^{3} - 1.1\times 10 ^{4}$ times compared with STT-CIM and CRAM based on spintronic memory. Finally, the recognition accuracy can reach 98.65% and 91.11% on MNIST and CIFAR-10, respectively. [ABSTRACT FROM AUTHOR]

Deep convolutional neural networks (DCNNs) have achieved state-of-the-art performance in classification, natural language processing (NLP), and regression tasks. However, there is still a great gap between DCNNs and the human brain in terms of computation efficiency. Inspired by neural synaptic plasticity and stochastic computing (SC), we propose neural synaptic plasticity-inspired computing (NSPC) to simulate the human brain’s neural network activity for inference tasks with simple logic gates. The multiplication and accumulation (MAC) is transformed by the wire connectivity in NSPC, which only requires bundles of wires and small width adders. To this end, the NSPC imitates the structure of neural synaptic plasticity from a circuit wires connection perspective. Furthermore, from the principle of NSPC, we use a data mapping method to convert the convolution operations to matrix multiplications. Based on the methodology of NSPC, fully-pipelined and low latency architecture is designed. The proposed NSPC accelerator exhibits high hardware efficiency while maintaining a comparable network accuracy level. The NSPC based DCNN accelerator (NSPC-CNN) processes DCNN at $1.5625M$ images/ $s$ with a power dissipation of $15.42~W$ and an area of $36.4~mm^{2}$. The NSPC based deep neural network (DNN) accelerator (NSPC-DNN) that implements three fully connected layers DNN consumes only $6.6~mm^{2}$ area and $2.93~W$ power, and achieves a throughput of $400M$ images/ $s$. Compared with conventional fixed-point implementations, the NSPC-CNN achieves $2.77 \times $ area efficiency, $2.25 \times $ power efficiency; the proposed NSPC-DNN exhibits $2.31 \times $ area efficiency and $2.09 \times $ power efficiency. [ABSTRACT FROM AUTHOR]

In recent years, convolutional neural network (CNN) has gained popularity in many engineering applications especially for computer vision. In order to achieve better performance, more complex structures and advanced operations are incorporated into neural networks, which results in very long inference time. For time-critical tasks such as autonomous driving and virtual reality, real-time processing is fundamental. In order to reach real-time processing speed, a lightweight, high-throughput CNN architecture namely RoadNet-RT is proposed for road segmentation in this article. It achieves 92.55% MaxF score on KITTI road segmentation dataset. The inference time is about 9 ms per frame when running on GTX 1080 GPU. Comparing to the state-of-the-art network, RoadNet-RT speeds up the inference time by a factor of 17.8 at the cost of only 3.75% loss in accuracy. What is more, on CamVid dataset its accuracy is 92.98%. Several techniques such as depthwise separable convolution and non-uniformed kernel size convolution are optimized in the hardware accelerator design. The proposed CNN architecture has been successfully implemented on a ZCU102 MPSoC FPGA that achieves the computation capability of 331 GOPS using INT8 quantization. The system throughput reaches 196.7 frames per second with input image size of $280\times 960$. The source code is published at https://github.com/linbaiwpi/RoadNet-RT. [ABSTRACT FROM AUTHOR]

Arithmetic precision scaling is mandatory to deploy Convolutional Neural Networks (CNNs) on resource-constrained devices such as microcontrollers (MCUs), and quantization via fixed-point or binarization are the most adopted techniques today. Despite being born by the same concept of bit-width lowering, these two strategies differ substantially each other, and hence are often conceived and implemented separately. However, their joint integration is feasible and, if properly implemented, can bring to large savings and high processing efficiency. This work elaborates on this aspect introducing a boosted collaborative mechanism that pushes CNNs towards higher performance and more predictive capability. Referred as BC-Net, the proposed solution consists of a self-adaptive conditional scheme where a lightweight binary net and an 8-bit quantized net are trained to cooperate dynamically. Experiments conducted on four different CNN benchmarks deployed on off-the-shelf boards powered with the MCUs of the Cortex-M family by ARM show that BC-Nets outperform classical quantization and binarization when applied as separate techniques (up to 81.49% speed-up and up to 3.8% of accuracy improvement). The comparative analysis with a previously proposed cooperative method also demonstrates BC-Nets achieve substantial savings in terms of both performance (+19%) and accuracy (+3.45%). [ABSTRACT FROM AUTHOR]

An energy-efficient convolutional neural network (CNN) processor is proposed for real-time image segmentation on mobile devices. The proposed processor utilizes Region of Interest (ROI) based image segmentation to speed up the process and reduce the overall external memory access. Although the ROI based image segmentation degrades the segmentation accuracy, the proposed dilation rate adjustment algorithm, which regulates the receptive field depending on the ROI resolution during dilated convolution, compensates for the accuracy degradation up to 0.2310 mean Intersection over Union (mIoU). In addition, the processor accelerates the dilated and transposed convolution by skipping the redundant zero computations with the proposed delay cells. As a result, the throughput of dilated and transposed convolution is increased up to × 159 and × 3.84. The delay cells can also support the variable dilation rates in dilated convolution caused by the dilation rate adjustment algorithm. Moreover, the processor selects the operating frequency based on the ROI resolution to save power consumption up to 81.2%. The processor is simulated in 65 nm CMOS technology, and the 6.8 mm2 processor consumes the 206 mW power consumption with the 4.66 ms of processing time and 3.22 TOPS/W energy-efficiency at the target image segmentation dataset. [ABSTRACT FROM AUTHOR]

A low power face recognition (FR) convolutional neural network (CNN) processor is proposed with high power efficiency to achieve always-on FR in mobile devices. Three key features enable a power-efficient FR CNN. First, tile-based clustering (THC) is proposed for reducing the computation overhead of hierarchical clustering. It generates an average of 37.2% duplicated input features in the entire network. Second, a low latency tile-based hierarchical clustering core is proposed. It supports an approximated clustering method that removes distance updates and increases pipeline utilization by up to 98.6%. Finally, a similar feature skipping binary convolution core with separated accumulation cores is proposed to increase power efficiency through skipping redundant MAC operation between duplicated input features and weights. It can skip 38.9% of CNN operations and reduce 35.8% of the total computation considering both CNN and THC. The proposed features enable the proposed processor to operate FR CNN with 17.3 TOPS/W power efficiency, which is $1.3\times $ higher compared to the previous state-of-the-art FR CNN processor. Implemented in 65nm CMOS technology, the 6mm2 FR CNN processor shows 0.5mW power consumption at 1 frames-per-second always-on face recognition. [ABSTRACT FROM AUTHOR]