Search

For a class of nontriangular nonlinear systems in presence of unknown disturbances, we propose a predictor-based neural dynamic surface control (PNDSC) strategy in this paper. This nontriangular system is transformed via the mean value theorem, and a predictor is then constructed. To avoid an algebraic loop problem, partial state vectors are employed as input signals of neural networks (NNs) for approximating unknown dynamics, and compensation items are designed to compensate for approximation errors from NNs. Different from the traditional NDSC, the PNDSC in this paper utilizes prediction errors to update learning parameters for improving NNs’ learning behaviors with overlarge adaptive gains. On the basis of improved NNs’ approximation behaviors, a predictor-based NNs disturbance observer (PNNDO) is constructed for compensation for external disturbances and approximation errors from NNs. Furthermore, with predictors, a normalization method of weights is developed to reduce the number of online learning parameters. On the basis of the aforementioned result, measurement noises are taken into account in our predictor-based neural control strategy. We employ predictor states, rather than measurement information paralyzed by noises, in design of our control strategy. This reduces high-frequency oscillations in control input. A Lyapunov-based stability analysis shows that all signals are ultimately bounded in the closed-loop system. Finally, the effectiveness of the proposed control strategy is verified by a numerical example and a permanent magnet brushless DC motor system. [ABSTRACT FROM AUTHOR]

Neural networks (NNs) with large scales of artificial neurons are increasingly used in recognition and classification tasks. In power-constrained scenarios, the tradeoff between performance and hardware consumptions must be carefully evaluated before silicon tape-out. In this paper, we proposed a systematic approach to design ultra-low power NN system. This work is motivated by the facts that NNs are resilient to approximation in many of the computations and NNs are outputting statistical tensors which are acceptable to less-than-perfect results. We resort to the front-back end approach with a twofold aim: (1) a fast and accurate design approach is proposed by estimating the computing quality of low-power approximate adder arrays, and it is adopted to evaluate the neural network system; (2) a quality configurable engine with different approximation degrees while processing NNs is implemented. The proposed work is demonstrated with a comprehensive keyword spotting (KWS) system as an ultra-low power NN engine. The experimental environment is setup with ten keywords from the google speech command dataset (GSCD) using an industrial 22-nm ultra-low-leakage (ULL) process. Comparing to the state-of-the-art KWS processors, the proposed approximate NN engine can demonstrate over 60% improvement in power efficiency and $1.1\times $ area efficiency while achieving similar recognition accuracy. [ABSTRACT FROM AUTHOR]

In this paper, a novel class of quaternion-valued memristive neural networks (QVMNNs) that considers both the spatial factor and Markov jump phenomenon is proposed, the finite/fixed-time anti-synchronization (F/FTAS) of which is investigated. It is worth mentioning that the considered master and slave systems are assumed to jump along two inconsistent Markov chains, which is a first attempt at the issue of the anti-synchronization of Markovian systems and may be more realistic than most existing Markovian systems’ models. Then, a suitable feedback controller is designed, such that the error system can be finite/fixed-time stable. Moreover, by integrating algebraic inequality technologies and Lyapunov theory, a new F/FTAS theorem can be obtained for the proposed systems. Finally, this paper provides two examples, so that the rationality, superiority, and practical value of the main results can be illustrated. [ABSTRACT FROM AUTHOR]

In this paper, we propose a low-power memory-based computing architecture, called selective computing architecture (SCA). It consists of multipliers and an LUT (Look-Up Table)-based component, that is multi-context ternary content-addressable memory (MC-TCAM). Either of them is selected by input-data conditions in neural-networks (NNs). Compared with quantized NNs, a higher accurate multiplication can be performed with low-power consumption in the proposed architecture. If input data stored in the MC-TCAM appears, the corresponding multiplication results for multiple weights are obtained. The MC-TCAM stores only shorter length of input data, resulting in achieving a low-power computing. The performance of the SCA is determined by three physical parameters concerning the configuration of MC-TCAM. The power dissipation of the target NN can be minimized by exploring these parameters in the design space. The hardware based on the proposed architecture is evaluated using TSMC 65 nm CMOS technology and MTJ model. In the case of speech command recognition, the power consumption at the multiplication of the first convolutional layer in a convolutional NN is reduced by 67% compared to the solution relying only on multipliers. [ABSTRACT FROM AUTHOR]

This paper provides a review of design approaches towards artificial intelligence (AI) System-on-Chip. AI algorithms have progressed over the past decades from perceptron-based neural network (NN) and neuro-fuzzy (NF) system to today’s deep neural network (DNN) and neuromorphic computing. Recent DNN hardware accelerators focus on energy-efficient integration of digital circuits to realize real-time DNN operation while neuromorphic processors deploy new memory technologies with analog computation for low power consumption. However, different design approaches can be applied to such processor implementation with their pros and cons. This paper reviews from the early processor designs for NN and NF in both mixed-mode and digital implementations to the recent DNN SoC designs that we have proposed for a decade. The former content deals with NN and NF processors used as a functional building block of a machine vision SoC, while the latter concentrates on integration of the whole DNN function. We also provide a discussion on the approaches, and provide perspective on future research directions. [ABSTRACT FROM AUTHOR]

This paper presents a high-performance solar energy harvesting system with improved adaptive maximum power point tracking (AMPPT) method utilizing neural network (NN) model as assistance. Under the guidance of the negative feedback control (NFC) model, a BJT based voltage control oscillator with three off-chip reconfigurable resistors is designed in this paper to improve the reusability of the solar energy harvesting system. Meanwhile, the AMPPT accuracy has much improvement by co-simulation of MATLAB/Simulink and Virtuoso with the help of NN model of Photovoltaic (PV) Cell. The complete system with output voltage of 4.2V to power the battery is designed and fabricated in $0.18~\mu \text{m}$ CMOS technology. According to the test results, the system can track the maximum power points (MPP) successfully with average voltage tracking errors of 0.23% (0.01-0.51%) of PV cell 1 and 0.29% (0.01-0.5%) of PV cell 2 when the light intensity changes from 3000lux to 10000lux. Without power hungry current sensor or voltage sensor and other complicated control circuits, the peak efficiency is about 89.39% @ 3000lux. [ABSTRACT FROM AUTHOR]

The memristor-based neuromorphic computing system (NCS) with emerging storage and computing integration architecture has drawn extensive attention. Because of the unique nonvolatility and programmability, the memristor is an ideal nano-device to realize neural synapses in VLSI circuit implementation of neural networks. However, in the hardware implementation, the performance of the memristive neural network is always affected by quantization error, writing error, and conductance drift, which seriously hinders its applications in practice. In this paper, a novel weight optimization scheme combining quantization and Bayesian inference is proposed to alleviate this problem. Specifically, the weight deviation in the memristive neural network is transformed into the weight uncertainty in the Bayesian neural network, which can make the network insensitive to unexpected weight changes. A quantization regularization term is designed and utilized during the training process of the Bayesian neural network, reducing the quantization error and improving the robustness of the network. Furthermore, a partial training method is raised to extend the applicability of the proposed scheme in large-scale neural networks. Finally, the experiments on a Multilayer Perceptron and LeNet demonstrate that the proposed weight optimization scheme can significantly enhance the robustness of memristive neural networks. [ABSTRACT FROM AUTHOR]

SRAM based computing-in-memory (SRAM-CIM) techniques have been widely studied for neural networks (NNs) to solve the “Von Neumann bottleneck”. However, as the scale of the NN model increasingly expands, the weight cannot be fully stored on-chip owing to the big device size (limited capacity) of SRAM. In this case, the NN weight data have to be frequently loaded from external memories, such as DRAM and Flash memory, which results in high energy consumption and low efficiency. In this paper, we propose a hybrid-device computing-in-memory (HD-CIM) architecture based on SRAM and MRAM (magnetic random-access memory). In our HD-CIM, the NN weight data are stored in on-chip MRAM and are loaded into SRAM-CIM core, significantly reducing energy and latency. Besides, in order to improve the data transfer efficiency between MRAM and SRAM, a high-speed pipelined MRAM readout structure is proposed to reduce the BL charging time. Our results show that the NN weight data loading energy in our design is only 0.242 pJ/bit, which is 289 $\times $ less in comparison with that from off-chip DRAM. Moreover, the energy breakdown and efficiency are analyzed based on different NN models, such as VGG19, ResNet18 and MobileNetV1. Our design can improve $\mathbf {58\times \,\,to\,\,124\times }$ energy efficiency. [ABSTRACT FROM AUTHOR]

Aiming at a class of switched uncertain nonlinear strict-feedback systems under the action of average dwell time switching signal, this paper proposes a novel adaptive neural network output feedback tracking control based on the consideration of the full state constraints. The controller is proposed based on neural networks. One of the key characteristics of the system discussed is that the state variables cannot be measured and the system states need to be kept within the constraint ranges. For the sake of estimating the unmeasured states, the observer is constructed. In order to ensure all states which are within the time-varying boundary, the tangent barrier Lyapunov function (BLF-Tan) is selected in the design process. The boundedness of the closed-loop signals with average dwell time is guaranteed by the designed controllers and all the states limit in their constrained sets. It has been proved that the output tracking error converge to a small neighborhood of zero. In addition, the significance of the presented control strategy is verified and tested by a simulation example. [ABSTRACT FROM AUTHOR]

Fault tolerance is a remarkable feature of biological systems and their self-repair capability influence modern electronic systems. In this paper, we propose a novel plastic neural network model, which establishes homeostasis in a spiking neural network. Combined with this plasticity and the inspiration from inhibitory interneurons, we develop a fault-resilient robotic controller implemented on an FPGA establishing obstacle avoidance task. We demonstrate the proposed methodology on a spiking neural network implemented on Xilinx Artix-7 FPGA. The system is able to maintain stable firing (tolerance ±10%) with a loss of up to 75% of the original synaptic inputs to a neuron. Our repair mechanism has minimal hardware overhead with a tuning circuit (repair unit) which consumes only three slices/neuron for implementing a threshold voltage-based homeostatic fault-tolerant unit. The overall architecture has a minimal impact on power consumption and, therefore, supports scalable implementations. This paper opens a novel way of implementing the behavior of natural fault tolerant system in hardware establishing homeostatic self-repair behavior. [ABSTRACT FROM PUBLISHER]

An automatic inductor design process is helpful to reduce the design cycle of radio frequency (RF) integrated circuit (IC). This paper proposed an efficient synthesis-analysis machine (SAM) for on-chip inductor synthesis and modeling, as well as an automatic dataset generation (ADG) topology for the generation of artificial neural networks (ANNs) training dataset. The SAM consists of a synthesis ANN, two analysis ANNs and a proposed self-inspection machine (SIM). For a given design request, the SAM synthesizes a layout first, followed by analyzing the performance of the layout automatically, and finally improves the layout confidence through self-inspection. Compared to the modeling functions of analysis ANN, the synthesis ANN behaves as an inverse model, of which the inputs are desired inductor performances while the outputs are geometrical parameters of the layout. However, multi-value problems might occur in obtaining geometries of inductors, suggesting with evidence from mathematic relationships between geometrical and electrical parameters of an inductor. A multi-valued training dataset will mislead the synthesis ANN, resulting in unqualified layouts. To deal with this issue, a solution space contraction technique (SSCM) is also proposed. Furthermore, the SIM suspends most of the failures by comparing the output of analysis ANN back to the input of synthesis ANN. An electromagnetic simulation tool is used for experiments, and the effectiveness of the proposed SAM is proven by 6,500 inductor samples. [ABSTRACT FROM AUTHOR]

When deploying deep neural networks (DNNs), edge devices training is practical to improve model adaptivity for various user-specific scenarios while avoiding privacy disclosure. However, the training computation is intolerable for edge devices. It inspires sparse DNN training (SDT) into the limelight, which reduces training computation by dynamic weight pruning. Generally, SDT has two strategies based on the pruning granularity: the structured or the unstructured. Unfortunately, both of them suffer from limited training efficiency due to the gap between pruning granularity and hardware implementation. The former is hardware-friendly but has a low pruning ratio, indicating limited computation reduction. The latter has a high pruning ratio, but the unbalanced workload decreases utilization and irregular sparsity distribution causes considerable sparsity processing overhead. This paper proposes a software-hardware co- design to bridge the gap for improving the efficiency of SDT. On the algorithm side, a sub-structured pruning method, achieved with hybrid shape-wise and line-wise pruning, generates a high sparsity ratio and keeps the hardware-friendly property. On the hardware side, a sub-structured weight processing unit (SWPU) effectively handles the hybrid sparsity with three techniques. First, SWPU dynamically reorders the computation sequence with hamming-distance-based clustering, balancing the irregular workload. Second, SWPU performs runtime scheduling by exploiting the feature of sub-structured sparse convolution through a detect-before-load controller, which skips redundant memory access and sparsity processing. Third, SWPU performs sparse convolution by compressing operands with spatial disconnect log-based routing and recovers their location with bi-directional switching, avoiding the power-consumed routing logic. Synthesized with 28nm CMOS technology, SWPU can enable 0.56V-to-1.0V supply voltage with a maximum frequency of 675 MHz. It achieves a 50.1% higher pruning ratio than structured pruning and $1.53\times $ higher energy efficiency than unstructured pruning. The peak energy efficiency of SWPU is 126.04TFLOPS/W, outperforming the state-of-the-art training processor by $1.67\times $. When training a ResNet-18 model, SWPU reduces $3.72\times $ energy and offers $4.69\times $ speedup than previous sparse training processors. [ABSTRACT FROM AUTHOR]

This paper proposes CMOS synapse and neuron for use in spiking neural networks to perform cognitive functions in a bio-inspired manner. The proposed synapse can trace the eligibility of the timing relationship between pre- and post-synaptic spikes, supporting a bio-plausible local learning rule called the spike timing-dependent plasticity (STDP) in an energy- and area-efficient manner. The proposed neuron can support neural functions such as synaptic current integration, threshold-based firing, neuronal leaking, membrane potential resetting, and adjustable refractory period with improved energy and area efficiency. The STDP curve shape of the synapse and the firing rate of the neuron can be adjusted as desired. Their variability due to process, voltage, and temperature (PVT) variations can also be minimized. The proposed CMOS neuron and synapse circuits were designed in a 28-nm CMOS process. The performance evaluation results indicate that the proposed synapse reduces energy consumption and area by up to 94% and 43% compared to conventional CMOS synapses. They also indicate that the proposed neuron achieves energy and area reductions of 37% and 23%, respectively, compared to conventional CMOS neurons. An associative neural network composed of the proposed neuron and synapse was designed to verify that they together work well for performing a cognitive function of associative learning and inferencing. [ABSTRACT FROM AUTHOR]

The divider is one of the most complex hardware units in Stochastic Computing (SC); even though several new designs have been presented to reduce the computation latency of the conventional divider, all of them still require a considerable number of clock cycles. Moreover, they incur in low performance due to the employed arithmetic computational scheme. In this paper, a Delta Sigma Modulator (DSM) based stochastic divider is proposed. As an entirely digital circuit, the proposed divider offers the best computation latency and accuracy over all existing stochastic dividers found in the technical literature (with a typical reduction between 66.8% and 96.9% in the number of clock cycles and a reduction from $10^{\mathrm {-3.4}}$ to $10^{\mathrm {-3.9}}$ in the average mean square error for a 10-bit resolution). An SC-based Neural Network (NN) is considered as an initial case study to evaluate the advantages of the proposed design in an emerging application; results show that the proposed divider enables an SC-based NN to achieve a higher classification accuracy and hardware efficiency than existing designs. To show the flexibility of the proposed divider design, its application to Sobol-based sequences is also presented; also in this case, its superiority over other designs is confirmed. These features make the proposed design very attractive for hardware-constrained platforms; moreover, such a novel design approach that incorporates ideas from analog/mixed signal circuit design into a digital circuit design, can motivate other researchers to design efficient SC designs using similar schemes. [ABSTRACT FROM AUTHOR]

This paper deals with modeling and simulation of a new family of two-terminal devices fabricated with vanadium dioxide material. Such devices allow realization of very compact relaxation nano-oscillators that can be connected electronically to form arrays of coupled oscillators. Challenging applications of oscillator arrays include the realization of multiphase signal generators and massively parallel brain-inspired neurocomputing. In the paper, a circuit-level model of the vanadium dioxide device is provided which enables extensive electrical simulations of oscillator systems built on the device. The proposed model is exploited to explain the dynamics of vanadium dioxide relaxation oscillators as well as to accomplish a robust parameter design. Applications to the realization of voltage-controlled oscillators and of multiphase oscillator arrays are illustrated. [ABSTRACT FROM AUTHOR]

As model size and the number of layers increase, Deep Neural Networks (DNNs) demand enormous computational power and throughput to meet exceedingly high prediction accuracy’s of today’s machine learning (ML) applications. Spatial hardware accelerators have been proposed that optimize the dataflow and exploit sparsity to provide a significant decrease in power consumption. As spatial architectures are traditionally designed with metallic interconnects, significant power is expended for data movement for different dataflows. In this paper, we exploit extended wireless technology to design a power-efficient and high-throughput DNN accelerator, e-WiNN, that can be configured for all representative dataflows and arithmetic precisions. We leverage novel circuit design by utilizing Dadda-algorithm based Multiply-and-Accumulate (MAC) circuits for 4-bit, 8-bit and 16-bit inputs to reduce area, power and delay constraints in 14 nm predictive technology. Our novel wireless transmitter integrates on- off keying (OOK) modulator with power amplifier that results in significant energy savings. To reduce the area overhead, we cluster wireless transceivers into groups of four such that both weights and input features can be effectively multicast to reduce the data movement. The energy efficient transceiver circuit is implemented in state-of-the-art BSIM 32 nm FinFET technology model and our link budget considers required RF power for different frequencies and inter-PE distance at three different antenna directivities including isotropic. Our detailed RTL modeling and cycle-accurate simulation results show that e-WiNN achieves 36.3% latency reduction and 76.1% energy saving when compared to state-of-art wire interconnected accelerators; 70.3% area reduction and 41.6% energy saving at the cost of 11% latency increase when compared to prior wireless accelerators on various neural networks (AlexNet, VGG16, and ResNet-9/50). [ABSTRACT FROM AUTHOR]

This paper deals with global synchronization problem of multiple discrete-time Markovian jump memristor-based neural networks (DTMJMNNs) with mixed mode-dependent delays via a novel event-triggered impulsive coupling control (ETICC). The parameters of the multiple DTMJMNNs and the mixed time delays (both discrete and distributed delays) switch randomly according to a Markov chain. In the ETICC strategy, the controller does not work all the time, but only works at impulse instants determined by specific events. In particular, the coupling matrix can be non-Laplacian. By using the Lyapunov stability theory, linear matrix inequalities (LMIs), and the Kronecker product, some sufficient conditions for global synchronization of multiple DTMJMNNs under the event-triggered strategy are derived. Two examples are presented to test the validity of the theoretical analysis results. [ABSTRACT FROM AUTHOR]

Convolutional neural networks (CNNs) are widely employed in modern computer vision algorithms, where the input image is convolved iteratively by many filters to extract the knowledge behind it. However, while the depth of convolutional layers gets deeper and deeper in recent years, the enormous computational complexity makes it difficult to be deployed on embedded systems with limited hardware resources. In this paper, inspired by rate-distortion optimization in image and video coding, we propose a computation-performance optimization (CPO) method to remove the redundant convolution filters in a CNN with performance constraints. To prove the effectiveness of the proposed method, CPO is applied to the networks for image super-resolution and image classification. Under almost the same PSNR drop and accuracy drop for performance evaluation in these two tasks, we can achieve the best parameter and computation reduction when compared with previous works. [ABSTRACT FROM AUTHOR]

This paper presents SensorNet which is a scalable and low-power embedded deep convolutional neural network (DCNN), designed to classify multimodal time series signals. Time series signals generated by different sensor modalities with different sampling rates are first converted to images (2-D signals), and then DCNN is utilized to automatically learn shared features in the images and perform the classification. SensorNet: 1) is scalable as it can process different types of time series data with variety of input channels and sampling rates; 2) does not need to employ separate signal processing techniques for processing the data generated by each sensor modality; 3) does not require expert knowledge for extracting features for each sensor data; 4) makes it easy and fast to adapt to new sensor modalities with a different sampling rate; 5) achieves very high detection accuracy for different case studies; and 6) has a very efficient architecture which makes it suitable to be deployed at Internet of Things and wearable devices. A custom low-power hardware architecture is also designed for the efficient deployment of SensorNet at embedded real-time systems. SensorNet performance is evaluated using three different case studies including physical activity monitoring, stand-alone tongue drive system, and stress detection, and it achieves an average detection accuracy of 98%, 96.2%, and 94% for each case study, respectively. We implement SensorNet using our custom hardware architecture on Xilinx FPGA (Artix-7) which on average consumes 246- $\mu \text{J}$ energy. To further reduce the power consumption, SensorNet is implemented using application-specified integrated circuit at the post layout level in 65-nm CMOS technology which consumes approximately $8\times $ lower power compared to the FPGA implementation. In addition, SensorNet is implemented on NVIDIA Jetson TX2 SoC (CPU + GPU) and compared to TX2 single-core CPU and GPU implementations, FPGA-based SensorNet obtains $15\times $ and $4\times $ improvement in energy consumption. [ABSTRACT FROM AUTHOR]

This paper proposes a battery management system, including a fast battery charger, battery aging diagnosis, and charge estimation and balancing. The charger adopts a single-inductor single-input dual-output architecture to achieve charge balancing among battery cells. Interleaved pulse charging is proposed to reduce the charging time and slow down the aging process of batteries as well. This method also significantly suppresses the variations of the temperature of battery cells and is beneficial to the implementation of charge balancing. An artificial neural network is proposed to detect the state of health (SOH) of battery cells and improve the accuracy of the state of charge (SOC) estimation. The prototype is implemented in TSMC 0.35- $\mu \text{m}$ process and TensorFlow tools are used. Measurement results show that the interleaved pulse charging reduces 30% variation of the battery temperature and saves 24% charging time when charging four battery cells concurrently. A mean absolute error of SOC estimation of 0.35% is achieved in this work. [ABSTRACT FROM AUTHOR]

In this paper, the observer based neural adaptive finite-time containment control strategy for non-strict feedback nonlinear multi-agent systems is studied. The finite-time command filter is used to overcome the explosion of complexity problem and the established fractional power based error compensation signal is applied to compensate the filtering error caused by the filter. The distributed finite-time command filtered backstepping control method combines with the neural adaptive control technology and state observer is given, which ensures the containment control errors reach to the desired neighborhood of the origin in finite-time in the presence of uncertain dynamics and unmeasurable states in the system. The given numerical simulations show the effectiveness of the proposed control strategy. [ABSTRACT FROM AUTHOR]

This paper investigates the secure consensus tracking problem for continuous-time nonlinear multiagent systems with sensor attacks. By designing a secure data selector, the unattacked output data is extracted from a group of output measurements under sparse sensor attacks. Then, by virtue of the obtained data and neural networks, a state observer is constructed to estimate the unavailable system states, where the convex combination theory is introduced to reduce the difficulty of solving observation gains. To utilize the limited communication resources reasonably, a novel saturated threshold event-triggered control strategy is proposed to reduce control updates, and then each update is encoded into a binary signal (0 or 1) to further reduce the occupation of communication bandwidth. The designed control scheme ensures that all closed-loop signals are semi-globally uniformly ultimately bounded, and its effectiveness is verified via a simulation of attitude control of unmanned aerial vehicles. [ABSTRACT FROM AUTHOR]

This paper investigates drive-response synchronization of a class of neural networks with time-varying delays and discontinuous activations. Discontinuous state feedback controller and adaptive controller are designed such that the considered model can realize exponential complete synchronization. Moreover, the convergence rate is explicitly estimated when state feedback control is utilized. The obtained results are also applicable to neural networks with continuous activations since they are a special case of neural networks with discontinuous activations. Results of this paper improve corresponding ones which only quasi-synchronization can be achieved for neural networks with discontinuous activations. Finally, numerical simulations are given to verify the effectiveness of the theoretical results. [ABSTRACT FROM PUBLISHER]

Deep neural networks are biologically inspired class of algorithms that have recently demonstrated the state-of-the-art accuracy in large-scale classification and recognition tasks. Hardware acceleration of deep networks is of paramount importance to ensure their ubiquitous presence in future computing platforms. Indeed, a major landmark that enables efficient hardware accelerators for deep networks is the recent advances from the machine learning community that have demonstrated the viability of aggressively scaled deep binary networks. In this paper, we demonstrate how deep binary networks can be accelerated in modified von Neumann machines by enabling binary convolutions within the static random access memory (SRAM) arrays. In general, binary convolutions consist of bit-wise exclusive-NOR (XNOR) operations followed by a population count (popcount). We present two proposals: one based on charge sharing approach to perform vector XNOR and approximate popcount and another based on bit-wise XNOR followed by a digital bit-tree adder for accurate popcount. We highlight the various tradeoffs in terms of circuit complexity, speed-up, and classification accuracy for both the approaches. Few key techniques presented as a part of the manuscript are the use of low-precision, low-overhead analog-to-digital converter (ADC), to achieve a fairly accurate popcount for the charge-sharing scheme and proposal for sectioning of the SRAM array by adding switches onto the read-bitlines, thereby achieving improved parallelism. Our results on benchmark image classification datasets for CIFAR-10 and SVHN on a binarized neural network architecture show energy improvements of up to $6.1\times $ and $2.3\times $ for the two proposals, compared to conventional SRAM banks. In terms of latency, improvements of up to $15.8\times $ and $8.1\times $ were achieved for the two respective proposals. [ABSTRACT FROM AUTHOR]

Current advances in emerging memory technologies enable novel and unconventional computing architectures for high-performance and low-power electronic systems, capable of carrying out massively parallel operations at the edge. One emerging technology, ReRAM, also known to belong in the family of memristors (memory resistors), is gathering attention due to its attractive features for logic and in-memory computing and benefits which follow from its technological attributes, such as nanoscale dimensions, low-power operation, and multi-state programming. At the same time, the design with CMOS is quickly reaching its physical and functional limitations, and further research toward novel logic families, such as threshold logic gates (TLGs), is scoped. In this paper, we introduce a physical implementation of a memristor-based current-mode TLG (MCMTLG) circuit and validate its design and operation through multiple experimental setups. We demonstrate two-input, three-input, and four-input MCMTLG configurations and showcase their reconfiguration capability. This is achieved by varying memristive weights arbitrarily for shaping the classification decision boundary, thus showing a promise as an alternative hardware-friendly implementation of artificial neural networks. Through the employment of real memristor devices as the equivalent of synaptic weights in TLGs, we are realizing components that can be used toward an in silico classifier. [ABSTRACT FROM AUTHOR]

This paper develops a neural network for solving the general nonsmooth convex optimization problems. The proposed neural network is modeled by a differential inclusion. Compared with the existing neural networks for solving nonsmooth convex optimization problems, this neural network has a wider domain for implementation. Under a suitable assumption on the constraint set, it is proved that for a given nonsmooth convex optimization problem and sufficiently large penalty parameters, any trajectory of the neural network can reach the feasible region in finite time and stays there thereafter. Moreover, we can prove that the trajectory of the neural network constructed by a differential inclusion and with arbitrarily given initial value, converges to the set consisting of the equilibrium points of the neural network, whose elements are all the optimal solutions of the primal constrained optimization problem. In particular, we give the condition that the equilibrium point set of the neural network coincides with the optimal solution set of the primal constrained optimization problem and the condition ensuring convergence to the optimal solution set in finite time. Furthermore, illustrative examples show the correctness of the results in this paper, and the good performance of the proposed neural network. [ABSTRACT FROM AUTHOR]

Multistable networks have attracted much interest in recent years, since multistability is of primary importance for some applications of recurrent neural networks where monostability exhibits some restrictions. This paper focuses on the analysis of dynamical property for a class of additive recurrent neural networks with nonsaturating linear threshold transfer functions. A milder condition is derived to guarantee the boundedness and global attractivity of the networks as compared to that presented in [6]. Dynamical properties of the equilibria of two-dimensional networks are analyzed theoretically, and the relationships between the equilibria features and network parameters (synaptic weights and external inputs) are revealed. In addition, the sufficient and necessary conditions for coexistence of multiple equilibria are obtained, which confirmed the observations in [14] with a cortex-inspired silicon circuit. The results obtained in this paper are applicable to both symmetric and nonsymmetric networks. Simulation examples are used to illustrate the theory developed in this paper. [ABSTRACT FROM AUTHOR]

In this paper, the sensitivity of the neural network (NN) outputs to device parameter uncertainties (non-idealities) in inverter-based memristor (IM) crossbar neuromorphic circuits is mathematically modeled and verified using exhaustive circuit and system-level simulations. The NN sensitivity is obtained by modeling the sensitivity of the IM neuron output to the non-idealities of its circuit elements. The analysis reveals a higher sensitivity of the output voltage of the IM neuron to the non-idealities of the inverters compared to the conductance variation of the memristors. Among the inverter non-idealities, horizontal shift of the inverters voltage transfer characteristic (VTC) shows the highest impact on the output voltage of the neuron. To reduce the accuracy loss due to the variations, a training approach which includes a sensitivity term in the cost function of the training phase, is suggested. The achievable improvements through the said NN training approach are evaluated. In the evaluation, the California Housing, MNIST, and Fashion MNIST datasets are employed. The results show up to 50% reduction in the NN output variations in the presence of circuit elements’ non-idealities. [ABSTRACT FROM AUTHOR]

Due to the low complexity in arithmetic unit design, stochastic computing (SC) has attracted considerable interest to implement Artificial Neural Networks (ANNs) for resources-limited applications, because ANNs must usually perform a large number of arithmetic operations. To attain a high computation accuracy in an SC-based ANN, extended stochastic logic is utilized together with standard SC units and thus, a stochastic divider is required to perform the conversion between these logic representations. However, the conventional divider incurs in a large computation latency, so limits an SC implementation for ANNs used in applications needing high performance. Therefore, there is a need to design fast stochastic dividers for SC-based ANNs. Recent works (e.g., a binary searching and triple modular redundancy (BS-TMR) based stochastic divider) are targeting a reduction in computation latency, while keeping the same accuracy compared with the traditional design. However, this divider still requires $N$ iterations to deal with $2^{N}$ -bit stochastic sequences, and thus the latency increases in proportion to the sequence length. In this paper, a decimal searching and TMR (DS-TMR) based stochastic divider is initially proposed to further reduce the computation latency; it only requires two iterations to calculate the quotient, so regardless of the sequence length. Moreover, a trade-off design between accuracy and hardware is also presented. An SC-based Multi-Layer Perceptron (MLP) is then considered to show the effectiveness of the proposed dividers over current designs. Results show that when utilizing the proposed dividers, the MLP achieves the lowest computation latency while keeping the same classification accuracy; although incurring in an area increase, the overhead due to the proposed dividers is low over the entire MLP. When using as combined metric for both hardware design and computation complexity the product of the implementation area, latency, power and number of clock cycles, the proposed designs are also shown to be superior to the SC-based MLPs (at the same level of accuracy) employing other dividers found in the technical literature as well as the commonly used 32-bit floating point implementation. [ABSTRACT FROM AUTHOR]

In this paper, a method for offline training of inverter-based memristive neural networks (IM-NNs), called ERIM, is presented. In this method, the output voltage of the inverter is modeled very accurately by considering the loading effect of the memristive crossbar. To properly choose the size of each inverter, its output load and the required slope of its voltage transfer characteristic (VTC) for an acceptable level of resiliency to the circuit element non-idealities are taken into account. The efficacy of ERIM is investigated by comparing its accuracy to those of two recently proposed offline training methods for IM-NNs (RIM and PHAX). The study is performed using IRIS, BCW, MNIST, and Fashion MNIST datasets. Simulation results show that 72% (56%) reduction in average energy consumption of the trained networks is achieved compared to RIM (PHAX) thanks to proper sizing of the inverters. In addition, due to the higher accuracy of the NN mathematical model, ERIM results in significant improvements in the match between the results of high-level modeling and HSPICE simulations while exhibiting lower sensitivity to circuit element variations. [ABSTRACT FROM AUTHOR]

Nowadays, research, modeling, simulation and realization of brain-like systems to reproduce brain behaviors have become urgent requirements. In this paper, neural bursting and synchronization are imitated by modeling two neural network models based on the Hopfield neural network (HNN). The first neural network model consists of four neurons, which correspond to realizing neural bursting firings. Theoretical analysis and numerical simulation show that the simple neural network can generate abundant bursting dynamics including multiple periodic bursting firings with different spikes per burst, multiple coexisting bursting firings, as well as multiple chaotic bursting firings with different amplitudes. The second neural network model simulates neural synchronization using a coupling neural network composed of two above small neural networks. The synchronization dynamics of the coupling neural network is theoretically proved based on the Lyapunov stability theory. Extensive simulation results show that the coupling neural network can produce different types of synchronous behaviors dependent on synaptic coupling strength, such as anti-phase bursting synchronization, anti-phase spiking synchronization, and complete bursting synchronization. Finally, two neural network circuits are designed and implemented to show the effectiveness and potential of the constructed neural networks. [ABSTRACT FROM AUTHOR]

In emerging Spiking Neural Network (SNN) based neuromorphic hardware design, energy efficiency and on-line learning are attractive advantages mainly contributed by bio-inspired local learning with nonlinear dynamics and at the cost of associated hardware complexity. This paper presents a novel SNN design employing fast COordinate Rotation DIgital Computer (CORDIC) algorithm to achieve fast spike timing–dependent plasticity (STDP) learning with high hardware efficiency. In this study, a system design and evaluation method of CORDIC-based SNN is proposed for finding optimal CORDIC type and precision, from theoretical CORDIC-level error to application-level learning performance. From the proposed design and evaluation method, a reconfigurable SNN design based on fast-convergence CORDIC is designed to achieve high classification accuracy on MNIST, fast on-line learning and good energy efficiency. By utilizing SNN’s fault tolerance and time-division-multiplexing (TDM) strategy, the reconfigurable SNN design employs 8-bit fast-convergence CORDIC and TDM-based hardware accelerator for high efficiency. FPGA implementation results confirm that the proposed fast-convergence CORDIC SNN design outperforms the state-of-the-art CORDIC method by 38.5%−45.3% in terms of learning speed and energy efficiency, with the STDP learning of 30.2 ns/SOP, energy efficiency of 176.6 pJ/SOP, processing speed of 6.1 ms/image, and on-line learning convergence of 21.4 s (time to reach the final accuracy, on average), on MNIST benchmark. [ABSTRACT FROM AUTHOR]

Nonvolatile memory (NVM) based neural network can directly perform in situ computation in memory to significantly reduce energy consumption resulting from the data movement. However, the energy consumption by the analog-to-digital converter (ADC) restricts the efficiency of the mixed-signal in-memory computing macro. The rate coding spike-driven in-memory computing macro can increase the energy efficiency via eliminating the ADC, but the improvement is limited because substantial energy is consumed for the coding of multiple spikes. In this work, we propose a discrete temporal coding spike-driven in-memory computing macro, including input coding scheme, weight mapping method, and improved leaky integrate-and-fire (LIF) neuron circuit, to perform the efficient forward inference of deep neural networks based on NVM array. We then optimize the designment of the proposed in-memory computing macro to mitigate the neural network accuracy loss due to the nonlinearity of the LIF neuron and voltage drop caused by interconnect resistance. Because the temporal coding scheme reduces spike numbers and the improved-LIF circuit simultaneously integrates two bit-lines current corresponding to positive and negative weight, the proposed macro achieves 46.63TOPS/W energy efficiency and 1.92TOPS throughput for 3bit temporal coding precision. [ABSTRACT FROM AUTHOR]

Large-scale simulation of spiking neural networks on hardware with a remarkable resemblance to their mathematical models is a key objective of the neuromorphic discipline. This issue is, however, considerably resource-intensive due to the presence of nonlinear terms in neuron models. This paper proposes a novel uniform piecewise linear segmentation approach for nonlinear function evaluations. Employing the proposed approach, we present a uniform-segmented adaptive exponential neuron model capable of accurately producing various responses exhibited by the original model and suitable for efficient large-scale implementation. In contrast to previous nonuniform-segmented neuron models, the proposed model enjoys a high-speed and extremely simple segment address encoder unit regardless of the number of segments. Simulations and hardware implementations on FPGA confirm that the proposed model significantly outperforms previous works in terms of resource utilization, accuracy, and maximum clock frequency. Moreover, employing the resource sharing technique, a fully pipelined hardware-efficient network containing one neuron circuit is proposed aimed at simulating a large number of neurons in both real and accelerated time scales. To assess the performance of the proposed network as a high-speed accelerator, four networks composed of 1k to 4k neurons are simulated using both FPGA and standard PC workstation. Results demonstrate that the proposed fully pipelined network accelerates the simulation process, on average, 89.96 times compared with the computer-based simulation. [ABSTRACT FROM AUTHOR]

Despite the increasing popularity of deep neural networks (DNNs), they cannot be trained efficiently on existing platforms, and efforts have thus been devoted to designing dedicated hardware for DNNs. In our recent work, we have provided direct support for the stochastic gradient descent (SGD) training algorithm by constructing the basic element of neural networks, the synapse, using emerging technologies, namely memristors. Due to the limited performance of SGD, optimization algorithms are commonly employed in DNN training. Therefore, DNN accelerators that only support SGD might not meet DNN training requirements. In this paper, we present a memristor-based synapse that supports the commonly used momentum algorithm. Momentum significantly improves the convergence of SGD and facilitates the DNN training stage. We propose two design approaches to support momentum: 1) a hardware friendly modification of the momentum algorithm using memory external to the synapse structure, and 2) updating each synapse with a built-in memory. Our simulations show that the proposed DNN training solutions are as accurate as training on a GPU platform while speeding up the performance by $886\times $ and decreasing energy consumption by $7\times $ , on average. [ABSTRACT FROM AUTHOR]

Emerging convolutional neural networks (CNNs) tend to be designed with varied per-layer data widths and sparse representations. However, these two features, which bring many redundant computations, have not been exploited simultaneously in existing hardware architectures for CNNs. This paper proposes an energy-quality scalable architecture, namely folded precision-adjustable processor (FPAP), to eliminate all computational redundancies by using folding techniques. On one hand, FPAP decomposes the dominant multiply-accumulate (MAC) operations into multiple adds and folds them into single arithmetic unit. Only effective adds (or part of them) are then calculated serially. Thus, FPAP can adapt to different per-layer data widths and enable precision-adjustable approximate computing. Particularly, FPAP adaptively selects either activation or weight to be decomposed in every single MAC to minimize the total number of adds and clock cycles. On the other hand, a 1-D convolution is undertaken by a multi-tap transposed finite impulse response (FIR) filter, which is folded into one tap to skip MACs with zero weights or activations. Besides, a judicious delay element remapping scheme and a novel genetic algorithm-based kernel reallocation scheme, are developed to reduce the power consumption in a folded FIR filter and mitigate the load imbalance issue caused by irregular sparsity, respectively. With all these optimizations, FPAP is able to reach comparable or even faster processing speed over the corresponding unfolded design in sparse CNNs while consuming smaller area. Experimental results on real CNN models demonstrate that FPAP can scale its energy efficiency from 4.28 to 23.63 TOP/s/W, and area efficiency from 37.79 to 164.15GOP/s/mm2, respectively, under the TSMC 28-nm HPC CMOS technology. [ABSTRACT FROM AUTHOR]

Learning vector quantization (LVQ) neural networks have already been successfully applied for image compression and object recognition. In this paper, we propose a modular and reconfigurable pipeline architecture (MRPA) for LVQ. The MRPA consists of dynamically reconfigurable modules and realizes a run-time and on-chip configuration for recognition and learning. Prototype fabrication in 65-nm CMOS technology verifies high integration density and memory-utilization efficiency, good performance, and considerable flexibility in vector dimensionality, number of weight-vectors, and adaption strategies. Compared with the embedded microprocessors, which rely on single-instruction-multiple-data processing, the developed prototype increases the performance of both recognition and learning operations. The MRPA prototype shows improvements by factors of approximately 40 and 101 on the well-established performance metrics million connections per second for recognition and million connection updates per second for learning, respectively. [ABSTRACT FROM AUTHOR]

In this paper, a hardware-realized neuromorphic system for pattern recognition is presented. The system directly captures images from the environment, and then conducts classification using a single layer neural network. Metal-oxide resistive random access memory (RRAM) is used as electronic synapses, and threshold-controlled neurons are proposed as postsynaptic neurons to save the system area and simplify the operation. In the proposed threshold-controlled neuron, no capacitor is utilized, which contributes to higher integration density. The total energy consumption of RRAM synapses for classifying an example is $0.31\mu \text{J}$ on average. The proposed system has been implemented on hardware, and has been experimentally demonstrated to show the capability of pattern recognition. [ABSTRACT FROM AUTHOR]

This paper proposes a novel artificial neural network (ANN) based control method for a dc/dc buck converter. The ANN is trained to implement optimal control based on approximate dynamic programming (ADP). Special characteristics of the proposed ANN control include: 1) The inputs to the ANN contain error signals and integrals of the error signals, enabling the ANN to have PI control ability; 2) The ANN receives voltage feedback signals from the dc/dc converter, making the combined system equivalent to a recurrent neural network; 3) The ANN is trained to minimize a cost function over a long time horizon, making the ANN have a stronger predictive control ability than a conventional predictive controller; 4) The ANN is trained offline, preventing the instability of the network caused by weight adjustments of an on-line training algorithm. The ANN performance is evaluated through simulation and hardware experiments and compared with conventional control methods, which shows that the ANN controller has a strong ability to track rapidly changing reference commands, maintain stable output voltage for a variable load, and manage maximum duty-ratio and current constraints properly. [ABSTRACT FROM AUTHOR]

The issue of finite-time stabilization (FTS) for the memristor-based inertial neural networks (MINNs) with mixed time-varying delays (MTVDs) is researched by virtue of a new analytical method in this brief. First, an appropriate reliable control strategy is proposed for MINNs, which takes the influence of actuator failures into account. Second, by combining Lyapunov functional theory with new analysis techniques, novel theoretical results to guarantee the FTS for the concerned MINNs are acquired, and the desired reliable controller gains are obtained simultaneously. In additions, compared with the previous research works, the FTS results obtained in this paper are established directly from the MINNs themselves without using variable transformation method. In the end, two simulations are exploited to show the correctness and practicability of the acquired theoretical results. [ABSTRACT FROM AUTHOR]

This paper provides new delay-dependent conditions that guarantee the robust exponential stability of stochastic Hopfield type neural networks with time-varying delays and parameter uncertainties. Both the cases of the time-varying delays which are differentiable and may not be differentiable are considered. The stability conditions are derived by using the recently developed free-weighting matrices technique and expressed in terms of linear matrix inequalities. Numerical examples are provided to demonstrate the effectiveness of the proposed stability criteria. It is shown that the proposed stability results are less conservative than some previous ones in the literature. [ABSTRACT FROM AUTHOR]

In this paper, we propose O4-DNN, a high-performance FPGA-based architecture for convolutional neural network (CNN) accelerators relying on operation packing and out-of-order (OoO) execution for DSP blocks augmented with LUT-based glue logic. The high-level architecture is comprised of a systolic array of processing elements (PEs), supporting output stationary dataflow. In this architecture, the computational unit of each PE is realized by using a DSP block as well as a small number of LUTs. Given the limited number of DSP blocks in FPGAs, the combination (DSP block and some LUTs) provides more computational power obtainable through each DSP block. The proposed computational unit performs eight convolutional operations on five input operands where one of them is an 8-bit weight and the others are four 8-bit input feature (IF) maps. In addition, to improve the energy efficiency of the proposed computational unit, we present an approximate form of the unit suitable for neural network applications. To reduce the memory bandwidth as well as increase the utilization of the computational units, a data reusing technique based on the weight sharing is also presented. To improve the performance of the proposed computational unit further, an addressing approach for computing the partial sums out-of-order is proposed. The efficacy of the architecture is assessed using two FPGA devices executing four state-of-the-art neural networks. Experimental results show that this architecture leads to, on average (up to), $2.5\times $ ($3.44\times$) higher throughput compared to a baseline structure. In addition, on average (maximum of), 12% (40%) energy efficiency improvement is achievable by employing the O4-DNN compared to the baseline structure. [ABSTRACT FROM AUTHOR]

Hardware design of deep convolutional neural networks (CNNs) faces challenges of high computational complexity and data bandwidth as well as huge divergence in different CNN network layers, in which the throughput of the convolutional layer would be bounded by available hardware resource, and throughput of the fully connected layer would be bounded by available data bandwidth. Thus, a highly flexible and efficient design is desired to meet these needs. This paper presents an end-to-end CNN accelerator that maximizes hardware utilization with run-time configurations of different kernel sizes. It also minimizes data bandwidth with the output first strategy to improve the data reuse of the convolutional layers by up to $300\times \sim 600\times $ compared with the non-reused case. The whole CNN implementation of the target network is generated optimally for both hardware and data efficiency under design resource constraints, which can be run-time reconfigured by the layer optimized parameters to achieve real-time and end-to-end CNN acceleration. An implementation example for AlexNet consumes a 1.783 M gate count for 216 MACs and a 142.64 kb internal buffer with TSMC 40 nm process, and achieves 99.7 and 61.6 f/s under 454 MHz clock frequency for the convolutional layers and the whole AlexNet, respectively. [ABSTRACT FROM PUBLISHER]

The complexity and size of recent deep neural network (DNN) models have increased significantly in pursuit of high inference accuracy. Chiplet-based accelerator is considered a viable scaling approach to provide substantial computation capability and on-chip memory for efficient process of such DNN models. However, communication using metallic interconnects in prior chiplet-based accelerators poses a major challenge to system performance, energy efficiency, and scalability. Photonic interconnects can adequately support communication across chiplets due to features such as distance-independent latency, high bandwidth density, and high energy efficiency. Furthermore, the salient ease of broadcast property makes photonic interconnects suitable for DNN inference which often incurs prevalent broadcast communication. In this paper, we propose a scalable chiplet-based DNN accelerator with photonic interconnects named ASCEND. ASCEND introduces (1) a novel photonic network that supports seamless intra- and inter- chiplet broadcast communication, and flexible mapping of diverse convolution layers, and (2) a tailored dataflow that exploits the ease of broadcast property and maximizes parallelism by simultaneously processing computations with shared input data. Simulation results using multiple DNN models show that ASCEND achieves 71% and 67% reduction in execution time and energy consumption, respectively, as compared to other state-of-the-art chiplet-based DNN accelerators with metallic or photonic interconnects. [ABSTRACT FROM AUTHOR]

This paper addresses the global \cal H\infty pinning synchronization problem for a class of directed networks with aperiodic sampled-data communications. Important yet challenging issues of how many and which nodes should be pinned for realizing global synchronization in a fixed directed network without external disturbances are first discussed. By using a combined tool from the input-delay approach and free-weighting matrices technique, some sufficient synchronizability conditions are then derived for such networks. Furthermore, a multi-step algorithm is designed to estimate the upper bound of the maximum allowable sampling intervals for achieving synchronization. Theoretical results are then extended to global \cal H\infty pinning synchronization in fixed and switched directed networks with external disturbances, showing that a finite \cal H\infty performance index can be guaranteed under some suitable conditions. Finally, numerical simulations are performed to demonstrate the effectiveness of the analytical results. [ABSTRACT FROM PUBLISHER]

Recent state-of-the-art deep convolutional neural networks (CNNs) have shown remarkable success in current intelligent systems for various tasks, such as image/speech recognition and classification. A number of recent efforts have attempted to design custom inference engines based on processing-in-memory (PIM) architecture, where the memory array is used for weighted sum computation, thereby avoiding the frequent data transfer between buffers and computation units. Prior PIM designs typically unroll each 3D kernel of the convolutional layers into a vertical column of a large weight matrix, where the input data needs to be accessed multiple times. In this paper, in order to maximize both weight and input data reuse for PIM architecture, we propose a novel weight mapping method and the corresponding data flow which divides the kernels and assign the input data into different processing-elements (PEs) according to their spatial locations. As a case study, resistive random access memory (RRAM) based 8-bit PIM design at 32 nm is benchmarked. The proposed mapping method and data flow yields $\sim 2.03\times $ speed up and $\sim 1.4\times $ improvement in throughput and energy efficiency for ResNet-34, compared with the prior design based on the conventional mapping method. To further optimize the hardware performance and throughput, we propose an optimal pipeline architecture, with ~50% area overhead, it achieves overall $913\times $ and $1.96\times $ improvement in throughput and energy efficiency, which are 132476 FPS and 20.1 TOPS/W, respectively. [ABSTRACT FROM AUTHOR]

Hardware accelerators for convolution neural networks (CNNs) enable real-time applications of artificial intelligence technology. However, most of the existing designs suffer from low hardware utilization or high area cost due to complex data flow. This paper proposes a hardware efficient vectorwise CNN accelerator that adopts a $3\times 3$ filter optimized systolic array using 1-D broadcast data flow to generate partial sum. This enables easy reconfiguration for different kinds of kernels with interleaved input or elementwise input data flow. This simple and regular data flow results in low area cost while attains high hardware utilization. The presented design achieves 99%, 97%, 93.7%, and 94% hardware utilization for VGG-16, ResNet-34, GoogLeNet, and Mobilenet, respectively. Hardware implementation with TSMC 40nm technology takes 266.9K NAND gate count and 191KB SRAM to support 168GOPS throughput while consumes only 154.98mW when running at 500MHz operating frequency, which has superior area and power efficiency than other designs. [ABSTRACT FROM AUTHOR]

Transcendental functions are commonly used in many fields such as nonlinear functions of artificial neural networks (ANNs). Due to nonlinearity of these functions, hardware implementation of the functions faces many challenges. Some methods cannot control the approximation accuracy beforehand, or they cannot approximate target functions accurately and efficiently. Other methods can only approximate a particular function without generality. A universal piecewise linear (PWL) approximation method is proposed to solve the problems existing in the above methods. In this paper, a general piecewise linear (PWL) approximation method with controllable maximum absolute error for transcend-ental functions is proposed. The method has the self-adaptive capability to choose the smallest number of segments under the constraint of a controllable maximum absolute error. Therefore, it requires fewer segments and incurs less hardware overhead. Moreover, it can approximate any transcendental functions and does not rely on any properties of the target function. Comparing our work with other methods used in three transcendental functions, including the sigmoid, hyperbolic tangent, and logarithmic function, our method improves the approximation accuracy by 4.5 times, 2.1 times, and 1.3 times, respectively; in addition, our method is more efficient and it reduces the circuit area by 49.9%, 22.4%, and 28.5%, respectively. [ABSTRACT FROM AUTHOR]

Always-on speech interfaces are prevailing in human-machine interaction, especially on wearable devices, Internet of Things, etc., which benefits from the recent breakthroughs in deep learning. For battery-powered devices, ultra-low power and real-time processing are critical. However, the massive memory access and computation of deep neural networks (DNNs) lead to long latency and huge energy consumption, which hinder their further integration in battery-powered devices. Extremely low-bit quantization shows the potential to enhance energy efficiency and speed by orders of magnitudes, but it suffers from the degradation of recognition accuracy. In this paper, we propose a binarized convolutional neural network (BCNN) based speech recognition processor, integrated with on-chip self-learning mechanism to compensate the accuracy loss caused by low-precision. We optimize the BCNN architecture by eliminating the computation redundancy, compressing the weights and tailoring approximate circuits. Fabricated in 28 nm CMOS, this processor supports real time speech recognition with power consumption of $141~\mu \text{W}$ and energy efficiency of 2.46 pJ/Neuron. Compared with state-of-the-art speech recognition implementations, this processor achieves $2.5\times $ reduction on energy consumption per neuron, and $8.0\times $ energy reduction per speech frame. [ABSTRACT FROM AUTHOR]

Computing-in-memory (CIM) is a promising approach to reduce the latency and improve the energy efficiency of deep neural network (DNN) artificial intelligence (AI) edge processors. However, SRAM-based CIM (SRAM-CIM) faces practical challenges in terms of area overhead, performance, energy efficiency, and yield against variations in data patterns and transistor performance. This paper employed a circuit-system co-design methodology to develop a SRAM-CIM unit-macro for a binary-based fully connected neural network (FCNN) layer of the DNN AI edge processors. The proposed SRAM-CIM unit-macro supports two binarized neural network models: an XNOR neural network (XNORNN) and a modified binary neural network (MBNN). To achieve compact area, fast access time, robust operations, and high energy-efficiency, our proposed SRAM-CIM uses a split-wordline compact-rule 6T SRAM and circuit techniques, including a dynamic input-aware reference generation (DIARG) scheme, an algorithm-dependent asymmetric control (ADAC) scheme, a write disturb-free (WDF) scheme, and a common-mode-insensitive small offset voltage-mode sensing amplifier (CMI-VSA). A fabricated 65-nm 4-Kb SRAM-CIM unit-macro achieved 2.4- and 2.3-ns product-sum access times for a FCNN layer using XNORNN and MBNN, respectively. The measured maximum energy efficiency reached 30.49 TOPS/W for XNORNN and 55.8 TOPS/W for the MBNN modes. [ABSTRACT FROM AUTHOR]