Search

For a class of nontriangular nonlinear systems in presence of unknown disturbances, we propose a predictor-based neural dynamic surface control (PNDSC) strategy in this paper. This nontriangular system is transformed via the mean value theorem, and a predictor is then constructed. To avoid an algebraic loop problem, partial state vectors are employed as input signals of neural networks (NNs) for approximating unknown dynamics, and compensation items are designed to compensate for approximation errors from NNs. Different from the traditional NDSC, the PNDSC in this paper utilizes prediction errors to update learning parameters for improving NNs’ learning behaviors with overlarge adaptive gains. On the basis of improved NNs’ approximation behaviors, a predictor-based NNs disturbance observer (PNNDO) is constructed for compensation for external disturbances and approximation errors from NNs. Furthermore, with predictors, a normalization method of weights is developed to reduce the number of online learning parameters. On the basis of the aforementioned result, measurement noises are taken into account in our predictor-based neural control strategy. We employ predictor states, rather than measurement information paralyzed by noises, in design of our control strategy. This reduces high-frequency oscillations in control input. A Lyapunov-based stability analysis shows that all signals are ultimately bounded in the closed-loop system. Finally, the effectiveness of the proposed control strategy is verified by a numerical example and a permanent magnet brushless DC motor system. [ABSTRACT FROM AUTHOR]

In this paper, we propose a low-power memory-based computing architecture, called selective computing architecture (SCA). It consists of multipliers and an LUT (Look-Up Table)-based component, that is multi-context ternary content-addressable memory (MC-TCAM). Either of them is selected by input-data conditions in neural-networks (NNs). Compared with quantized NNs, a higher accurate multiplication can be performed with low-power consumption in the proposed architecture. If input data stored in the MC-TCAM appears, the corresponding multiplication results for multiple weights are obtained. The MC-TCAM stores only shorter length of input data, resulting in achieving a low-power computing. The performance of the SCA is determined by three physical parameters concerning the configuration of MC-TCAM. The power dissipation of the target NN can be minimized by exploring these parameters in the design space. The hardware based on the proposed architecture is evaluated using TSMC 65 nm CMOS technology and MTJ model. In the case of speech command recognition, the power consumption at the multiplication of the first convolutional layer in a convolutional NN is reduced by 67% compared to the solution relying only on multipliers. [ABSTRACT FROM AUTHOR]

The memristor-based neuromorphic computing system (NCS) with emerging storage and computing integration architecture has drawn extensive attention. Because of the unique nonvolatility and programmability, the memristor is an ideal nano-device to realize neural synapses in VLSI circuit implementation of neural networks. However, in the hardware implementation, the performance of the memristive neural network is always affected by quantization error, writing error, and conductance drift, which seriously hinders its applications in practice. In this paper, a novel weight optimization scheme combining quantization and Bayesian inference is proposed to alleviate this problem. Specifically, the weight deviation in the memristive neural network is transformed into the weight uncertainty in the Bayesian neural network, which can make the network insensitive to unexpected weight changes. A quantization regularization term is designed and utilized during the training process of the Bayesian neural network, reducing the quantization error and improving the robustness of the network. Furthermore, a partial training method is raised to extend the applicability of the proposed scheme in large-scale neural networks. Finally, the experiments on a Multilayer Perceptron and LeNet demonstrate that the proposed weight optimization scheme can significantly enhance the robustness of memristive neural networks. [ABSTRACT FROM AUTHOR]

An automatic inductor design process is helpful to reduce the design cycle of radio frequency (RF) integrated circuit (IC). This paper proposed an efficient synthesis-analysis machine (SAM) for on-chip inductor synthesis and modeling, as well as an automatic dataset generation (ADG) topology for the generation of artificial neural networks (ANNs) training dataset. The SAM consists of a synthesis ANN, two analysis ANNs and a proposed self-inspection machine (SIM). For a given design request, the SAM synthesizes a layout first, followed by analyzing the performance of the layout automatically, and finally improves the layout confidence through self-inspection. Compared to the modeling functions of analysis ANN, the synthesis ANN behaves as an inverse model, of which the inputs are desired inductor performances while the outputs are geometrical parameters of the layout. However, multi-value problems might occur in obtaining geometries of inductors, suggesting with evidence from mathematic relationships between geometrical and electrical parameters of an inductor. A multi-valued training dataset will mislead the synthesis ANN, resulting in unqualified layouts. To deal with this issue, a solution space contraction technique (SSCM) is also proposed. Furthermore, the SIM suspends most of the failures by comparing the output of analysis ANN back to the input of synthesis ANN. An electromagnetic simulation tool is used for experiments, and the effectiveness of the proposed SAM is proven by 6,500 inductor samples. [ABSTRACT FROM AUTHOR]

This paper investigates drive-response synchronization of a class of neural networks with time-varying delays and discontinuous activations. Discontinuous state feedback controller and adaptive controller are designed such that the considered model can realize exponential complete synchronization. Moreover, the convergence rate is explicitly estimated when state feedback control is utilized. The obtained results are also applicable to neural networks with continuous activations since they are a special case of neural networks with discontinuous activations. Results of this paper improve corresponding ones which only quasi-synchronization can be achieved for neural networks with discontinuous activations. Finally, numerical simulations are given to verify the effectiveness of the theoretical results. [ABSTRACT FROM PUBLISHER]

Due to the low complexity in arithmetic unit design, stochastic computing (SC) has attracted considerable interest to implement Artificial Neural Networks (ANNs) for resources-limited applications, because ANNs must usually perform a large number of arithmetic operations. To attain a high computation accuracy in an SC-based ANN, extended stochastic logic is utilized together with standard SC units and thus, a stochastic divider is required to perform the conversion between these logic representations. However, the conventional divider incurs in a large computation latency, so limits an SC implementation for ANNs used in applications needing high performance. Therefore, there is a need to design fast stochastic dividers for SC-based ANNs. Recent works (e.g., a binary searching and triple modular redundancy (BS-TMR) based stochastic divider) are targeting a reduction in computation latency, while keeping the same accuracy compared with the traditional design. However, this divider still requires $N$ iterations to deal with $2^{N}$ -bit stochastic sequences, and thus the latency increases in proportion to the sequence length. In this paper, a decimal searching and TMR (DS-TMR) based stochastic divider is initially proposed to further reduce the computation latency; it only requires two iterations to calculate the quotient, so regardless of the sequence length. Moreover, a trade-off design between accuracy and hardware is also presented. An SC-based Multi-Layer Perceptron (MLP) is then considered to show the effectiveness of the proposed dividers over current designs. Results show that when utilizing the proposed dividers, the MLP achieves the lowest computation latency while keeping the same classification accuracy; although incurring in an area increase, the overhead due to the proposed dividers is low over the entire MLP. When using as combined metric for both hardware design and computation complexity the product of the implementation area, latency, power and number of clock cycles, the proposed designs are also shown to be superior to the SC-based MLPs (at the same level of accuracy) employing other dividers found in the technical literature as well as the commonly used 32-bit floating point implementation. [ABSTRACT FROM AUTHOR]

In this paper, we propose O4-DNN, a high-performance FPGA-based architecture for convolutional neural network (CNN) accelerators relying on operation packing and out-of-order (OoO) execution for DSP blocks augmented with LUT-based glue logic. The high-level architecture is comprised of a systolic array of processing elements (PEs), supporting output stationary dataflow. In this architecture, the computational unit of each PE is realized by using a DSP block as well as a small number of LUTs. Given the limited number of DSP blocks in FPGAs, the combination (DSP block and some LUTs) provides more computational power obtainable through each DSP block. The proposed computational unit performs eight convolutional operations on five input operands where one of them is an 8-bit weight and the others are four 8-bit input feature (IF) maps. In addition, to improve the energy efficiency of the proposed computational unit, we present an approximate form of the unit suitable for neural network applications. To reduce the memory bandwidth as well as increase the utilization of the computational units, a data reusing technique based on the weight sharing is also presented. To improve the performance of the proposed computational unit further, an addressing approach for computing the partial sums out-of-order is proposed. The efficacy of the architecture is assessed using two FPGA devices executing four state-of-the-art neural networks. Experimental results show that this architecture leads to, on average (up to), $2.5\times $ ($3.44\times$) higher throughput compared to a baseline structure. In addition, on average (maximum of), 12% (40%) energy efficiency improvement is achievable by employing the O4-DNN compared to the baseline structure. [ABSTRACT FROM AUTHOR]

Hardware design of deep convolutional neural networks (CNNs) faces challenges of high computational complexity and data bandwidth as well as huge divergence in different CNN network layers, in which the throughput of the convolutional layer would be bounded by available hardware resource, and throughput of the fully connected layer would be bounded by available data bandwidth. Thus, a highly flexible and efficient design is desired to meet these needs. This paper presents an end-to-end CNN accelerator that maximizes hardware utilization with run-time configurations of different kernel sizes. It also minimizes data bandwidth with the output first strategy to improve the data reuse of the convolutional layers by up to $300\times \sim 600\times $ compared with the non-reused case. The whole CNN implementation of the target network is generated optimally for both hardware and data efficiency under design resource constraints, which can be run-time reconfigured by the layer optimized parameters to achieve real-time and end-to-end CNN acceleration. An implementation example for AlexNet consumes a 1.783 M gate count for 216 MACs and a 142.64 kb internal buffer with TSMC 40 nm process, and achieves 99.7 and 61.6 f/s under 454 MHz clock frequency for the convolutional layers and the whole AlexNet, respectively. [ABSTRACT FROM PUBLISHER]

Hardware accelerators for convolution neural networks (CNNs) enable real-time applications of artificial intelligence technology. However, most of the existing designs suffer from low hardware utilization or high area cost due to complex data flow. This paper proposes a hardware efficient vectorwise CNN accelerator that adopts a $3\times 3$ filter optimized systolic array using 1-D broadcast data flow to generate partial sum. This enables easy reconfiguration for different kinds of kernels with interleaved input or elementwise input data flow. This simple and regular data flow results in low area cost while attains high hardware utilization. The presented design achieves 99%, 97%, 93.7%, and 94% hardware utilization for VGG-16, ResNet-34, GoogLeNet, and Mobilenet, respectively. Hardware implementation with TSMC 40nm technology takes 266.9K NAND gate count and 191KB SRAM to support 168GOPS throughput while consumes only 154.98mW when running at 500MHz operating frequency, which has superior area and power efficiency than other designs. [ABSTRACT FROM AUTHOR]

Transcendental functions are commonly used in many fields such as nonlinear functions of artificial neural networks (ANNs). Due to nonlinearity of these functions, hardware implementation of the functions faces many challenges. Some methods cannot control the approximation accuracy beforehand, or they cannot approximate target functions accurately and efficiently. Other methods can only approximate a particular function without generality. A universal piecewise linear (PWL) approximation method is proposed to solve the problems existing in the above methods. In this paper, a general piecewise linear (PWL) approximation method with controllable maximum absolute error for transcend-ental functions is proposed. The method has the self-adaptive capability to choose the smallest number of segments under the constraint of a controllable maximum absolute error. Therefore, it requires fewer segments and incurs less hardware overhead. Moreover, it can approximate any transcendental functions and does not rely on any properties of the target function. Comparing our work with other methods used in three transcendental functions, including the sigmoid, hyperbolic tangent, and logarithmic function, our method improves the approximation accuracy by 4.5 times, 2.1 times, and 1.3 times, respectively; in addition, our method is more efficient and it reduces the circuit area by 49.9%, 22.4%, and 28.5%, respectively. [ABSTRACT FROM AUTHOR]

The paper addresses convergence of solutions for a class of differential inclusions termed differential variational inequalities (DVIs). Each DVI describes the dynamics of a neural network (NN) evolving in a closed hypercube of \BBR ^n and defined by a continuously differentiable, cooperative and (possibly) nonreciprocal vector field f. The main result in the paper is that under a new condition on f, which is called strong Kamke-Müller (SKM) condition, the solution semiflow generated by the DVI is strongly order preserving (SOP) and hence it satisfies a Limit Set Dichotomy and enjoys generic convergence properties. A characterization of the SKM condition is given in terms of the interconnection properties of the Jacobian matrix of f. In the case where f is an affine, or a linear, vector field the considered DVIs include two relevant classes of NNs, namely, the linear systems operating on a closed hypercube, also known as linear systems in saturated mode (LSSMs), and the full-range (FR) model of cellular neural networks (CNNs). By applying the results to LSSMs it is obtained that any cooperative LSSM with a (possibly) nonsymmetric and fully interconnected matrix is generically convergent. Analogous results hold for FRCNNs. All the obtained convergence results hold in the general case where the DVIs, and the LSSMs and FRCNNs, possess multiple equilibrium points. [ABSTRACT FROM PUBLISHER]

In this paper, we propose a neural network modeled by a differential inclusion to solve a class of nonsmooth convex optimization problems, where the constraints are defined by a class of nonsmooth convex inequality constraints and a class of affine equality constraints. For any initial point, the solution of the proposed network is global existent, unique and uniformly bounded, which is just the “slow solution” of network. By the regularization item, without any estimation on the exact penalty parameters, the solution of proposed network is convergent to the optimal solution set of optimization problem. Moreover, when the feasible region satisfies another condition, the solution of proposed network converges to the feasible region in finite time and to the particular element in the optimal solution set with the smallest norm, which indicates that our proposed neural network is globally attractive. Some numerical examples are presented to illustrate the effectiveness of the proposed neural network for solving nonsmooth convex optimization problems. [ABSTRACT FROM AUTHOR]

This paper studies synchronization in an array of coupled neural networks with Markovian jumping and random coupling strength. The array of neural networks are coupled in a random fashion which is governed by Bernoulli random variable and each node has an interval time-varying delay. By designing a novel Lyapunov functional, using some inequalities and the properties of random variables, several delay-dependent synchronization criteria are derived for the coupled networks of continuous-time version. Discrete-time analogues of the continuous-time networks are also formulated and studied. Some new lemmas are developed to obtain less conservative synchronization criteria of both continuous-time model and its discrete-time analogues. Numerical examples of both continuous-time system and its discrete-time analogues are finally given to demonstrate the effectiveness of the theoretical results. [ABSTRACT FROM AUTHOR]

This paper focuses on studying the H∞ and generalized H2 filtering problems for a class of delayed neural networks. The time-varying delay is only required to be continuous and bounded. Delay-dependent criteria are proposed such that the resulting filtering error system is globally exponentially stable with a guaranteed H∞ or generalized H2 performance. It is also shown that the designs of the desired filters are achieved by solving a set of linear matrix inequalities, which can be facilitated efficiently by resorting to standard numerical algorithms. It should be noted that, based on a novel bounding technique, several slack variables are introduced to reduce the conservatism of the derived conditions. Three examples with simulation results are provided to illustrate the effectiveness and performance of the developed approaches. [ABSTRACT FROM AUTHOR]

In this paper, a novel paradigm for blind source separation in the presence of nonlinear mixtures is presented. In particular, the paper addresses the problem of post-nonlinear mixing followed by another instantaneous mixing system. This model is called here the post-nonlinear-linear model. The method is based on the use of the recently introduced flexible activation function whose control points are adaptively changed: a neural model based on adaptive B-spline functions is employed. The signal separation is achieved through an information maximization criterion. Experimental results and comparison with existing solutions confirm the effectiveness of the proposed architecture. [ABSTRACT FROM AUTHOR]

In the past few years, the demand for real-time hardware implementations of deep neural networks (DNNs), especially convolutional neural networks (CNNs), has dramatically increased, thanks to their excellent performance on a wide range of recognition and classification tasks. When considering real-time action recognition and video/image classification systems, latency is of paramount importance. Therefore, applications strive to maximize the accuracy while keeping the latency under a given application-specific maximum: in most cases, this threshold cannot exceed a few hundred milliseconds. Until now, the research on DNNs has mainly focused on achieving a better classification or recognition accuracy, whereas very few works in literature take in account the computational complexity of the model. In this paper, we propose an efficient computational method, which is inspired by a computational core of fully connected neural networks, to process convolutional layers of state-of-the-art deep CNNs within strict latency requirements. To this end, we implemented our method customized for VGG and VGG-based networks which have shown state-of-the-art performance on different classification/recognition data sets. The implementation results in 65-nm CMOS technology show that the proposed accelerator can process convolutional layers of VGGNet up to 9.5 times faster than state-of-the-art accelerators reported to-date while occupying 3.5 mm2. [ABSTRACT FROM AUTHOR]

This paper focuses on network-based master-slave synchronization for delayed neural networks through a remote controller. The insertion of communication networks in a master-slave synchronization scheme inevitably induces network delays, packet dropouts and stochastic fluctuations. The data packets may be received with a different temporal order from that they are sent due to the fact that the network-induced delay is time-varying. A logic data processor and a logic zero order hold are proposed in the master-slave synchronization framework. Then an error system for the master system and the slave system is formulated. By combining a generalized Jensen integral inequality and a convex combination technique, some synchronization criteria are derived to ensure the mean-square global exponential synchronization of state trajectories for the master system and the slave system. The controller gain matrix is obtained by solving a minimization problem in terms of linear matrix inequalities using a cone complementary technique. As a special case in which only network-induced delays and packet dropouts are occurred in the signal transmission channels, some results are also presented. Finally, two illustrative examples are provided to show the effectiveness and applicability of the proposed scheme. [ABSTRACT FROM AUTHOR]

This paper focuses on the problem of global exponential stability analysis of impulsive neural networks with variable delay. Three types of impulses are considered: the impulses are input disturbances; the impulses are "neutral" type (that is, they are neither helpful for stability of neural networks nor destabilizing); and the impulses are stabilizing. For each type of impulses, by using Lyapunov function and Razumikhin-type techniques, the sufficient conditions for global exponential stability are developed in terms of linear matrix inequalities with respect to suitable classes of impulse time sequences. The new sufficient stability conditions do not impose any restriction on the size of time-delay. Numerical examples are given which show our results are less conservative than the existing sufficient stability conditions. [ABSTRACT FROM AUTHOR]

Sequential Monte Carlo (SMC) methods, namely, particle filters, are powerful simulation techniques for sampling sequentially from a complex probability distribution. SMC can be used to solve some problems associated with nonlinear non-Gaussian probability distribution. Sampling is a key step for these methods and has vital effects on simulation results. Various sampling strategies have been proposed to improve the simulation results of SMC methods, but degeneracy of particles sometimes is very severe so that there are only a few particles having significant weights. Diversity of particle samples is reduced significantly so that only a few particles are used to represent the corresponding probability distribution. This kind of sampling is not reasonable to approximate probability distribution. This paper addresses a new method which can avoid the phenomenon of particle degeneracy. We split particles with very big weights into two small ones and use the strategy of neural network to adjust positions of tail particles in order to increase their weights. Another advantage is that this method can efficiently make simulation results approach the actual object. Our simulation results of the typical tracking problem show that not only the phenomenon of particle degeneracy is effectively avoided but also tracking results are much better than those of the traditional particle filters. Compared with the move-resample method, our method shows better results under the same conditions. [ABSTRACT FROM AUTHOR]

In this paper, the problem of stability analysis for a class of neural networks with distributed delays is investigated. Applying the M-matrix theory and new analysis technique, novel sufficient conditions for the existence, uniqueness, and global asymptotic stability of the equilibrium point of neural networks with distributed delays are derived. The new stability criteria can be applied to the case when the nondelayed terms cannot dominate the delayed terms, which have great significance in the design and application of neural networks with distributed delays. Three illustrative examples are presented which demonstrate the usefulness of the proposed results. [ABSTRACT FROM AUTHOR]

In this paper, we propose a neural network for ranking a given set of N numbers In 0(1/N) time. The ordering of a set of numbers based on their relative magnitudes, which is analogous to sorting,. is a fundamental operation in many algorithms. In comparison with other sorting networks reported in the literature, the,: proposed network requires fewer neurons, and fewer interconnections between neurons. The interconnections use nonlinear synapses which are composed of comparators, and do not require any weighted interconnections between neurons, as used in conventional neural networks. The proposed network has many applications, including as a component of self-organizing feature maps aid other systems where sorting is a frequent operation. [ABSTRACT FROM AUTHOR]

The softmax function is widely used in deep neural networks (DNNs), its hardware performance plays an important role in the training and inference of DNN accelerators. However, due to the complexity of the traditional softmax, the existing hardware architectures are resource-consuming or have low precision. In order to address the challenges, we study a base-2 softmax function in terms of its suitability for neural network training and efficient hardware implementation. Compared to the classical base- $e$ softmax function, the base-2 softmax function is a new softmax function that uses 2 as the exponential base instead of $e$. From the aspects of mathematical derivation and software simulation, we first demonstrate the feasibility and good accuracy of the base-2 softmax function in the application of neural network training. Then, we use the symmetric-mapping lookup table (SM-LUT) method to design a low-complexity architecture but with high precision to implement it. Under TSMC 28nm CMOS technology, an example design of our architecture has the area of $5676 ~\mu m^{2}$ and the power consumption of 13.12 mW for circuit synthesis at the frequency of 3 GHz. Compared with the latest works, our architecture achieves the best performance and efficiency. [ABSTRACT FROM AUTHOR]

Multi-bit-width convolutional neural network (CNN) maintains the balance between network accuracy and hardware efficiency, thus enlightening a promising method for accurate yet energy-efficient edge computing. In this work, we develop state-of-the-art multi-bit-width accelerator for NAS Optimized deep learning neural networks. To efficiently process the multi-bit-width network inferencing, multi-level optimizations have been proposed. Firstly, differential Neural Architecture Search (NAS) method is adopted for the high accuracy multi-bit-width network generation. Secondly, hybrid Booth based multi-bit-width multiply-add-accumulation (MAC) unit is developed for data processing. Thirdly, vector systolic array is proposed for effectively accelerating the matrix multiplications. With vector-style systolic dataflow, both the processing time and logic resources consumption can be reduced when compared with the classical systolic array. Finally, The proposed multi-bit-width CNN acceleration scheme has been practically deployed on FPGA platform of Xilinx ZCU102. Average performance on accelerating the full NAS optimized VGG16 network is 784.2 GOPS, and peek performance of the convolutional layer can reach as high as 871.26 GOPS for INT8, 1676.96 GOPS for INT4, and 2863.29 GOPS for INT2 respectively, which is among the best results in previous CNN accelerator benchmarks. [ABSTRACT FROM AUTHOR]

We present a novel deep learning model for a neural network that reduces both computation and data storage overhead. To do so, the proposed model proposes and combines a binary-weight neural network (BNN) training, a storage reuse technique, and an incremental training scheme. The storage requirements can be tuned to meet the desired classification accuracy, storing more parameters on an on-chip memory, and thereby reducing off-chip data storage accesses. Our experiments show 4–6 $\times $ reduction in weight storage footprint when training binary deep neural network models. On the FPGA platform, this results in a reduced amount of off-chip accesses, enabling our model to train a neural network in $14\times $ shorter latency, as compared to the conventional BNN training method. [ABSTRACT FROM AUTHOR]

A deep neural network (DNN) online learning processor is proposed with high throughput and low power consumption to achieve real-time object tracking in mobile devices. Four key features enable a low-power DNN online learning. First, a proposed processor is designed with a unified core architecture and it achieves $1.33\times $ higher throughput than the previous state-of-the-art DNN learning processor. Second, the new algorithms, binary feedback alignment (BFA), and dynamic fixed-point based run-length compression (RLC), are proposed and reduce power consumption through the reduction of external memory accesses (EMA). The BFA and dynamic fixed-point-based RLC reduce the EMA by 11.4% and 32.5%, respectively. Third, the new data feeding units, including an integral RLC (iRLC) decoder and a transpose RLC (tRLC) decoder, are co-designed to maximize throughput alongside the proposed algorithms. Finally, a dropout controller in this processor reduces redundant power consumption coming from the unified core and the data feeding architecture by the proposed dynamic clock-gating scheme. This enables the proposed processor to operate DNN online learning with 38.1% lower power consumption. Implemented with 65 nm CMOS technology, the 3.52 mm2 DNN online learning processor shows 126 mW power consumption and the processor achieves 30.4 frames-per-second throughput in the object tracking application. [ABSTRACT FROM AUTHOR]

As a result of the increasing demand for deep neural network (DNN)-based services, efforts to develop hardware accelerators for DNNs are growing rapidly. However, while highly efficient accelerators on convolutional DNNs (Conv-DNNs) have been developed, less progress has been made with regards to fully-connected DNNs. Based on analysis of bit-level SRAM errors, we propose memory adaptive training with in-situ canaries (MATIC), a methodology that enables aggressive voltage scaling of accelerator weight memories to improve the energy-efficiency of DNN accelerators. To enable accurate operation with voltage overscaling, MATIC combines characteristics of SRAM bit failures with the error resilience of neural networks in a memory-adaptive training (MAT) process. Furthermore, PVT-related voltage margins are eliminated using bit-cells from synaptic weights as in-situ canaries to track runtime environmental variation. Demonstrated on a low-power DNN accelerator fabricated in 65 nm CMOS, MATIC enables up to $3.3\times $ energy reduction versus the nominal voltage, or $18.6\times $ application error reduction. We also perform a simulation study that extends MAT to Conv-DNNs, and characterize the accuracy impact of bit failure statistics. Finally, we develop a weight refinement algorithm to improve the performance of MAT, and show that it improves absolute accuracy by 0.8–1.3% or reduces training time by 5– $10\times $. [ABSTRACT FROM AUTHOR]

This paper proposes a new three-layered biased memory polynomial for behavioral modeling and digital predistortion of highly nonlinear transmitters/power amplifiers (PAs) for 3G wireless applications. The proposed model considers the possibility that the nonlinearity order of the dynamic part of the PA characteristics is different from the nonlinearity order of the static part. For highly nonlinear PAs, the proposed model offers some benefits, such as a low dispersion of coefficients, numerical stability and a low number of coefficients. Moreover, with the measurement setup, better in-band performance is reported. To establish the performance of the linearized PA under realistic conditions, experiments have been carried out for a deep biased class-AB PA and a Doherty PA for various modeling and signal quality norms defined for 3G signals. [ABSTRACT FROM AUTHOR]

Coupled local minimizers (CLMs) turn out to be a potential global optimization strategy to explore a search space, avoid overfitting and produce good generalization. In this paper, convergence properties of CLMs based on an augmented Lagrangian function in the context of equality constrained minimization, are studied. We first consider the augmented Lagrangian by taking the objective of minimizing the average cost of an ensemble of local minimizers subject to pairwise synchronization constraints. Then we study an array of CLMs within the Lagrange programming network framework and analyze the local stability of CLMs using a linearization strategy. We further show that, under some mild conditions, global asymptotical stability of the unique equilibrium point of the network can be guaranteed. Afterwards, some sufficient conditions are presented to ensure the stability of synchronization between any two minimizers via a directed graph method. The results show that the CLMs usually can be synchronized if the penalty factors in the array of CLMs are chosen large enough. It is worth pointing out that CLMs possess the capability of global exploration in the search space and the advantage of faster running time on convex problems in comparison with most of the neural network approaches, which is also illustrated through two test functions and their numerical simulations. [ABSTRACT FROM AUTHOR]

The conventional digital hardware computational blocks with different structures are designed to compute the precise results of the assigned calculations. The main contribution of our proposed Bio-inspired Imprecise Computational blocks (BICs) is that they are designed to provide an applicable estimation of the result instead of its precise value at a lower cost. These novel structures are more efficient in terms of area, speed, and power consumption with respect to their precise rivals. Complete descriptions of sample BIC adder and multiplier structures as well as their error behaviors and synthesis results are introduced in this paper. It is then shown that these BIC structures can be exploited to efficiently implement a three-layer face recognition neural network and the hardware defuzzification block of a fuzzy processor. [ABSTRACT FROM AUTHOR]

Under the background of the wanton spread of the coronavirus disease (COVID-19), the pandemic is changing and hitting lives all over the world. Fortunately, Spatio-temporal processes bear essential importance in many applied scientific fields. And the disease pandemic can be viewed as Spatio-temporal dynamics processes. Generally, partial differential equations (PDEs) have been widely used to investigate interfacial dynamic processes. In this work, we use a physical-constraint neural network learning the Spatio-temporal mean-field dynamics to control the propagation of epidemics. Also, we use the AI-based algorithm physical-constraint learning (PCL) to solve the minimization problems of the mean-field game (MFG) and control (MFC) problems instead of the traditional computational method. In PCL, the PDEs are encoded into the loss function, where partial derivatives can be obtained through automatic differentiation (AD). We demonstrate how they can be applied in practice by considering the problem of controlling the propagation of epidemics. Numerical experiments on different input data are implemented to demonstrate the effectiveness and superiority of the proposed models compared to the state-of-the-art approach and illustrate how to separate the infected patients in a spatial domain effectively. [ABSTRACT FROM AUTHOR]

The increase in the number of edge devices has led to the emergence of edge computing where the computations are performed on the device. In recent years, deep neural networks (DNNs) have become the state-of-the-art method in a broad range of applications, from image recognition, to cognitive tasks to control. However, neural network models are typically large and computationally expensive and therefore not deployable on power and memory constrained edge devices. Sparsification techniques have been proposed to reduce the memory foot-print of neural network models. However, they typically lead to substantial hardware and memory overhead. In this article, we propose a hardware-aware pruning method using linear feedback shift register (LFSRs) to generate the locations of non-zero weights in real-time during inference. We call this LFSR-generated pseudo-random sequence based sparsity (LGPS) technique. We explore two different architectures for our hardware-friendly LGPS technique, based on (1) row/column indexing with LFSRs and (2) column-wise indexing with nested LFSRs, respectively. Using the proposed method, we present a total saving of energy and area up to 37.47% and 49.93% respectively and speed up of $1.53\times $ w.r.t the baseline pruning method, for the VGG-16 network on down-sampled ImageNet. [ABSTRACT FROM AUTHOR]

A neural network is said to be convergent (or completely stable) when each trajectory tends to an equilibrium point (a stationary state). A stronger property is that of absolute stability, which means that convergence holds for any choice of the neural network parameters, and any choice of the nonlinear functions, within specified and well characterized sets. In particular, the property of absolute stability requires that the neural network be convergent also when, for some parameter values, it possesses nonisolated equilibrium points (e.g., a manifold of equilibria). Such a property, which is really well suited for solving several classes of signal processing tasks in real time, cannot be in general established via the classical LaSalle approach, due to its inherent limitations to study convergence in situations where the neural network has nonisolated equilibrium points. In this paper, a new method to address absolute stability is developed, based on proving that the total length of the neural network trajectories is finite. A fundamental result on absolute stability is given, under the hypothesis that the neural network possesses a Lyapunov function, and the nonlinearities involved (neuron activations, inhibitions, etc.) are modeled by analytic functions. At the core of the proof of finiteness of trajectory length is the use of some basic inequalities for analytic functions due to Łojasiewicz. The result is applicable to a large class of neural networks, which includes the networks pro- posed by Vidyasagar, the Hopfield neural networks, and the standard cellular neural networks introduced by Chua and Yang. [ABSTRACT FROM AUTHOR]

Convolutional neural network (CNN) offers significant accuracy in image detection. To implement image detection using CNN in the Internet of Things (IoT) devices, a streaming hardware accelerator is proposed. The proposed accelerator optimizes the energy efficiency by avoiding unnecessary data movement. With unique filter decomposition technique, the accelerator can support arbitrary convolution window size. In addition, max-pooling function can be computed in parallel with convolution by using separate pooling unit, thus achieving throughput improvement. A prototype accelerator was implemented in TSMC 65-nm technology with a core size of 5 mm2. The accelerator can support major CNNs and achieve 152GOPS peak throughput and 434GOPS/W energy efficiency at 350 mW, making it a promising hardware accelerator for intelligent IoT devices. [ABSTRACT FROM PUBLISHER]

This contribution introduces a new digital predistorter to compensate serious distortions caused by memory high power amplifiers (HPAs) which exhibit output saturation characteristics. The proposed design is based on direct learning using a data-driven B-spline Wiener system modeling approach. The nonlinear HPA with memory is first identified based on the B-spline neural network model using the Gauss-Newton algorithm, which incorporates the efficient De Boor algorithm with both B-spline curve and first derivative recursions. The estimated Wiener HPA model is then used to design the Hammerstein predistorter. In particular, the inverse of the amplitude distortion of the HPA's static nonlinearity can be calculated effectively using the Newton-Raphson formula based on the inverse of De Boor algorithm. A major advantage of this approach is that both the Wiener HPA identification and the Hammerstein predistorter inverse can be achieved very efficiently and accurately. Simulation results obtained are presented to demonstrate the effectiveness of this novel digital predistorter design. [ABSTRACT FROM PUBLISHER]

Neuromorphic computing is an attractive avenue of research for processing and learning complex real-world data. With technology migration into nano and molecular scales several area and power efficient approaches to the design and implementation of artificial neural networks have been proposed. The discovery of the memristor has further enabled the realization of denser nanoscale logic and memory systems by facilitating the implementation of multilevel logic. Specifically, the innate reconfigurability of memristors can be exploited to realize synapses in artificial neural networks. This work focuses on the development of a variation-tolerant training methodology to efficiently reconfigure memristive synapses in a Trainable Threshold Gate Array (TTGA) system. The training process is inspired from the gradient descent machine learning algorithm commonly used to train artificial threshold neural networks, perceptrons. The design and CMOS/Nano implementation of the TTGA system from trainable perceptron based threshold gates is detailed and results are provided to showcase the training process and performance characteristics of the proposed system. Also shown are the results for training a 1T1M (1 Transistor and 1 Memristor) multilevel memristive memory and its performance characteristics. [ABSTRACT FROM PUBLISHER]

We first present a new (differential) entropy estimator for complex random variables by approximating the entropy estimate using a numerically computed maximum entropy bound. The associated maximum entropy distributions belong to the class of weighted linear combinations and elliptical distributions, and together, they provide a rich array of bivariate distributions for density matching. Next, we introduce a new complex independent component analysis (ICA) algorithm, complex ICA by entropy-bound minimization (complex ICA-EBM), using this new entropy estimator and a line search optimization procedure. We present simulation results to demonstrate the superior separation performance and computational efficiency of complex ICA-EBM in separation of complex sources that come from a wide range of bivariate distributions. [ABSTRACT FROM AUTHOR]

The growing interest in pulse-mode processing by neural networks is encouraging the development of hardware implementations of massively parallel networks of integrate-and-fire neurons distributed over multiple chips. Address-event representation (AER) has long been considered a convenient transmission protocol for spike based neuromorphic devices. One missing, long-needed feature of AER-based systems is the ability to acquire data from complex neuromorphic systems and to stimulate them using suitable data. We have implemented a general-purpose solution in the form of a peripheral component interconnect (PCI) board (the PCI-AER board) supported by software. We describe the main characteristics of the PCI-AER board, and of the related supporting software. To show the functionality of the PCI-AER infrastructure we demonstrate a reconfigurable multichip neuromorphic system for feature selectivity which models orientation tuning properties of cortical neurons. [ABSTRACT FROM AUTHOR]

A new result is developed for separating nonlinearly mixed signals in which the nonlinearity is characterized by a class of strictly monotonic continuously differentiable functions. The structure of the blind inverse system is explicitly derived within the framework of maximum likelihood estimation and the system culminates to a special architecture of the 3-layer perceptron neural network where the parameters in the first layer are inversely related to the output layer. The proposed approach exploits both the structural and signal constraints to search for the solution and assumes that the cumulants of the source signals are known a priori. A novel statistical algorithm based on the hybridization of the generalized gradient algorithm and metropolis algorithm has been derived for training the proposed perceptron which results in improved performance in terms of accuracy and convergence speed. Simulations and real-life experiment have also been conducted to verify the efficacy of the proposed scheme in separating the nonlinearly mixed signals. [ABSTRACT FROM AUTHOR]

A custom Human Activity Recognition system is presented based on the resource-constrained Hardware (HW) implementation of a new partially binarized Hybrid Neural Network. The system processes data in real-time from a single tri-axial accelerometer, and is able to classify between 5 different human activities with an accuracy of 97.5% when the Output Data Rate of the sensor is set to 25 Hz. The new Hybrid Neural Network (HNN) has binary weights (i.e. constrained to +1 or −1) but uses non-binarized activations for some layers. This, in conjunction with a custom pre-processing module, achieves much higher accuracy than Binarized Neural Network. During pre-processing, the measurements are made independent from the spatial orientation of the sensor by exploiting a reference frame transformation. A prototype has been realized in a Xilinx Artix 7 FPGA, and synthesis results have been obtained with TSMC CMOS 65 nm LP HVT and 90 nm standard cells. Best result shows a power consumption of $6.3~\mu \text{W}$ and an area occupation of 0.2 mm $^{\mathbf {2}}$ when real-time operations are set, enabling in this way, the possibility to integrate the entire HW accelerator in the auxiliary circuitry that normally equips inertial Micro Electro-Mechanical Systems (MEMS). [ABSTRACT FROM AUTHOR]

A correction to the article "LMI Approach for Global Periodicity of Neural Networks With Time-Varying Delays," published in a previous issue is presented.