Search

For a class of nontriangular nonlinear systems in presence of unknown disturbances, we propose a predictor-based neural dynamic surface control (PNDSC) strategy in this paper. This nontriangular system is transformed via the mean value theorem, and a predictor is then constructed. To avoid an algebraic loop problem, partial state vectors are employed as input signals of neural networks (NNs) for approximating unknown dynamics, and compensation items are designed to compensate for approximation errors from NNs. Different from the traditional NDSC, the PNDSC in this paper utilizes prediction errors to update learning parameters for improving NNs’ learning behaviors with overlarge adaptive gains. On the basis of improved NNs’ approximation behaviors, a predictor-based NNs disturbance observer (PNNDO) is constructed for compensation for external disturbances and approximation errors from NNs. Furthermore, with predictors, a normalization method of weights is developed to reduce the number of online learning parameters. On the basis of the aforementioned result, measurement noises are taken into account in our predictor-based neural control strategy. We employ predictor states, rather than measurement information paralyzed by noises, in design of our control strategy. This reduces high-frequency oscillations in control input. A Lyapunov-based stability analysis shows that all signals are ultimately bounded in the closed-loop system. Finally, the effectiveness of the proposed control strategy is verified by a numerical example and a permanent magnet brushless DC motor system. [ABSTRACT FROM AUTHOR]

SRAM based computing-in-memory (SRAM-CIM) techniques have been widely studied for neural networks (NNs) to solve the “Von Neumann bottleneck”. However, as the scale of the NN model increasingly expands, the weight cannot be fully stored on-chip owing to the big device size (limited capacity) of SRAM. In this case, the NN weight data have to be frequently loaded from external memories, such as DRAM and Flash memory, which results in high energy consumption and low efficiency. In this paper, we propose a hybrid-device computing-in-memory (HD-CIM) architecture based on SRAM and MRAM (magnetic random-access memory). In our HD-CIM, the NN weight data are stored in on-chip MRAM and are loaded into SRAM-CIM core, significantly reducing energy and latency. Besides, in order to improve the data transfer efficiency between MRAM and SRAM, a high-speed pipelined MRAM readout structure is proposed to reduce the BL charging time. Our results show that the NN weight data loading energy in our design is only 0.242 pJ/bit, which is 289 $\times $ less in comparison with that from off-chip DRAM. Moreover, the energy breakdown and efficiency are analyzed based on different NN models, such as VGG19, ResNet18 and MobileNetV1. Our design can improve $\mathbf {58\times \,\,to\,\,124\times }$ energy efficiency. [ABSTRACT FROM AUTHOR]

An automatic inductor design process is helpful to reduce the design cycle of radio frequency (RF) integrated circuit (IC). This paper proposed an efficient synthesis-analysis machine (SAM) for on-chip inductor synthesis and modeling, as well as an automatic dataset generation (ADG) topology for the generation of artificial neural networks (ANNs) training dataset. The SAM consists of a synthesis ANN, two analysis ANNs and a proposed self-inspection machine (SIM). For a given design request, the SAM synthesizes a layout first, followed by analyzing the performance of the layout automatically, and finally improves the layout confidence through self-inspection. Compared to the modeling functions of analysis ANN, the synthesis ANN behaves as an inverse model, of which the inputs are desired inductor performances while the outputs are geometrical parameters of the layout. However, multi-value problems might occur in obtaining geometries of inductors, suggesting with evidence from mathematic relationships between geometrical and electrical parameters of an inductor. A multi-valued training dataset will mislead the synthesis ANN, resulting in unqualified layouts. To deal with this issue, a solution space contraction technique (SSCM) is also proposed. Furthermore, the SIM suspends most of the failures by comparing the output of analysis ANN back to the input of synthesis ANN. An electromagnetic simulation tool is used for experiments, and the effectiveness of the proposed SAM is proven by 6,500 inductor samples. [ABSTRACT FROM AUTHOR]

When deploying deep neural networks (DNNs), edge devices training is practical to improve model adaptivity for various user-specific scenarios while avoiding privacy disclosure. However, the training computation is intolerable for edge devices. It inspires sparse DNN training (SDT) into the limelight, which reduces training computation by dynamic weight pruning. Generally, SDT has two strategies based on the pruning granularity: the structured or the unstructured. Unfortunately, both of them suffer from limited training efficiency due to the gap between pruning granularity and hardware implementation. The former is hardware-friendly but has a low pruning ratio, indicating limited computation reduction. The latter has a high pruning ratio, but the unbalanced workload decreases utilization and irregular sparsity distribution causes considerable sparsity processing overhead. This paper proposes a software-hardware co- design to bridge the gap for improving the efficiency of SDT. On the algorithm side, a sub-structured pruning method, achieved with hybrid shape-wise and line-wise pruning, generates a high sparsity ratio and keeps the hardware-friendly property. On the hardware side, a sub-structured weight processing unit (SWPU) effectively handles the hybrid sparsity with three techniques. First, SWPU dynamically reorders the computation sequence with hamming-distance-based clustering, balancing the irregular workload. Second, SWPU performs runtime scheduling by exploiting the feature of sub-structured sparse convolution through a detect-before-load controller, which skips redundant memory access and sparsity processing. Third, SWPU performs sparse convolution by compressing operands with spatial disconnect log-based routing and recovers their location with bi-directional switching, avoiding the power-consumed routing logic. Synthesized with 28nm CMOS technology, SWPU can enable 0.56V-to-1.0V supply voltage with a maximum frequency of 675 MHz. It achieves a 50.1% higher pruning ratio than structured pruning and $1.53\times $ higher energy efficiency than unstructured pruning. The peak energy efficiency of SWPU is 126.04TFLOPS/W, outperforming the state-of-the-art training processor by $1.67\times $. When training a ResNet-18 model, SWPU reduces $3.72\times $ energy and offers $4.69\times $ speedup than previous sparse training processors. [ABSTRACT FROM AUTHOR]

This paper proposes CMOS synapse and neuron for use in spiking neural networks to perform cognitive functions in a bio-inspired manner. The proposed synapse can trace the eligibility of the timing relationship between pre- and post-synaptic spikes, supporting a bio-plausible local learning rule called the spike timing-dependent plasticity (STDP) in an energy- and area-efficient manner. The proposed neuron can support neural functions such as synaptic current integration, threshold-based firing, neuronal leaking, membrane potential resetting, and adjustable refractory period with improved energy and area efficiency. The STDP curve shape of the synapse and the firing rate of the neuron can be adjusted as desired. Their variability due to process, voltage, and temperature (PVT) variations can also be minimized. The proposed CMOS neuron and synapse circuits were designed in a 28-nm CMOS process. The performance evaluation results indicate that the proposed synapse reduces energy consumption and area by up to 94% and 43% compared to conventional CMOS synapses. They also indicate that the proposed neuron achieves energy and area reductions of 37% and 23%, respectively, compared to conventional CMOS neurons. An associative neural network composed of the proposed neuron and synapse was designed to verify that they together work well for performing a cognitive function of associative learning and inferencing. [ABSTRACT FROM AUTHOR]

The divider is one of the most complex hardware units in Stochastic Computing (SC); even though several new designs have been presented to reduce the computation latency of the conventional divider, all of them still require a considerable number of clock cycles. Moreover, they incur in low performance due to the employed arithmetic computational scheme. In this paper, a Delta Sigma Modulator (DSM) based stochastic divider is proposed. As an entirely digital circuit, the proposed divider offers the best computation latency and accuracy over all existing stochastic dividers found in the technical literature (with a typical reduction between 66.8% and 96.9% in the number of clock cycles and a reduction from $10^{\mathrm {-3.4}}$ to $10^{\mathrm {-3.9}}$ in the average mean square error for a 10-bit resolution). An SC-based Neural Network (NN) is considered as an initial case study to evaluate the advantages of the proposed design in an emerging application; results show that the proposed divider enables an SC-based NN to achieve a higher classification accuracy and hardware efficiency than existing designs. To show the flexibility of the proposed divider design, its application to Sobol-based sequences is also presented; also in this case, its superiority over other designs is confirmed. These features make the proposed design very attractive for hardware-constrained platforms; moreover, such a novel design approach that incorporates ideas from analog/mixed signal circuit design into a digital circuit design, can motivate other researchers to design efficient SC designs using similar schemes. [ABSTRACT FROM AUTHOR]

The complexity and size of recent deep neural network (DNN) models have increased significantly in pursuit of high inference accuracy. Chiplet-based accelerator is considered a viable scaling approach to provide substantial computation capability and on-chip memory for efficient process of such DNN models. However, communication using metallic interconnects in prior chiplet-based accelerators poses a major challenge to system performance, energy efficiency, and scalability. Photonic interconnects can adequately support communication across chiplets due to features such as distance-independent latency, high bandwidth density, and high energy efficiency. Furthermore, the salient ease of broadcast property makes photonic interconnects suitable for DNN inference which often incurs prevalent broadcast communication. In this paper, we propose a scalable chiplet-based DNN accelerator with photonic interconnects named ASCEND. ASCEND introduces (1) a novel photonic network that supports seamless intra- and inter- chiplet broadcast communication, and flexible mapping of diverse convolution layers, and (2) a tailored dataflow that exploits the ease of broadcast property and maximizes parallelism by simultaneously processing computations with shared input data. Simulation results using multiple DNN models show that ASCEND achieves 71% and 67% reduction in execution time and energy consumption, respectively, as compared to other state-of-the-art chiplet-based DNN accelerators with metallic or photonic interconnects. [ABSTRACT FROM AUTHOR]

As model size and the number of layers increase, Deep Neural Networks (DNNs) demand enormous computational power and throughput to meet exceedingly high prediction accuracy’s of today’s machine learning (ML) applications. Spatial hardware accelerators have been proposed that optimize the dataflow and exploit sparsity to provide a significant decrease in power consumption. As spatial architectures are traditionally designed with metallic interconnects, significant power is expended for data movement for different dataflows. In this paper, we exploit extended wireless technology to design a power-efficient and high-throughput DNN accelerator, e-WiNN, that can be configured for all representative dataflows and arithmetic precisions. We leverage novel circuit design by utilizing Dadda-algorithm based Multiply-and-Accumulate (MAC) circuits for 4-bit, 8-bit and 16-bit inputs to reduce area, power and delay constraints in 14 nm predictive technology. Our novel wireless transmitter integrates on- off keying (OOK) modulator with power amplifier that results in significant energy savings. To reduce the area overhead, we cluster wireless transceivers into groups of four such that both weights and input features can be effectively multicast to reduce the data movement. The energy efficient transceiver circuit is implemented in state-of-the-art BSIM 32 nm FinFET technology model and our link budget considers required RF power for different frequencies and inter-PE distance at three different antenna directivities including isotropic. Our detailed RTL modeling and cycle-accurate simulation results show that e-WiNN achieves 36.3% latency reduction and 76.1% energy saving when compared to state-of-art wire interconnected accelerators; 70.3% area reduction and 41.6% energy saving at the cost of 11% latency increase when compared to prior wireless accelerators on various neural networks (AlexNet, VGG16, and ResNet-9/50). [ABSTRACT FROM AUTHOR]

Neural networks (NNs) with large scales of artificial neurons are increasingly used in recognition and classification tasks. In power-constrained scenarios, the tradeoff between performance and hardware consumptions must be carefully evaluated before silicon tape-out. In this paper, we proposed a systematic approach to design ultra-low power NN system. This work is motivated by the facts that NNs are resilient to approximation in many of the computations and NNs are outputting statistical tensors which are acceptable to less-than-perfect results. We resort to the front-back end approach with a twofold aim: (1) a fast and accurate design approach is proposed by estimating the computing quality of low-power approximate adder arrays, and it is adopted to evaluate the neural network system; (2) a quality configurable engine with different approximation degrees while processing NNs is implemented. The proposed work is demonstrated with a comprehensive keyword spotting (KWS) system as an ultra-low power NN engine. The experimental environment is setup with ten keywords from the google speech command dataset (GSCD) using an industrial 22-nm ultra-low-leakage (ULL) process. Comparing to the state-of-the-art KWS processors, the proposed approximate NN engine can demonstrate over 60% improvement in power efficiency and $1.1\times $ area efficiency while achieving similar recognition accuracy. [ABSTRACT FROM AUTHOR]

This paper deals with global synchronization problem of multiple discrete-time Markovian jump memristor-based neural networks (DTMJMNNs) with mixed mode-dependent delays via a novel event-triggered impulsive coupling control (ETICC). The parameters of the multiple DTMJMNNs and the mixed time delays (both discrete and distributed delays) switch randomly according to a Markov chain. In the ETICC strategy, the controller does not work all the time, but only works at impulse instants determined by specific events. In particular, the coupling matrix can be non-Laplacian. By using the Lyapunov stability theory, linear matrix inequalities (LMIs), and the Kronecker product, some sufficient conditions for global synchronization of multiple DTMJMNNs under the event-triggered strategy are derived. Two examples are presented to test the validity of the theoretical analysis results. [ABSTRACT FROM AUTHOR]

This paper proposes a battery management system, including a fast battery charger, battery aging diagnosis, and charge estimation and balancing. The charger adopts a single-inductor single-input dual-output architecture to achieve charge balancing among battery cells. Interleaved pulse charging is proposed to reduce the charging time and slow down the aging process of batteries as well. This method also significantly suppresses the variations of the temperature of battery cells and is beneficial to the implementation of charge balancing. An artificial neural network is proposed to detect the state of health (SOH) of battery cells and improve the accuracy of the state of charge (SOC) estimation. The prototype is implemented in TSMC 0.35- $\mu \text{m}$ process and TensorFlow tools are used. Measurement results show that the interleaved pulse charging reduces 30% variation of the battery temperature and saves 24% charging time when charging four battery cells concurrently. A mean absolute error of SOC estimation of 0.35% is achieved in this work. [ABSTRACT FROM AUTHOR]

In this paper, the observer based neural adaptive finite-time containment control strategy for non-strict feedback nonlinear multi-agent systems is studied. The finite-time command filter is used to overcome the explosion of complexity problem and the established fractional power based error compensation signal is applied to compensate the filtering error caused by the filter. The distributed finite-time command filtered backstepping control method combines with the neural adaptive control technology and state observer is given, which ensures the containment control errors reach to the desired neighborhood of the origin in finite-time in the presence of uncertain dynamics and unmeasurable states in the system. The given numerical simulations show the effectiveness of the proposed control strategy. [ABSTRACT FROM AUTHOR]

This paper investigates the secure consensus tracking problem for continuous-time nonlinear multiagent systems with sensor attacks. By designing a secure data selector, the unattacked output data is extracted from a group of output measurements under sparse sensor attacks. Then, by virtue of the obtained data and neural networks, a state observer is constructed to estimate the unavailable system states, where the convex combination theory is introduced to reduce the difficulty of solving observation gains. To utilize the limited communication resources reasonably, a novel saturated threshold event-triggered control strategy is proposed to reduce control updates, and then each update is encoded into a binary signal (0 or 1) to further reduce the occupation of communication bandwidth. The designed control scheme ensures that all closed-loop signals are semi-globally uniformly ultimately bounded, and its effectiveness is verified via a simulation of attitude control of unmanned aerial vehicles. [ABSTRACT FROM AUTHOR]

In this paper, a novel class of quaternion-valued memristive neural networks (QVMNNs) that considers both the spatial factor and Markov jump phenomenon is proposed, the finite/fixed-time anti-synchronization (F/FTAS) of which is investigated. It is worth mentioning that the considered master and slave systems are assumed to jump along two inconsistent Markov chains, which is a first attempt at the issue of the anti-synchronization of Markovian systems and may be more realistic than most existing Markovian systems’ models. Then, a suitable feedback controller is designed, such that the error system can be finite/fixed-time stable. Moreover, by integrating algebraic inequality technologies and Lyapunov theory, a new F/FTAS theorem can be obtained for the proposed systems. Finally, this paper provides two examples, so that the rationality, superiority, and practical value of the main results can be illustrated. [ABSTRACT FROM AUTHOR]

This paper provides a review of design approaches towards artificial intelligence (AI) System-on-Chip. AI algorithms have progressed over the past decades from perceptron-based neural network (NN) and neuro-fuzzy (NF) system to today’s deep neural network (DNN) and neuromorphic computing. Recent DNN hardware accelerators focus on energy-efficient integration of digital circuits to realize real-time DNN operation while neuromorphic processors deploy new memory technologies with analog computation for low power consumption. However, different design approaches can be applied to such processor implementation with their pros and cons. This paper reviews from the early processor designs for NN and NF in both mixed-mode and digital implementations to the recent DNN SoC designs that we have proposed for a decade. The former content deals with NN and NF processors used as a functional building block of a machine vision SoC, while the latter concentrates on integration of the whole DNN function. We also provide a discussion on the approaches, and provide perspective on future research directions. [ABSTRACT FROM AUTHOR]

The memristor-based neuromorphic computing system (NCS) with emerging storage and computing integration architecture has drawn extensive attention. Because of the unique nonvolatility and programmability, the memristor is an ideal nano-device to realize neural synapses in VLSI circuit implementation of neural networks. However, in the hardware implementation, the performance of the memristive neural network is always affected by quantization error, writing error, and conductance drift, which seriously hinders its applications in practice. In this paper, a novel weight optimization scheme combining quantization and Bayesian inference is proposed to alleviate this problem. Specifically, the weight deviation in the memristive neural network is transformed into the weight uncertainty in the Bayesian neural network, which can make the network insensitive to unexpected weight changes. A quantization regularization term is designed and utilized during the training process of the Bayesian neural network, reducing the quantization error and improving the robustness of the network. Furthermore, a partial training method is raised to extend the applicability of the proposed scheme in large-scale neural networks. Finally, the experiments on a Multilayer Perceptron and LeNet demonstrate that the proposed weight optimization scheme can significantly enhance the robustness of memristive neural networks. [ABSTRACT FROM AUTHOR]

This paper presents a high-performance solar energy harvesting system with improved adaptive maximum power point tracking (AMPPT) method utilizing neural network (NN) model as assistance. Under the guidance of the negative feedback control (NFC) model, a BJT based voltage control oscillator with three off-chip reconfigurable resistors is designed in this paper to improve the reusability of the solar energy harvesting system. Meanwhile, the AMPPT accuracy has much improvement by co-simulation of MATLAB/Simulink and Virtuoso with the help of NN model of Photovoltaic (PV) Cell. The complete system with output voltage of 4.2V to power the battery is designed and fabricated in $0.18~\mu \text{m}$ CMOS technology. According to the test results, the system can track the maximum power points (MPP) successfully with average voltage tracking errors of 0.23% (0.01-0.51%) of PV cell 1 and 0.29% (0.01-0.5%) of PV cell 2 when the light intensity changes from 3000lux to 10000lux. Without power hungry current sensor or voltage sensor and other complicated control circuits, the peak efficiency is about 89.39% @ 3000lux. [ABSTRACT FROM AUTHOR]

In this paper, the sensitivity of the neural network (NN) outputs to device parameter uncertainties (non-idealities) in inverter-based memristor (IM) crossbar neuromorphic circuits is mathematically modeled and verified using exhaustive circuit and system-level simulations. The NN sensitivity is obtained by modeling the sensitivity of the IM neuron output to the non-idealities of its circuit elements. The analysis reveals a higher sensitivity of the output voltage of the IM neuron to the non-idealities of the inverters compared to the conductance variation of the memristors. Among the inverter non-idealities, horizontal shift of the inverters voltage transfer characteristic (VTC) shows the highest impact on the output voltage of the neuron. To reduce the accuracy loss due to the variations, a training approach which includes a sensitivity term in the cost function of the training phase, is suggested. The achievable improvements through the said NN training approach are evaluated. In the evaluation, the California Housing, MNIST, and Fashion MNIST datasets are employed. The results show up to 50% reduction in the NN output variations in the presence of circuit elements’ non-idealities. [ABSTRACT FROM AUTHOR]

Due to the low complexity in arithmetic unit design, stochastic computing (SC) has attracted considerable interest to implement Artificial Neural Networks (ANNs) for resources-limited applications, because ANNs must usually perform a large number of arithmetic operations. To attain a high computation accuracy in an SC-based ANN, extended stochastic logic is utilized together with standard SC units and thus, a stochastic divider is required to perform the conversion between these logic representations. However, the conventional divider incurs in a large computation latency, so limits an SC implementation for ANNs used in applications needing high performance. Therefore, there is a need to design fast stochastic dividers for SC-based ANNs. Recent works (e.g., a binary searching and triple modular redundancy (BS-TMR) based stochastic divider) are targeting a reduction in computation latency, while keeping the same accuracy compared with the traditional design. However, this divider still requires $N$ iterations to deal with $2^{N}$ -bit stochastic sequences, and thus the latency increases in proportion to the sequence length. In this paper, a decimal searching and TMR (DS-TMR) based stochastic divider is initially proposed to further reduce the computation latency; it only requires two iterations to calculate the quotient, so regardless of the sequence length. Moreover, a trade-off design between accuracy and hardware is also presented. An SC-based Multi-Layer Perceptron (MLP) is then considered to show the effectiveness of the proposed dividers over current designs. Results show that when utilizing the proposed dividers, the MLP achieves the lowest computation latency while keeping the same classification accuracy; although incurring in an area increase, the overhead due to the proposed dividers is low over the entire MLP. When using as combined metric for both hardware design and computation complexity the product of the implementation area, latency, power and number of clock cycles, the proposed designs are also shown to be superior to the SC-based MLPs (at the same level of accuracy) employing other dividers found in the technical literature as well as the commonly used 32-bit floating point implementation. [ABSTRACT FROM AUTHOR]

Aiming at a class of switched uncertain nonlinear strict-feedback systems under the action of average dwell time switching signal, this paper proposes a novel adaptive neural network output feedback tracking control based on the consideration of the full state constraints. The controller is proposed based on neural networks. One of the key characteristics of the system discussed is that the state variables cannot be measured and the system states need to be kept within the constraint ranges. For the sake of estimating the unmeasured states, the observer is constructed. In order to ensure all states which are within the time-varying boundary, the tangent barrier Lyapunov function (BLF-Tan) is selected in the design process. The boundedness of the closed-loop signals with average dwell time is guaranteed by the designed controllers and all the states limit in their constrained sets. It has been proved that the output tracking error converge to a small neighborhood of zero. In addition, the significance of the presented control strategy is verified and tested by a simulation example. [ABSTRACT FROM AUTHOR]

In this paper, a method for offline training of inverter-based memristive neural networks (IM-NNs), called ERIM, is presented. In this method, the output voltage of the inverter is modeled very accurately by considering the loading effect of the memristive crossbar. To properly choose the size of each inverter, its output load and the required slope of its voltage transfer characteristic (VTC) for an acceptable level of resiliency to the circuit element non-idealities are taken into account. The efficacy of ERIM is investigated by comparing its accuracy to those of two recently proposed offline training methods for IM-NNs (RIM and PHAX). The study is performed using IRIS, BCW, MNIST, and Fashion MNIST datasets. Simulation results show that 72% (56%) reduction in average energy consumption of the trained networks is achieved compared to RIM (PHAX) thanks to proper sizing of the inverters. In addition, due to the higher accuracy of the NN mathematical model, ERIM results in significant improvements in the match between the results of high-level modeling and HSPICE simulations while exhibiting lower sensitivity to circuit element variations. [ABSTRACT FROM AUTHOR]

Nowadays, research, modeling, simulation and realization of brain-like systems to reproduce brain behaviors have become urgent requirements. In this paper, neural bursting and synchronization are imitated by modeling two neural network models based on the Hopfield neural network (HNN). The first neural network model consists of four neurons, which correspond to realizing neural bursting firings. Theoretical analysis and numerical simulation show that the simple neural network can generate abundant bursting dynamics including multiple periodic bursting firings with different spikes per burst, multiple coexisting bursting firings, as well as multiple chaotic bursting firings with different amplitudes. The second neural network model simulates neural synchronization using a coupling neural network composed of two above small neural networks. The synchronization dynamics of the coupling neural network is theoretically proved based on the Lyapunov stability theory. Extensive simulation results show that the coupling neural network can produce different types of synchronous behaviors dependent on synaptic coupling strength, such as anti-phase bursting synchronization, anti-phase spiking synchronization, and complete bursting synchronization. Finally, two neural network circuits are designed and implemented to show the effectiveness and potential of the constructed neural networks. [ABSTRACT FROM AUTHOR]

In emerging Spiking Neural Network (SNN) based neuromorphic hardware design, energy efficiency and on-line learning are attractive advantages mainly contributed by bio-inspired local learning with nonlinear dynamics and at the cost of associated hardware complexity. This paper presents a novel SNN design employing fast COordinate Rotation DIgital Computer (CORDIC) algorithm to achieve fast spike timing–dependent plasticity (STDP) learning with high hardware efficiency. In this study, a system design and evaluation method of CORDIC-based SNN is proposed for finding optimal CORDIC type and precision, from theoretical CORDIC-level error to application-level learning performance. From the proposed design and evaluation method, a reconfigurable SNN design based on fast-convergence CORDIC is designed to achieve high classification accuracy on MNIST, fast on-line learning and good energy efficiency. By utilizing SNN’s fault tolerance and time-division-multiplexing (TDM) strategy, the reconfigurable SNN design employs 8-bit fast-convergence CORDIC and TDM-based hardware accelerator for high efficiency. FPGA implementation results confirm that the proposed fast-convergence CORDIC SNN design outperforms the state-of-the-art CORDIC method by 38.5%−45.3% in terms of learning speed and energy efficiency, with the STDP learning of 30.2 ns/SOP, energy efficiency of 176.6 pJ/SOP, processing speed of 6.1 ms/image, and on-line learning convergence of 21.4 s (time to reach the final accuracy, on average), on MNIST benchmark. [ABSTRACT FROM AUTHOR]

In this paper, an adaptive admittance control scheme is developed for robots to interact with time-varying environments. Admittance control is adopted to achieve a compliant physical robot–environment interaction, and the uncertain environment with time-varying dynamics is defined as a linear system. A critic learning method is used to obtain the desired admittance parameters based on the cost function composed of interaction force and trajectory tracking without the knowledge of the environmental dynamics. To deal with dynamic uncertainties in the control system, a neural-network (NN)-based adaptive controller with a dynamic learning framework is developed to guarantee the trajectory tracking performance. Experiments are conducted and the results have verified the effectiveness of the proposed method. [ABSTRACT FROM AUTHOR]

Dropout and DropConnect are two techniques to facilitate the regularization of neural network models, having achieved the state-of-the-art results in several benchmarks. In this paper, to improve the generalization capability of spiking neural networks (SNNs), the two drop techniques are first applied to the state-of-the-art SpikeProp learning algorithm resulting in two improved learning algorithms called SPDO (SpikeProp with Dropout) and SPDC (SpikeProp with DropConnect). In view that a higher membrane potential of a biological neuron implies a higher probability of neural activation, three adaptive drop algorithms, SpikeProp with Adaptive Dropout (SPADO), SpikeProp with Adaptive DropConnect (SPADC), and SpikeProp with Group Adaptive Drop (SPGAD), are proposed by adaptively adjusting the keep probability for training SNNs. A convergence theorem for SPDC is proven under the assumptions of the bounded norm of connection weights and a finite number of equilibria. In addition, the five proposed algorithms are carried out in a collaborative neurodynamic optimization framework to improve the learning performance of SNNs. The experimental results on the four benchmark data sets demonstrate that the three adaptive algorithms converge faster than SpikeProp, SPDO, and SPDC, and the generalization errors of the five proposed algorithms are significantly smaller than that of SpikeProp. Furthermore, the experimental results also show that the five algorithms based on collaborative neurodynamic optimization can be improved in terms of several measures. [ABSTRACT FROM AUTHOR]

This paper proposes a novel artificial neural network (ANN) based control method for a dc/dc buck converter. The ANN is trained to implement optimal control based on approximate dynamic programming (ADP). Special characteristics of the proposed ANN control include: 1) The inputs to the ANN contain error signals and integrals of the error signals, enabling the ANN to have PI control ability; 2) The ANN receives voltage feedback signals from the dc/dc converter, making the combined system equivalent to a recurrent neural network; 3) The ANN is trained to minimize a cost function over a long time horizon, making the ANN have a stronger predictive control ability than a conventional predictive controller; 4) The ANN is trained offline, preventing the instability of the network caused by weight adjustments of an on-line training algorithm. The ANN performance is evaluated through simulation and hardware experiments and compared with conventional control methods, which shows that the ANN controller has a strong ability to track rapidly changing reference commands, maintain stable output voltage for a variable load, and manage maximum duty-ratio and current constraints properly. [ABSTRACT FROM AUTHOR]

The issue of finite-time stabilization (FTS) for the memristor-based inertial neural networks (MINNs) with mixed time-varying delays (MTVDs) is researched by virtue of a new analytical method in this brief. First, an appropriate reliable control strategy is proposed for MINNs, which takes the influence of actuator failures into account. Second, by combining Lyapunov functional theory with new analysis techniques, novel theoretical results to guarantee the FTS for the concerned MINNs are acquired, and the desired reliable controller gains are obtained simultaneously. In additions, compared with the previous research works, the FTS results obtained in this paper are established directly from the MINNs themselves without using variable transformation method. In the end, two simulations are exploited to show the correctness and practicability of the acquired theoretical results. [ABSTRACT FROM AUTHOR]

In this paper, we propose a low-power memory-based computing architecture, called selective computing architecture (SCA). It consists of multipliers and an LUT (Look-Up Table)-based component, that is multi-context ternary content-addressable memory (MC-TCAM). Either of them is selected by input-data conditions in neural-networks (NNs). Compared with quantized NNs, a higher accurate multiplication can be performed with low-power consumption in the proposed architecture. If input data stored in the MC-TCAM appears, the corresponding multiplication results for multiple weights are obtained. The MC-TCAM stores only shorter length of input data, resulting in achieving a low-power computing. The performance of the SCA is determined by three physical parameters concerning the configuration of MC-TCAM. The power dissipation of the target NN can be minimized by exploring these parameters in the design space. The hardware based on the proposed architecture is evaluated using TSMC 65 nm CMOS technology and MTJ model. In the case of speech command recognition, the power consumption at the multiplication of the first convolutional layer in a convolutional NN is reduced by 67% compared to the solution relying only on multipliers. [ABSTRACT FROM AUTHOR]

In this paper, we are concerned with the finite-time synchronization of a class of inertial neural networks with time delays. Without applying some finite-time stability theorems, which are widely applied to studying the finite-time synchronization for neural networks, by constructing two Lyapunov functions and using integral inequality method, two sufficient conditions on the finite-time synchronization for a class of inertial neural networks with time delays are derived. Considering that the method and research results of the finite-time synchronization are different from some existing works, this paper extends the works on the finite-time synchronization of neural networks. [ABSTRACT FROM AUTHOR]

In this paper, we propose O4-DNN, a high-performance FPGA-based architecture for convolutional neural network (CNN) accelerators relying on operation packing and out-of-order (OoO) execution for DSP blocks augmented with LUT-based glue logic. The high-level architecture is comprised of a systolic array of processing elements (PEs), supporting output stationary dataflow. In this architecture, the computational unit of each PE is realized by using a DSP block as well as a small number of LUTs. Given the limited number of DSP blocks in FPGAs, the combination (DSP block and some LUTs) provides more computational power obtainable through each DSP block. The proposed computational unit performs eight convolutional operations on five input operands where one of them is an 8-bit weight and the others are four 8-bit input feature (IF) maps. In addition, to improve the energy efficiency of the proposed computational unit, we present an approximate form of the unit suitable for neural network applications. To reduce the memory bandwidth as well as increase the utilization of the computational units, a data reusing technique based on the weight sharing is also presented. To improve the performance of the proposed computational unit further, an addressing approach for computing the partial sums out-of-order is proposed. The efficacy of the architecture is assessed using two FPGA devices executing four state-of-the-art neural networks. Experimental results show that this architecture leads to, on average (up to), $2.5\times $ ($3.44\times$) higher throughput compared to a baseline structure. In addition, on average (maximum of), 12% (40%) energy efficiency improvement is achievable by employing the O4-DNN compared to the baseline structure. [ABSTRACT FROM AUTHOR]

This paper aims to give a design strategy on the prescribed performance $ {H_{\infty }}$ tracking control problem for a class of strict-feedback stochastic nonlinear systems based on the backstepping technique. Generally, by using the backstepping design method, the stochastic nonlinear systems can only be made to be bounded in probability and it is difficult to achieve the $ {H_{\infty }}$ performance criterion due to the positive constant term appeared in the stability analysis. Thus, a novel concept with regard to the bounded- $ {H_{\infty }}$ performance is proposed in this paper to overcome the design difficulty. By using the new concept and the adaptive neural network technique as well as Gronwall inequality, an adaptive neural network prescribed performance bounded- $ {H_{\infty }}$ tracking controller is designed. Therein, neural networks are used to approximate the unknown packaged nonlinear functions. The assumption that the approximation errors of neural networks are square-integrable in some literature is eliminated. The designed controller guarantees that all the signals in the closed-loop stochastic nonlinear systems are bounded in probability, the tracking error is constrained into an adjustable neighborhood of the origin with the prescribed performance bounds, and the controlled system has a given $ {H_{\infty }}$ disturbance attenuation performance for external disturbances. Finally, the simulation results are provided to illustrate the effectiveness and feasibility of the proposed approach. [ABSTRACT FROM AUTHOR]

This paper presents the multistability analysis of almost periodic state solutions for memristive Cohen–Grossberg neural networks (MCGNNs) with both distributed delay and discrete delay. The activation function of the considered MCGNNs is generalized to be nonmonotonic and nonpiecewise linear. It is shown that the MCGNNs with $n$ -neuron have $(K+1)^{n}$ locally exponentially stable almost periodic solutions, where nature number $K$ depends on the geometrical structure of the considered activation function. Compared with the previous related works, the number of almost periodic state solutions of the MCGNNs is extensively increased. The obtained conclusions in this paper are also capable of studying the multistability of equilibrium points or periodic solutions of the MCGNNs. Moreover, the enlarged attraction basins of attractors are estimated based on original partition. Some comparisons and convincing numerical examples are provided to substantiate the superiority and efficiency of obtained results. [ABSTRACT FROM AUTHOR]

This paper is concerned with the stubborn state estimation of delayed neural networks that subject to a general class of disturbances in measurements, including outliers and impulsive disturbances as its special cases. This class of disturbances may be unbounded, irregular, and assorted; therefore, they can hardly be suppressed by existing identification-based estimation approaches. In this paper, a stubborn state estimator is constructed by intentionally devising a saturation scheme on the injection of output estimation error. The embedded saturation can effectively resist the influences from these measurement disturbances by saturating them. Moreover, the saturation threshold in the designed scheme is not constant but governed by a dynamic equation with parameters to be designed. Benefiting from this adaptiveness, the estimator obtains more freedom in dealing with various disturbances. By combining a novel Lyapunov functional, the generalized sector condition and two latest integral inequalities, a delay-dependent criterion is derived in a less conservative way to check whether the estimation error system with this dynamic saturation is globally stable. A sufficient condition with two tuning scalars is further provided to codesign the gain of the state estimator and the evolution law of the saturation threshold. Finally, two numerical examples are used to illustrate the stubbornness of this state estimator in the presence of measurement outliers or impulsive disturbances. [ABSTRACT FROM AUTHOR]

This paper proposes a novel end-to-end learning model, called skip-connected covariance (SCCov) network, for remote sensing scene classification (RSSC). The innovative contribution of this paper is to embed two novel modules into the traditional convolutional neural network (CNN) model, i.e., skip connections and covariance pooling. The advantages of newly developed SCCov are twofold. First, by means of the skip connections, the multi-resolution feature maps produced by the CNN are combined together, which provides important benefits to address the presence of large-scale variance in RSSC data sets. Second, by using covariance pooling, we can fully exploit the second-order information contained in such multi-resolution feature maps. This allows the CNN to achieve more representative feature learning when dealing with RSSC problems. Experimental results, conducted using three large-scale benchmark data sets, demonstrate that our newly proposed SCCov network exhibits very competitive or superior classification performance when compared with the current state-of-the-art RSSC techniques, using a much lower amount of parameters. Specifically, our SCCov only needs 10% of the parameters used by its counterparts. [ABSTRACT FROM AUTHOR]

Recent state-of-the-art deep convolutional neural networks (CNNs) have shown remarkable success in current intelligent systems for various tasks, such as image/speech recognition and classification. A number of recent efforts have attempted to design custom inference engines based on processing-in-memory (PIM) architecture, where the memory array is used for weighted sum computation, thereby avoiding the frequent data transfer between buffers and computation units. Prior PIM designs typically unroll each 3D kernel of the convolutional layers into a vertical column of a large weight matrix, where the input data needs to be accessed multiple times. In this paper, in order to maximize both weight and input data reuse for PIM architecture, we propose a novel weight mapping method and the corresponding data flow which divides the kernels and assign the input data into different processing-elements (PEs) according to their spatial locations. As a case study, resistive random access memory (RRAM) based 8-bit PIM design at 32 nm is benchmarked. The proposed mapping method and data flow yields $\sim 2.03\times $ speed up and $\sim 1.4\times $ improvement in throughput and energy efficiency for ResNet-34, compared with the prior design based on the conventional mapping method. To further optimize the hardware performance and throughput, we propose an optimal pipeline architecture, with ~50% area overhead, it achieves overall $913\times $ and $1.96\times $ improvement in throughput and energy efficiency, which are 132476 FPS and 20.1 TOPS/W, respectively. [ABSTRACT FROM AUTHOR]

Hardware accelerators for convolution neural networks (CNNs) enable real-time applications of artificial intelligence technology. However, most of the existing designs suffer from low hardware utilization or high area cost due to complex data flow. This paper proposes a hardware efficient vectorwise CNN accelerator that adopts a $3\times 3$ filter optimized systolic array using 1-D broadcast data flow to generate partial sum. This enables easy reconfiguration for different kinds of kernels with interleaved input or elementwise input data flow. This simple and regular data flow results in low area cost while attains high hardware utilization. The presented design achieves 99%, 97%, 93.7%, and 94% hardware utilization for VGG-16, ResNet-34, GoogLeNet, and Mobilenet, respectively. Hardware implementation with TSMC 40nm technology takes 266.9K NAND gate count and 191KB SRAM to support 168GOPS throughput while consumes only 154.98mW when running at 500MHz operating frequency, which has superior area and power efficiency than other designs. [ABSTRACT FROM AUTHOR]

Transcendental functions are commonly used in many fields such as nonlinear functions of artificial neural networks (ANNs). Due to nonlinearity of these functions, hardware implementation of the functions faces many challenges. Some methods cannot control the approximation accuracy beforehand, or they cannot approximate target functions accurately and efficiently. Other methods can only approximate a particular function without generality. A universal piecewise linear (PWL) approximation method is proposed to solve the problems existing in the above methods. In this paper, a general piecewise linear (PWL) approximation method with controllable maximum absolute error for transcend-ental functions is proposed. The method has the self-adaptive capability to choose the smallest number of segments under the constraint of a controllable maximum absolute error. Therefore, it requires fewer segments and incurs less hardware overhead. Moreover, it can approximate any transcendental functions and does not rely on any properties of the target function. Comparing our work with other methods used in three transcendental functions, including the sigmoid, hyperbolic tangent, and logarithmic function, our method improves the approximation accuracy by 4.5 times, 2.1 times, and 1.3 times, respectively; in addition, our method is more efficient and it reduces the circuit area by 49.9%, 22.4%, and 28.5%, respectively. [ABSTRACT FROM AUTHOR]

This paper studies an adaptive neural network (NN) tracking control method for a class of uncertain nonlinear strict-feedback systems with time-varying full-state constraints. As we all know, the states are inevitably constrained in the actual systems because of the safety and performance factors. The main contributions of this paper are that: 1) in order to ensure that the states do not violate the asymmetric time-varying constraint regions, an adaptive NN controller is constructed by introducing the asymmetric time-varying barrier Lyapunov function (TVBLF) and 2) the amount of the learning parameters is reduced by introducing a TVBLF at each step of the backstepping. Based on the Lyapunov stability analysis, it can be proven that all the signals in the closed-loop system are the semiglobal ultimately uniformly bounded and the time-varying full-state constraints are never violated. Finally, a numerical simulation is given, and the effectiveness of this adaptive control method can be verified. [ABSTRACT FROM AUTHOR]

Always-on speech interfaces are prevailing in human-machine interaction, especially on wearable devices, Internet of Things, etc., which benefits from the recent breakthroughs in deep learning. For battery-powered devices, ultra-low power and real-time processing are critical. However, the massive memory access and computation of deep neural networks (DNNs) lead to long latency and huge energy consumption, which hinder their further integration in battery-powered devices. Extremely low-bit quantization shows the potential to enhance energy efficiency and speed by orders of magnitudes, but it suffers from the degradation of recognition accuracy. In this paper, we propose a binarized convolutional neural network (BCNN) based speech recognition processor, integrated with on-chip self-learning mechanism to compensate the accuracy loss caused by low-precision. We optimize the BCNN architecture by eliminating the computation redundancy, compressing the weights and tailoring approximate circuits. Fabricated in 28 nm CMOS, this processor supports real time speech recognition with power consumption of $141~\mu \text{W}$ and energy efficiency of 2.46 pJ/Neuron. Compared with state-of-the-art speech recognition implementations, this processor achieves $2.5\times $ reduction on energy consumption per neuron, and $8.0\times $ energy reduction per speech frame. [ABSTRACT FROM AUTHOR]

In this paper, the adaptive neural network (NN) tracking control problem is addressed for robot manipulators subject to dead-zone input. The control objective is to design an adaptive NN controller to guarantee the stability of the systems and obtain good performance. Different from the existing results, which used NN to approximate the nonlinearities directly, NNs are employed to identify the originally designed virtual control signals with unknown nonlinear items in this paper. Moreover, a sequence of virtual control signals and real controller are designed. The adaptive backstepping control method and Lyapunov stability theory are used to prove the proposed controller can ensure all the signals in the systems are semiglobally uniformly ultimately bounded, and the output of the systems can track the reference signal closely. Finally, the proposed adaptive control strategy is applied to the Puma 560 robot manipulator to demonstrate its effectiveness. [ABSTRACT FROM AUTHOR]

Computing-in-memory (CIM) is a promising approach to reduce the latency and improve the energy efficiency of deep neural network (DNN) artificial intelligence (AI) edge processors. However, SRAM-based CIM (SRAM-CIM) faces practical challenges in terms of area overhead, performance, energy efficiency, and yield against variations in data patterns and transistor performance. This paper employed a circuit-system co-design methodology to develop a SRAM-CIM unit-macro for a binary-based fully connected neural network (FCNN) layer of the DNN AI edge processors. The proposed SRAM-CIM unit-macro supports two binarized neural network models: an XNOR neural network (XNORNN) and a modified binary neural network (MBNN). To achieve compact area, fast access time, robust operations, and high energy-efficiency, our proposed SRAM-CIM uses a split-wordline compact-rule 6T SRAM and circuit techniques, including a dynamic input-aware reference generation (DIARG) scheme, an algorithm-dependent asymmetric control (ADAC) scheme, a write disturb-free (WDF) scheme, and a common-mode-insensitive small offset voltage-mode sensing amplifier (CMI-VSA). A fabricated 65-nm 4-Kb SRAM-CIM unit-macro achieved 2.4- and 2.3-ns product-sum access times for a FCNN layer using XNORNN and MBNN, respectively. The measured maximum energy efficiency reached 30.49 TOPS/W for XNORNN and 55.8 TOPS/W for the MBNN modes. [ABSTRACT FROM AUTHOR]

In this paper, we believe that the mixed selectivity of neuron in the top layer encodes distributed information produced from other neurons to offer a significant computational advantage over recognition accuracy. Thus, this paper proposes a hierarchical network framework that the learning behaviors of features combined from hundreds of midlayers. First, a subnetwork neuron, which itself could be constructed by other nodes, is functional as a subspace features extractor. The top layer of a hierarchical network needs subspace features produced by the subnetwork neurons to get rid of factors that are not relevant, but at the same time, to recast the subspace features into a mapping space so that the hierarchical network can be processed to generate more reliable cognition. Second, this paper shows that with noniterative learning strategy, the proposed method has a wider and shallower structure, providing a significant role in generalization performance improvements. Hence, compared with other state-of-the-art methods, multiple channel features with the proposed method could provide a comparable or even better performance, which dramatically boosts the learning speed. Our experimental results show that our platform can provide a much better generalization performance than 55 other state-of-the-art methods. [ABSTRACT FROM AUTHOR]

Deep neural networks are biologically inspired class of algorithms that have recently demonstrated the state-of-the-art accuracy in large-scale classification and recognition tasks. Hardware acceleration of deep networks is of paramount importance to ensure their ubiquitous presence in future computing platforms. Indeed, a major landmark that enables efficient hardware accelerators for deep networks is the recent advances from the machine learning community that have demonstrated the viability of aggressively scaled deep binary networks. In this paper, we demonstrate how deep binary networks can be accelerated in modified von Neumann machines by enabling binary convolutions within the static random access memory (SRAM) arrays. In general, binary convolutions consist of bit-wise exclusive-NOR (XNOR) operations followed by a population count (popcount). We present two proposals: one based on charge sharing approach to perform vector XNOR and approximate popcount and another based on bit-wise XNOR followed by a digital bit-tree adder for accurate popcount. We highlight the various tradeoffs in terms of circuit complexity, speed-up, and classification accuracy for both the approaches. Few key techniques presented as a part of the manuscript are the use of low-precision, low-overhead analog-to-digital converter (ADC), to achieve a fairly accurate popcount for the charge-sharing scheme and proposal for sectioning of the SRAM array by adding switches onto the read-bitlines, thereby achieving improved parallelism. Our results on benchmark image classification datasets for CIFAR-10 and SVHN on a binarized neural network architecture show energy improvements of up to $6.1\times $ and $2.3\times $ for the two proposals, compared to conventional SRAM banks. In terms of latency, improvements of up to $15.8\times $ and $8.1\times $ were achieved for the two respective proposals. [ABSTRACT FROM AUTHOR]

Current advances in emerging memory technologies enable novel and unconventional computing architectures for high-performance and low-power electronic systems, capable of carrying out massively parallel operations at the edge. One emerging technology, ReRAM, also known to belong in the family of memristors (memory resistors), is gathering attention due to its attractive features for logic and in-memory computing and benefits which follow from its technological attributes, such as nanoscale dimensions, low-power operation, and multi-state programming. At the same time, the design with CMOS is quickly reaching its physical and functional limitations, and further research toward novel logic families, such as threshold logic gates (TLGs), is scoped. In this paper, we introduce a physical implementation of a memristor-based current-mode TLG (MCMTLG) circuit and validate its design and operation through multiple experimental setups. We demonstrate two-input, three-input, and four-input MCMTLG configurations and showcase their reconfiguration capability. This is achieved by varying memristive weights arbitrarily for shaping the classification decision boundary, thus showing a promise as an alternative hardware-friendly implementation of artificial neural networks. Through the employment of real memristor devices as the equivalent of synaptic weights in TLGs, we are realizing components that can be used toward an in silico classifier. [ABSTRACT FROM AUTHOR]

This paper aims to investigate the fuzzy constrained matrix game (MG) problems using the concepts of recurrent neural networks (RNNs). To the best of our knowledge, this paper is the first in attempting to find a solution for fuzzy game problems using RNN models. For this purpose, a fuzzy game problem is reformulated into a weighting problem. Then, the Karush–Kuhn–Tucker (KKT) optimality conditions are provided for the weighting problem. The KKT conditions are used to propose the RNN model. Moreover, the Lyapunov stability and the global convergence of the RNN model are also confirmed. Finally, three illustrative examples are presented to demonstrate the effectiveness of this approach. The obtained results are compared with the results obtained by the previous approaches for solving fuzzy constrained MG. [ABSTRACT FROM AUTHOR]

Fault tolerance is a remarkable feature of biological systems and their self-repair capability influence modern electronic systems. In this paper, we propose a novel plastic neural network model, which establishes homeostasis in a spiking neural network. Combined with this plasticity and the inspiration from inhibitory interneurons, we develop a fault-resilient robotic controller implemented on an FPGA establishing obstacle avoidance task. We demonstrate the proposed methodology on a spiking neural network implemented on Xilinx Artix-7 FPGA. The system is able to maintain stable firing (tolerance ±10%) with a loss of up to 75% of the original synaptic inputs to a neuron. Our repair mechanism has minimal hardware overhead with a tuning circuit (repair unit) which consumes only three slices/neuron for implementing a threshold voltage-based homeostatic fault-tolerant unit. The overall architecture has a minimal impact on power consumption and, therefore, supports scalable implementations. This paper opens a novel way of implementing the behavior of natural fault tolerant system in hardware establishing homeostatic self-repair behavior. [ABSTRACT FROM PUBLISHER]

Deep learning is now being used to translate between languages, predict how proteins fold, analyze medical scans, and play games as complex as Go, to name just a few applications of a technique that is now becoming pervasive. Success in those and other realms has brought this machine-learning technique from obscurity in the early 2000s to dominance today. • Although deep learning's rise to fame is relatively recent, its origins are not. In 1958, back when mainframe computers filled rooms and ran on vacuum tubes, knowledge of the interconnections between neurons in the brain inspired Frank Rosenblatt at Cornell to design the first artificial neural network, which he presciently described as a “pattern-recognizing device.” But Rosenblatt's ambitions outpaced the capabilities of his era—and he knew it. Even his inaugural paper was forced to acknowledge the voracious appetite of neural networks for computational power, bemoaning that “as the number of connections in the network increases…the burden on a conventional digital computer soon becomes excessive.” • Fortunately for such artificial neural networks—later rechristened “deep learning” when they included extra layers of neurons—decades of Moore's Law and other improvements in computer hardware yielded a roughly 10-million-fold increase in the number of computations that a computer could do in a second. So when researchers returned to deep learning in the late 2000s, they wielded tools equal to the challenge. [ABSTRACT FROM AUTHOR]

This paper solves the finite-time switching control issue for the nonstrict-feedback nonlinear switched systems. The controlled plants contain immeasurable states, arbitrarily switchings, and the unknown functions which are constructed with the whole states. Neural network is used to simulate the uncertain systems and a filter-based state observer is designed to estimate the immeasurable states in this paper, respectively. Based on the backstepping recursive technique and the common Lyapunov function method, a finite-time switching control method is presented. Due to the developed finite-time control strategy, the closed-loop signals can be ensured to be bounded under arbitrarily switchings, and the outputs of systems can quickly track the desired reference signals in finite time. The effectiveness of the proposed method is given through its application to a mass-spring-damper system. [ABSTRACT FROM AUTHOR]

Memristive crossbars have become a popular means for realizing unsupervised and supervised learning techniques. In previous neuromorphic architectures with leaky integrate-and-fire neurons, the crossbar itself has been separated from the neuron capacitors to preserve mathematical rigor. In this paper, we sought to design a simplified sparse coding circuit without this restriction, resulting in a fast circuit that approximated a sparse coding operation at a minimal loss in accuracy. We showed that connecting the neurons directly to the crossbar resulted in a more energy-efficient sparse coding architecture and alleviated the need to prenormalize receptive fields. This paper provides derivations for the design of such a network, named the simple spiking locally competitive algorithm, as well as CMOS designs and results on the CIFAR and MNIST data sets. Compared to a nonspiking, nonapproximate model which scored 33% on CIFAR-10 with a single-layer classifier, this hardware scored 32% accuracy. When used with a state-of-the-art deep learning classifier, the nonspiking model achieved 82% and our simplified, spiking model achieved 80% while compressing the input data by 92%. Compared to a previously proposed spiking model, our proposed hardware consumed 99% less energy to do the same work at 21 $\times $ the throughput. Accuracy held out with online learning to a write variance of 3%, suitable for the often reported 4-bit resolution required for neuromorphic algorithms, with offline learning to a write variance of 27%, and with read variance to 40%. The proposed architecture’s excellent accuracy, throughput, and significantly lower energy usage demonstrate the utility of our innovations. [ABSTRACT FROM AUTHOR]

This paper is concerned with the global nonfragile Mittag–Leffler synchronization and the global synchronization in finite time for fractional-order discontinuous neural networks, where activation functions are discontinuous at 0, or modeled as a local Hölder functions with the nonlinear growth property in a neighborhood of 0. First, two lemmas concerned with the convergence with respect to an absolutely continuous function are developed. Second, a new property, which introduces an inequality of the fractional derivative for the variable upper limit integral with respect to the nonsmooth integrable function, is presented and applied in the synchronization results’ analysis. In addition, under the fractional Filippov differential inclusion framework, by utilizing the Lur’e Postnikov-type Lyapunov functional, nonsmooth analysis method, and the convergence properties developed in this paper, the synchronization conditions are derived in the form of linear matrix inequalities. Moreover, the upper bound of the setting time for the global nonfragile synchronization in finite time is calculated accurately. Finally, two illustrations are presented to verify the correctness of the theoretical results. [ABSTRACT FROM AUTHOR]