Search

Spiking neural networks (SNNs) have garnered a great amount of interest for supervised and unsupervised learning applications. This paper deals with the problem of training multi-layer feedforward SNNs. The non-linear integrate-and-fire dynamics employed by spiking neurons make it difficult to train SNNs to generate desired spike trains in response to a given input. To tackle this, first the problem of training a multi-layer SNN is formulated as an optimization problem such that its objective function is based on the deviation in membrane potential rather than the spike arrival instants. Then, an optimization method named Normalized Approximate Descent (NormAD), hand-crafted for such non-convex optimization problems, is employed to derive the iterative synaptic weight update rule. Next, it is reformulated to efficiently train multi-layer SNNs, and is shown to be effectively performing spatio-temporal error backpropagation. The learning rule is validated by training 2-layer SNNs to solve a spike based formulation of the XOR problem as well as training 3-layer SNNs for generic spike based training problems. Thus, the new algorithm is a key step towards building deep spiking neural networks capable of efficient event-triggered learning. [ABSTRACT FROM AUTHOR]

Kinetic Monte Carlo simulations of resistive memory devices have been performed by paying attention to the vacancy–interstitial generation near the Hafnia-metal electrode interface. In our model, an oxygen vacancy is generated in Hafnia near the interface, with the corresponding oxygen atom residing in the metal electrode. These oxygen atoms form a thin insulating oxide layer at the Hafnia-active electrode interface. This interfacial layer is essential to thicken the filament, even after the filament bridges the two metal electrodes at low current levels. This thickening of the conducting filament is captured by the model and it naturally explains the trend of resistance decrease with an increase in compliance current found in some experiments. Simulations results as a function of the bonding energy between vacancies show a large increase in retention time with an increase in bonding energy. We also find that as the compliance current increases, the morphology of the filament transitions from conical to dumbbell-shaped. Finally, using a single set of values for various energies, our simulations capture the SET, RESET, and retention processes. [ABSTRACT FROM AUTHOR]

We demonstrate supervised learning in Spiking Neural Networks (SNNs) for the problem of handwritten digit recognition using the spike triggered Normalized Approximate Descent (NormAD) algorithm. Our network that employs neurons operating at sparse biological spike rates below 300 Hz achieves a classification accuracy of 98 . 17 % on the MNIST test database with four times fewer parameters compared to the state-of-the-art. We present several insights from extensive numerical experiments regarding optimization of learning parameters and network configuration to improve its accuracy. We also describe a number of strategies to optimize the SNN for implementation in memory and energy constrained hardware, including approximations in computing the neuronal dynamics and reduced precision in storing the synaptic weights. Experiments reveal that even with 3-bit synaptic weights, the classification accuracy of the designed SNN does not degrade beyond 1% as compared to the floating-point baseline. Further, the proposed SNN, which is trained based on the precise spike timing information outperforms an equivalent non-spiking artificial neural network (ANN) trained using back propagation, especially at low bit precision. Thus, our study shows the potential for realizing efficient neuromorphic systems that use spike based information encoding and learning for real-world applications. [ABSTRACT FROM AUTHOR]

Thermoelectric effects are envisioned to reduce programming currents in nanopillar phase-change memory (PCM) cells. However, due to the inherent symmetry in such a structure, the contribution due to thermoelectric effects on programming currents is minimal. In this paper, we propose a hybrid PCM structure, which incorporates a twofold asymmetry specifically aimed to favorably enhance the thermoelectric effects. The first asymmetry is introduced via an interface layer of low thermal conductivity and high negative Seebeck coefficient, such as polycrystalline SiGe, between the bottom electrode contact and the active region comprising the phase-change material. This results in an enhanced Peltier heating of the active material. The second one is introduced structurally via a taper that results in an angle-dependent Thomson heating within the active region. Various device geometries are analyzed using 2-D-axis-symmetric simulations to predict the effect on programming currents as well as for different thicknesses of the interface layer. A programming current reduction of up to 60% is predicted for specific cell geometries. Remarkably, we find that due to an interplay of Thomson cooling in the electrode and the asymmetric heating profile inside the active region, the predicted programming current reduction is resilient to fabrication variability. [ABSTRACT FROM PUBLISHER]

The goal of neuromorphic engineering is to build electronic systems that mimic the ability of the brain to perform fuzzy, fault-tolerant, and stochastic computation, without sacrificing either its space or power efficiency. In this paper, we determine the operating characteristics of novel nanoscale devices that could be used to fabricate such systems. We also compare the performance metrics of a million neuron learning system based on these nanoscale devices with an equivalent implementation that is entirely based on end-of-scaling digital CMOS technology and determine the technology targets to be satisfied by these new devices. We show that neuromorphic systems based on new nanoscale devices can potentially improve density and power consumption by at least a factor of 10, as compared with conventional CMOS implementations. [ABSTRACT FROM AUTHOR]

Laser annealing can be used for electrical activation of dopants without excessively heating the material deeper within the work piece. We demonstrate that laser annealing could be used for activating the dopants in the upper levels of an exemplary 3-D integrated circuit structure without affecting the operation of the devices below. We then use a 450 °C low-temperature oxide deposition process for forming the gate oxide and laser annealing for activating the dopants at the source/drain and gate regions to fabricate CMOS transistors. This process can be used to fabricate the transistors on the upper levels of a general 3-D IC structure without affecting the quality of the devices below. [ABSTRACT FROM AUTHOR]

Neuromorphic data carries information in spatio-temporal patterns encoded by spikes. Accordingly, a central problem in neuromorphic computing is training spiking neural networks (SNNs) to reproduce spatio-temporal spiking patterns in response to given spiking stimuli. Most existing approaches model the input-output behavior of an SNN in a deterministic fashion by assigning each input to a specific desired output spiking sequence. In contrast, in order to fully leverage the time-encoding capacity of spikes, this work proposes to train SNNs so as to match distributions of spiking signals rather than individual spiking signals. To this end, the paper introduces a novel hybrid architecture comprising a conditional generator, implemented via an SNN, and a discriminator, implemented by a conventional artificial neural network (ANN). The role of the ANN is to provide feedback during training to the SNN within an adversarial iterative learning strategy that follows the principle of generative adversarial network (GANs). In order to better capture multi-modal spatio-temporal distribution, the proposed approach – termed SpikeGAN – is further extended to support Bayesian learning of the generator's weight. Finally, settings with time-varying statistics are addressed by proposing an online meta-learning variant of SpikeGAN. Experiments bring insights into the merits of the proposed approach as compared to existing solutions based on (static) belief networks and maximum likelihood (or empirical risk minimization). In our experiments, handwritten digit images generated by SpikeGAN are observed to train an ANN classifier with $20\%$ 20 % higher accuracy than a comparable belief network. Our experiments also demonstrate the use of SpikeGAN to generate neuromorphic data sets from handwritten digits. It is shown that these data can be used to train an SNN classifier that achieves an accuracy level approaching the baseline accuracy of an SNN classifier trained on rate-encoded real data. [ABSTRACT FROM AUTHOR]

The computational efficiency of the human brain is believed to stem from the parallel information processing capability of neurons with integrated storage in synaptic interconnections programmed by local spike triggered learning rules such as spike timing dependent plasticity (STDP). The extremely low operating voltages (approximately 100 mV) used to trigger neuronal signaling and synaptic adaptation is believed to be a critical reason for the brain's power efficiency. We demonstrate the feasibility of spike triggered STDP behavior in a two-terminal Cu/SiO 2 /W memristive device capable of operating below 500 mV. We analyze the state-dependent nature of conductance updates in the device to develop a phenomenological model. Using the model, we evaluate the potential of such devices to generate precise spike times under supervised learning conditions and classify handwritten digits from the MNIST dataset in an unsupervised learning setting. The results form a promising step towards creating a low power synaptic device capable of on-chip learning. Unlabelled Image • STDP demonstration in Cu/SiO 2 /W memristor device. • Sub-quantization conductance range and below 500 mV operating voltage. • Supervised and unsupervised learning using phenomenological model. [ABSTRACT FROM AUTHOR]

Phase-change memory (PCM) is an emerging non-volatile memory technology that is based on the reversible and rapid phase transition between the amorphous and crystalline phases of certain phase-change materials. The ability to alter the conductance levels in a controllable way makes PCM devices particularly well-suited for synaptic realizations in neuromorphic computing. A key attribute that enables this application is the progressive crystallization of the phase-change material and subsequent increase in device conductance by the successive application of appropriate electrical pulses. There is significant inter- and intra-device randomness associated with this cumulative conductance evolution, and it is essential to develop a statistical model to capture this. PCM also exhibits a temporal evolution of the conductance values (drift), which could also influence applications in neuromorphic computing. In this paper, we have developed a statistical model that describes both the cumulative conductance evolution and conductance drift. This model is based on extensive characterization work on 10 000 memory devices. Finally, the model is used to simulate the supervised training of both spiking and non-spiking artificial neuronal networks. [ABSTRACT FROM AUTHOR]

A study is conducted to investigate the relative advantages of different driving devices for phase-change memory cells using 3-D numerical device simulation. Among various possible choices, p-n diodes and vertical gate-all-around (GAA) metal–oxide–semiconductor field-effect transistors (MOSFETs) are studied in detail as they represent distinct classes of driving devices. Different performance parameters including cell size, current drive, disturb immunity, power dissipation, and scalability are carefully compared. While p-n diodes show superiority in technology nodes with large device dimensions, the scaling process has improved the performance of GAA MOSFETs significantly to outperform that of p-n diodes in extremely scaled technologies. [ABSTRACT FROM PUBLISHER]

Spiking neural networks (SNN) are computational models inspired by the brain's ability to naturally encode and process information in the time domain. The added temporal dimension is believed to render them more computationally efficient than the conventional artificial neural networks, though their full computational capabilities are yet to be explored. Recently, in-memory computing architectures based on non-volatile memory crossbar arrays have shown great promise to implement parallel computations in artificial and spiking neural networks. In this work, we evaluate the feasibility to realize high-performance event-driven in-situ supervised learning systems using nanoscale and stochastic analog memory synapses. For the first time, the potential of analog memory synapses to generate precisely timed spikes in SNNs is experimentally demonstrated. The experiment targets applications which directly integrates spike encoded signals generated from bio-mimetic sensors with in-memory computing based learning systems to generate precisely timed control signal spikes for neuromorphic actuators. More than 170,000 phase-change memory (PCM) based synapses from our prototype chip were trained based on an event-driven learning rule, to generate spike patterns with more than 85% of the spikes within a 25 ms tolerance interval in a 1250 ms long spike pattern. We observe that the accuracy is mainly limited by the imprecision related to device programming and temporal drift of conductance values. We show that an array level scaling scheme can significantly improve the retention of the trained SNN states in the presence of conductance drift in the PCM. Combining the computational potential of supervised SNNs with the parallel compute power of in-memory computing, this work paves the way for next-generation of efficient brain-inspired systems. [ABSTRACT FROM AUTHOR]

Abstract Deep Neural Networks (DNN) have proven to be highly effective in extracting high level abstractions of input data using multiple neural network layers. However, the huge training times for DNNs in traditional von-Neumann machines have hindered their ubiquitous adoption in IoT and other mobile computing platforms. Recently, acceleration of DNN with a time complexity of O (1) was proposed using the idea of stochastic weight update with resistive processing units (RPU). However, it has been projected that RPU devices require more than 1000 reliable conductance levels, which is a stringent requirement to realize in memristive devices. Here, we study the optimization of stochastic learning for DNNs for the hand-written digit classification benchmark using the characteristics of non-filamentary Pr 0.7 Ca 0.3 MnO 3 (PCMO) devices that are fabricated using a standard lithography process. The electrical characteristics of these devices exhibit a linear conductance response with an on-off ratio of 1.8 with 26 levels and significant programming variability. We captured these non-ideal behaviors of experimental PCMO device in the simulations to demonstrate stochastic learning with O (1) time complexity, achieving a test accuracy of 88.1% for the hand-written digit recognition benchmark. While the linearity, dynamic range, bit resolution, programming variability and the reset rate of the device conductance to account for its limited dynamic range have to be co-optimized for improving the training efficiency, we show that programming variability has the paramount role in determining the network performance. We also show that if devices with reduced programming variability (5x smaller compared to our experimental device) can be developed keeping all other parameters constant, it is possible to boost the network performance as high as 95%. We also observe that the performance of stochastic DNNs with memristive synapses is independent of the on-off ratio of the devices for a fixed programming variability. Thus, programming variability represents a new optimization corner for on-chip learning of stochastic DNNs. Further, we also evaluate the performance of stochastic inference engines to noise corrupted input test data as a function of the variability in the memristive devices. We demonstrate that noise-resilient inference engines can be achieved if 100 bits are used for stochastic encoding during inference even though the expensive network training can be done with as few as 10 bits. Thus, our studies emphasize the need for optimization of learning strategies for DNNs based on the non-ideal characteristics of experimental nanoscale devices. [ABSTRACT FROM AUTHOR]

Memristive devices, whose conductance depends on previous programming history, are of significant interest for building nonvolatile memory and brain-inspired computing systems. Here, we report half-integer quantized conductance transitions G = (n/2) (2e²/h) for n = 1, 2, 3, etc., in Cu/SiO2/W memristive devices observed below 300 mV at room temperature. This is attributed to the nanoscale filamentary nature of Cu conductance pathways formed inside SiO2. Retention measurements also show spontaneous filament decay with quantized conductance levels. Numerical simulations shed light into the dynamics underlying the data retention loss mechanisms and provide new insights into the nanoscale physics of memristive devices and trade-offs involved in engineering them for computational applications. [ABSTRACT FROM AUTHOR]

This report discusses the electrical characteristics of two-terminal synaptic memory devices capable of demonstrating an analog change in conductance in response to the varying amplitude and pulse-width of the applied signal. The devices are based on Mn doped HfO2 material. The mechanism behind reconfiguration was studied and a unified model is presented to explain the underlying device physics. The model was then utilized to show the application of these devices in speech recognition. A comparison between a 20 nm * 20 nm sized synaptic memory device with that of a state-of-the-art VLSI SRAM synapse showed ~10* reduction in area and >106 times reduction in the power consumption per learning cycle. [ABSTRACT FROM AUTHOR]

Thin-film ferromagnetic inductors show great potential as the energy storage element for integrated circuits containing on-chip power management. In order to achieve the high energy storage required for power management, on-chip inductors require relatively thick magnetic yoke materials (several microns or more), which can be readily deposited by electroplating through a photoresist mask as demonstrated in this paper, the yoke material of choice being Ni45Fe55, whose properties of relatively high moment and electrical resistivity make it an attractive model yoke material for inductors. Inductors were designed with a variety of yoke geometries, and included both single-turn and multi-turn coil designs, which were fabricated on 200 mm silicon wafers in a CMOS back-end-of-line (BEOL) facility. Each inductor consisted of electroplated copper coils enclosed by the electroplated Ni45Fe55 yokes; aspects of the fabrication of the inductors are discussed. Magnetic properties of the electroplated yoke materials are described, including high frequency permeability measurements. The inductance of 2-turn coil inductors, for example, was enhanced up to about 6 times over the air core equivalent, with an inductance density of 130 nH/mm2 being achieved. The resistance of these non-laminated inductors was relatively large at high frequency due to magnetic and eddy current losses but is expected to improve as the yoke material/structure is further optimized, making electroplated yoke-containing inductors attractive for dc-dc power converters. [ABSTRACT FROM AUTHOR]