Search

Deep learning (DL) has proven to be one of the most pivotal components of machine learning given its notable performance in a variety of application domains. Neural networks (NNs) for DL are tailored to specific application domains by varying in their topology and activation nodes. Nevertheless, the major operation type (with the largest computational complexity) is commonly multiply‐accumulate operation irrespective of their topology. Recent trends in DL highlight the evolution of NNs such that they become deeper and larger, and thus their prohibitive computational complexity. To cope with the consequent prohibitive latency for computation, 1) general‐purpose hardware, e.g., central processing units and graphics processing units, has been redesigned, and 2) various DL accelerators have been newly introduced, e.g., neural processing units, and computing‐in‐memory units for deep NN‐based DL, and neuromorphic processors for spiking NN‐based DL. In this review, these accelerators and their pros and cons are overviewed with particular focus on their performance and memory bandwidth. [ABSTRACT FROM AUTHOR]

Feedforward deep neural networks (DNNs) commonly involve layer-wise linear operations and subsequent nonlinear operations, which are repeated through all layers. The nonlinear operations by nonlinear activations in each layer remarkably enhance the expressiveness of DNNs, resulting in the great success in a variety of application domains. Although the necessity of layer-wise nonlinear operations is agreed, the optimal nonlinearity for each layer in a given DNN is not clear. In this regard, we propose an easy-to-use method to layer-wise measure the optimal nonlinearity for a given DNN using its replica termed a linear-nonlinear network (LiNLNet). The key to the LiNLNet is the use of linear-nonlinear units (LiNLUs) whose degree of nonlinearity is parameterized by a trainable parameter p. The parameter p is shared among all LiNLUs in a given layer, thus indicating the layer-wise optimal nonlinearity. This method allows layer-level pruning such that the layers that do not require nonlinearity are merged into the subsequent layers, reducing computational complexity. For proofs of concept, we applied the proposed method to a MLP, AlexNet, VGG16, and ResNet18 on CIFAR-10 and ImageNet. The results commonly indicate the last hidden layer as a linear layer that may be merged into the output layer, reducing memory usage by 27% while maintaining the accuracy for LiNL-AlexNet on ImageNet. [ABSTRACT FROM AUTHOR]

Computing‐in‐memory (CIM) is considered a feasible solution to the acceleration of multiply‐accumulate (MAC) operations at low power. The key to CIM is parallel MAC operations in the memory domain, and thus reductions in power consumption and memory‐access latency. Resistive random access memory (RRAM) can be a good candidate for the memory for CIM given its data nonvolatility, high data density, low‐latency read‐out, multilevel representation, and inherent current accumulation capability. Particularly, the last two attributes offer analog MAC operations in parallel in the memory domain. However, the fully analog MAC operation scheme causes significant power and area overheads for its peripheral circuits, particularly, analog‐to‐digital converters. To compensate for these downsides using digital processing, a method for sub‐array‐wise partial MAC operations over weight‐resistors that are optimally split to minimize power and area overheads for the peripheral circuits is proposed. The simulations performed highlight the optimal sub‐array of 4×w/2 in size. That is, weight‐splitting such that a single w‐bit weight is represented by w/2 RRAM cells, i.e., 2‐bit for each cell. For 8‐bit weights, the figure of merit (FOM) for this optimal case reaches ≈28.3× FOM for the case of no weight‐splitting. [ABSTRACT FROM AUTHOR]

Spiking neural networks (SNNs), as time-dependent hypotheses consisting of spiking nodes (neurons) and directed edges (synapses), are believed to offer unique solutions to reward prediction tasks and the related feedback that are classified as reinforcement learning. Generally, temporal difference (TD) learning renders it possible to optimize a model network to predict the delayed reward in an ad hoc manner. Neuromorphic SNNs—networks built using dedicated hardware—particularly leverage such TD learning for not only reward prediction but also temporal sequence prediction in a physical time domain. In this tutorial, such learning in a physical time domain is referred to as temporal learning to distinguish it from conventional TD learning-based methods that generally involve algorithmic (rather than physical) time. This tutorial addresses neuromorphic SNNs for temporal learning from scratch. It first concerns the general characteristics of SNNs including spiking neurons and information coding schemes and then moves on to temporal learning including its general concept, feasible algorithms, and its association with neurophysiological learning rules that have been intensively enriched in the last few decades. [ABSTRACT FROM AUTHOR]

Mobile species and matrix materials in ion motion‐mediated memristors predominantly determine the switching characteristics and device performance. As a result of exploring a new type of mobile species, a Ru ion‐mediated electrochemical metallization‐like memristor with an amorphous oxide matrix is recently suggested to achieve a low switching current, voltage, and good retention simultaneously. Although the ion migration of Ru in the oxide matrix is previously confirmed, no in‐depth study on how the crystallinity of the oxide matrix influences the Ru ion motion and switching characteristics has not been reported. Therefore, in this study, the crystallinity‐dependent resistive switching behavior of the Pt/HfO2/Ru structure device is investigated. With the crystallized HfO2 layer, the preferred Ru ion migration through the grain boundaries occurs owing to the enhanced ion mobility, resulting in a high switching current (≈100 µA) with continuous metallic Ru conducting filaments. The discontinuous conducting filaments with amorphous HfO2 exhibit a low switching current. In addition, highly linear and symmetric conductance modulation properties are achieved, and over 91.5% accuracy in the Mixed National Institute of Standards and Technology (MNIST) pattern recognition test is demonstrated. [ABSTRACT FROM AUTHOR]

Reducing computational complexity is essential in future computing systems for processing a large amount of unstructured data simultaneously. Dot‐product operations using crossbar array devices have attracted considerable attention owing to their simple device structure, intuitive operation scheme, and high computational efficiency of parallel operation. The resistive switching device is considered a promising candidate as the main data storage in the crossbar array owing to its highly reliable performance. In this study, a tri‐layer TaOx/Al2O3/Ti:SiOx‐based resistive switching device is proposed. The proposed device exhibits a high electrical selectivity of 2.5 × 105 based on the optimized biasing scheme, a stable non‐volatility, and reliable read disturbance characteristic of up to 108. Additionally, the device achieves high reading current of 1 µA and a low off‐leakage current of 1 pA, which favors the reliable characteristics in the data writing sequence and the dot‐product operation in the crossbar array device. Furthermore, the resistive switching mechanism based on the material and electrical conduction characteristics is analyzed. Lastly, the dot‐product operation in an 8 × 8 crossbar array is performed. As a result, the calculated and measured signal values in each column in the crossbar array of the device are found to be in good agreement. [ABSTRACT FROM AUTHOR]

Conductive bridging random access memory (CBRAM) has been considered to be a promising emerging device for artificial synapses in neuromorphic computing systems. Good analog synaptic behaviors, such as linear and symmetric synapse updates, are desirable to provide high learning accuracy. Although numerous efforts have been made to develop analog CBRAM for years, the stochastic and abrupt formation of conductive filaments hinders its adoption. In this study, we propose a novel approach to enhance the synaptic behavior of a SiNx/a-Si bilayer memristor through Ge implantation. The SiNx and a-Si layers serve as switching and internal current limiting layers, respectively. Ge implantation induces structural defects in the bulk and surface regions of the a-Si layer, enabling spatially uniform Ag migration and nanocluster formation in the upper SiNx layer and increasing the conductance of the a-Si layer. As a result, the analog synaptic behavior of the SiNx/a-Si bilayer memristor, such as the nonlinearity, on/off ratio, and retention time, is remarkably improved. An artificial neural network simulation shows that the neuromorphic system with the implanted SiNx/a-Si memristor provides a 91.3% learning accuracy mainly due to the improved linearity. Memory devices: Ionic implants help mimic the synapse An approach that tweaks the structures of a new type of memory chip shows promise for neuromorphic computing applications. In conductive bridging random access memory (CBRAM) technology, nanoscale metal filaments inside special insulators are used to store data in either high or low conductance states. Inho Kim from the Korea Institute of Science and Technology in Seoul, South Korea and colleagues now report beneficial effects when germanium ions are implanted into silicon-based CBRAM using high-energy beams. The team's microscopy experiments revealed that the implantation caused metal nanoclusters to distribute evenly within the device's switching layer. These particles enabled smoother transitions between conductance states than conventional CBRAM, better mimicking the analog switching seen in synapses. Neural network simulations demonstrated that ion-implanted devices had significantly improved learning accuracy compared to unmodified CBRAM chips. We propose a novel approach to enhance the synaptic behavior of a SiNx/a-Si bilayer memristor through Ge implantation. The SiNx and a-Si layers serve as switching and internal current limiting layers, respectively. Ge implantation induces structural defects in the bulk and surface regions of the a-Si layer, enabling spatially uniform Ag migration and nanocluster formation in the upper SiNx layer and increasing the conductance of the a-Si layer. As a result, the analog synaptic behavior of the SiNx/a-Si bilayer memristor, such as the nonlinearity, on/off ratio, and retention time, is remarkably improved. [ABSTRACT FROM AUTHOR]

In this study, highly reliable and accurate weight‐modification behaviors are realized using a W/Al2O3 (3 nm)/HfO2 (7 nm)/TiN memristive device. The accuracy of the simulated inference of the MNIST dataset when considering the weight‐modification behavior is ≈95%. It is determined the optimal programming voltage pulsing conditions considering i) a high linearity in the weight‐modification, ii) symmetry between potentiation and depression, and iii) an alleviation of the voltage‐driving circuit overhead for the related part of weight‐modification process. Particular emphasis is placed on the last concern, and thus, the fixed shape of each programming pulse for both potentiation and depression are utilized. The optimal pulse design is 500 µs for the pulse rising, plateau, and falling times and a 2 V amplitude at the absolute scale. Additionally, the nonparametric method to evaluate the linearity and symmetry as opposed to the application of several parametric methods are proposed. The nonparametric method is based on an evaluation of actual data rather than models, and thus considers the actual variability in the conductance change, which is otherwise often ignored in the parameter optimization process. [ABSTRACT FROM AUTHOR]

The energetics of nitrogen-doped Ge2Sb2Te5 (GST225) with a hexagonal structure was investigated using first-principles calculations. Nitrogen was considered to be incorporated into the GST225 matrix as either N atoms (2.7 at. %) or N2 molecules (5.3 at. %) at four possible interstitial sites. The formation energies of nitrogen-doped GST225 were all positive for both the atomic state and molecular state nitrogen, implying that the incorporation of nitrogen is not a spontaneous reaction with respect to the N2 molecule. The formation energies of GST225 doped with N2 molecule are much lower than those of GST225 doped with nitrogen atom for all of the interstitial sites considered. The changes in the electron densities caused by the addition of nitrogen showed that atomic nitrogen forms bonds with neighboring constituent atoms of GST225 and prefers Ge as the first nearest atom. On the other hand, N2 stays almost still and remains in the molecular state at the interstitial sites. The calculation results support two different experimental reports on the state of nitrogen in GST225. [ABSTRACT FROM AUTHOR]

Electroforming effects on the composition, structure, and electrical resistance of Pt/TiO2/Pt switching cells are investigated. The correlation between the electroforming procedure and the resulting bipolar switching behavior is discussed. The dependence of electroforming behavior on atmosphere is also identified, from which we define symmetric or asymmetric electroforming. The symmetry of electroforming is a key factor determining the resulting bipolar switching characteristics. From the experimental results we suggest a possible mechanism for electroforming in Pt/TiO2/Pt in terms of the formation of oxygen gas and vacancies in the vicinity of the anode. [ABSTRACT FROM AUTHOR]

The mechanism of the differential negative resistance behavior observable in transition metal oxides such as TiO2, NiO, and Nb2O5 is theoretically developed in terms of the order to disorder transition of local conducting pathways, so called filaments, by applying statistical mechanics. Joule heat is taken as a heat source giving rise to an increase of temperature in the filaments. The free energy of the filaments is evaluated by taking the site percolation model as well as the nonhomogeneity of an order parameter into account. It is concluded that the transition of a filament from the conducting state, which is dominated by the internal bonding energy term, to the nonconducting state is due to an increased contribution of the entropy term to the system’s total energy at a higher temperature. The calculation results also show that the rupture of a filament occurs in the middle of the filament when the filaments are in contact with thermally conducting metal electrodes, such as Pt. [ABSTRACT FROM AUTHOR]

The physical unclonable function (PUF) based on resistive random-access memory (RRAM) possesses a distinctive advantage that can offer higher security and lower cost than the traditional complementary metal–oxide–semiconductor-based cryptographic devices and other conventional PUFs. The intrinsic stochasticity of RRAM devices successfully provides attractive properties to implement PUF. In this paper, we present a novel multistate-based RRAM PUF to realize strong tolerance against attack. By applying multilevel states with bit shuffling to the RRAM PUF, the randomness and uniqueness were enhanced close to ideal values. In addition, the bit error rate was dramatically reduced using the temperature compensation mechanism. Moreover, our new method not only enables the generation of larger challenge–response pairs (CRPs) with less footprint but also can reconfigure CRPs. [ABSTRACT FROM AUTHOR]

Emerging non-volatile reduction-oxidation (redox)-based resistive switching memories (ReRAMs) exhibit a unique set of characteristics that make them promising candidates for the next generation of low-cost, low-power, tiny, and secure physical unclonable functions (PUFs). Their underlying stochastic ionic conduction behavior, intrinsic nonlinear current-voltage characteristics, and their well-known nano-fabrication process variability might normally be considered disadvantageous ReRAM features. However, using a combination of a novel architecture and special peripheral circuitry, this paper exploits these non-idealities in a physical one-way function, nonlinear resistive PUF, potentially applicable to a variety of cyber-physical security applications. We experimentally verify the performance of valency change mechanism (VCM)-based ReRAM in nano-fabricated crossbar arrays across multiple dies and runs. In addition to supporting a massive pool of challenge-response pairs (CRPs), using a combination of experiment and simulation our proposed PUF exhibits a reliability of 98.67%, a uniqueness of 49.85%, a diffuseness of 49.86%, a uniformity of 47.28%, and a bit-aliasing of 47.48%. [ABSTRACT FROM PUBLISHER]

In this Review, memristors are examined from the frameworks of both von Neumann and neuromorphic computing architectures. For the former, a new logic computational process based on the material implication is discussed. It consists of several memristors which play roles of combined logic processor and memory, called stateful logic circuit. In this circuit configuration, the logic process flows primarily along a time dimension, whereas in current von Neumann computers it occurs along a spatial dimension. In the stateful logic computation scheme, the energy required for the data transfer between the logic and memory chips can be saved. The non‐volatile memory in this circuit also saves the energy required for the data refresh. Neuromorphic (cognitive) computing refers to a computing paradigm that mimics the human brain. Currently, the neuromorphic or cognitive computing mainly relies on the software emulation of several brain functionalities, such as image and voice recognition utilizing the recently highlighted deep learning algorithm. However, the human brain typically consumes ≈10–20 Watts for selected “human‐like” tasks, which can be currently mimicked by a supercomputer with power consumption of several tens of kilo‐ to megawatts. Therefore, hardware implementation of such brain functionality must be eventually sought for power‐efficient computation. Several fundamental ideas for utilizing the memristors and their recent progresses in these regards are reviewed. Finally, material and processing issues are dealt with, which is followed by the conclusion and outlook of the field. These technical improvements will substantially decrease the energy consumption for futuristic information technology. [ABSTRACT FROM AUTHOR]

Issues in the circuitry, integration, and material properties of the two-dimensional (2D) and three-dimensional (3D) crossbar array (CBA)-type resistance switching memories are described. Two important quantitative guidelines for the memory integration are provided with respect to the required numbers of signal wires and sneak current paths. The advantage of 3D CBAs over 2D CBAs (i.e., the decrease in effect memory cell size) can be exploited only under certain limited conditions due to the increased area and layout complexity of the periphery circuits. The sneak current problem can be mitigated by the adoption of different voltage application schemes and various selection devices. These have critical correlations, however, and depend on the involved types of resistance switching memory. The problem is quantitatively dealt with using the generalized equation for the overall resistance of the parasitic current paths. Atomic layer deposition is discussed in detail as the most feasible fabrication process of 3D CBAs because it can provide the device with the necessary conformality and atomic-level accuracy in thickness control. Other subsidiary issues related to the line resistance, maximum available current, and fabrication technologies are also reviewed. Finally, a summary and outlook on various other applications of 3D CBAs are provided. [ABSTRACT FROM AUTHOR]

Chemical synapses are important components of the large-scaled neural network in the hippocampus of the mammalian brain, and a change in their weight is thought to be in charge of learning and memory. Thus, the realization of artificial chemical synapses is of crucial importance in achieving artificial neural networks emulating the brain’s functionalities to some extent. This kind of research is often referred to as neuromorphic engineering. In this study, we report short-term memory behaviours of electrochemical capacitors (ECs) utilizing TiO2 mixed ionic–electronic conductor and various reactive electrode materials e.g. Ti, Ni, and Cr. By experiments, it turned out that the potentiation behaviours did not represent unlimited growth of synaptic weight. Instead, the behaviours exhibited limited synaptic weight growth that can be understood by means of an empirical equation similar to the Bienenstock–Cooper–Munro rule, employing a sliding threshold. The observed potentiation behaviours were analysed using the empirical equation and the differences between the different ECs were parameterized. [ABSTRACT FROM AUTHOR]