Search

Recent advancements in neuromorphic computing have led to the development of hardware architectures inspired by Spiking Neural Networks (SNNs) to emulate the efficiency and parallel processing capabilities of the human brain. This work focuses on testing the HEENS architecture, specifically designed for high parallel processing and biological realism in SNN emulation, implemented on a ZYNQ family FPGA. The study applies this architecture to the classification of digits using the well-known MNIST database. The image resolutions were adjusted to match HEENS' processing capacity. Results were compared with existing work, demonstrating HEENS' performance comparable to other solutions. This study highlights the importance of balancing accuracy and efficiency in the execution of applications. HEENS offers a flexible solution for SNN emulation, allowing for the implementation of programmable neural and synapticmodels. It encourages the exploration of novel algorithms and network architectures, providing an alternative for real-time processing with efficient energy consumption. [ABSTRACT FROM AUTHOR]

At present, implementation of learning mechanisms in spiking neural networks (SNN) cannot be considered as a solved scientific problem despite plenty of SNN learning algorithms proposed. It is also true for SNN implementation of reinforcement learning (RL), while RL is especially important for SNNs because of its close relationship to the domains most promising from the viewpoint of SNN application such as robotics. In the present paper, an SNN structure is described which, seemingly, can be used in wide range of RL tasks. The distinctive feature of our approach is usage of only the spike forms of all signals involved — sensory input streams, output signals sent to actuators and reward/punishment signals. Besides that, selection of the neuron/plasticity models was determined by the requirement that they should be easily implemented on modern neurochips. The SNN structure considered in the paper includes spiking neurons described by a generalization of the LIFAT (leaky integrate-and-fire neuron with adaptive threshold) model and a simple spike timing dependent synaptic plasticity model (a generalization of dopamine-modulated plasticity). In this study, we use the model-free approach to RL but it is based on very general assumptions about RL task characteristics and has no visible limitations on its applicability (inside the class of model-free RL tasks). To test our SNN, we apply it to a simple but non-trivial task of training the network to keep a chaotically moving light spot in the view field of an emulated Dynamic Vision Sensor (DVS) camera. Successful solution of this RL problem can be considered as an evidence in favor of efficiency of our approach. [ABSTRACT FROM AUTHOR]

Artificial Neural Networks (ANN) have gained significant popularity thanks to their ability to learn using the well-known backpropagation algorithm. Conversely, Spiking Neural Networks (SNNs), despite having broader capabilities than ANNs, have always posed challenges in the training phase. This paper shows a new method to perform supervised learning on SNNs, using Spike Timing Dependent Plasticity (STDP) and homeostasis, aiming at training the network to identify spatial patterns. Spatial patterns refer to spike patterns without a time component, where all spike events occur simultaneously. The method is tested using the SpiNNaker digital architecture. A SNN is trained to recognise one or multiple patterns and performance metrics are extracted to measure the performance of the network. Some considerations are drawn from the results showing that, in the case of a single trained pattern, the network behaves as the ideal detector, with 100% accuracy in detecting the trained pattern. However, as the number of trained patterns on a single network increases, the accuracy of identification is linked to the similarities between these patterns. This method of training an SNN to detect spatial patterns may be applied to pattern recognition in static images or traffic analysis in computer networks, where each network packet represents a spatial pattern. It will be stipulated that the homeostatic factor may enable the network to detect patterns with some degree of similarity, rather than only perfectly matching patterns. The principles outlined in this article serve as the fundamental building blocks for more complex systems that utilise both spatial and temporal patterns by converting specific features of input signals into spikes. One example of such a system is a computer network packet classifier, tasked with real-time identification of packet streams based on features within the packet content. [ABSTRACT FROM AUTHOR]

Spiking Neural Networks are often touted as brain-inspired learning models for the third wave of Artificial Intelligence. Although recent SNNs trained with supervised backpropagation show classification accuracy comparable to deep networks, the performance of unsupervised learning-based SNNs remains much lower. This paper presents a heterogeneous recurrent spiking neural network (HRSNN) with unsupervised learning for spatio-temporal classification of video activity recognition tasks on RGB (KTH, UCF11, UCF101) and event-based datasets (DVS128 Gesture). We observed an accuracy of 94.32%for the KTHdataset, 79.58%and 77.53%for theUCF11 and UCF101 datasets, respectively, and an accuracy of 96.54% on the event-based DVS Gesture dataset using the novel unsupervised HRSNN model. The key novelty of the HRSNN is that the recurrent layer in HRSNN consists of heterogeneous neurons with varying firing/relaxation dynamics, and they are trained via heterogeneous spike-time-dependent-plasticity (STDP) with varying learning dynamics for each synapse. We show that this novel combination of heterogeneity in architecture and learning method outperforms current homogeneous spiking neural networks. We further show that HRSNN can achieve similar performance to state-of-the-art backpropagation trained supervised SNN, but with less computation (fewer neurons and sparse connection) and less training data. [ABSTRACT FROM AUTHOR]

Spiking Neural Networks (SNNs), recognized for their biological plausibility and energy efficiency, employ sparse and asynchronous spikes for communication. However, the training of SNNs encounters difficulties coming from non-differentiable activation functions and the movement of spike-based inter-layer data. Spike-Timing Dependent Plasticity (STDP), inspired by neurobiology, plays a crucial role in SNN's learning. Various models of STDP are introduced in the literature with adaptability and integrability but they still lacks the conscious part of the brain used for learning. Considering the issue, this research work proposes a Consciousness Driven STDP (C D -STDP), an improved solution addressing inherent limitations observed in conventional STDP models. C D -STDP, designed to infuse the conscious part as coefficients of long-term potentiation (LTP) and long-term depression (LTD), exhibit a dynamic nature. The model connects LTP and LTD coefficients to current and past state of synaptic activities, respectively, enhancing consciousness and adaptability. The conscious coefficient adjustment in response to current and past synaptic activity extends the model's conscious and other cognitive capabilities, offering a refined and efficient approach for real-world applications. Evaluations on Modified National Institute of Standards and Technology (MNIST), FashionMNIST and California Institute of Technology (CALTECH) datasets showcase C D -STDP's remarkable accuracy of 98.6%, 85.61% and 99.0%, respectively, in a single hidden layer SNN. In addition, analysis of conscious elements and consciousness of the proposed model on SNN is performed. [ABSTRACT FROM AUTHOR]

A major characteristic of spiking neural networks (SNNs) over conventional artificial neural networks (ANNs) is their ability to spike, enabling them to use spike timing for coding and efficient computing. In this paper, we assess if neuromorphic datasets recorded from static images are able to evaluate the ability of SNNs to use spike timings in their calculations. We have analyzed N-MNIST, N-Caltech101 and DvsGesture along these lines, but focus our study on N-MNIST. First we evaluate if additional information is encoded in the time domain in a neuromorphic dataset. We show that an ANN trained with backpropagation on frame-based versions of N-MNIST and N-Caltech101 images achieve 99.23 and 78.01% accuracy. These are comparable to the state of the art—showing that an algorithm that purely works on spatial data can classify these datasets. Second we compare N-MNIST and DvsGesture on two STDP algorithms, RD-STDP, that can classify only spatial data, and STDP-tempotron that classifies spatiotemporal data. We demonstrate that RD-STDP performs very well on N-MNIST, while STDP-tempotron performs better on DvsGesture. Since DvsGesture has a temporal dimension, it requires STDP-tempotron, while N-MNIST can be adequately classified by an algorithm that works on spatial data alone. This shows that precise spike timings are not important in N-MNIST. N-MNIST does not, therefore, highlight the ability of SNNs to classify temporal data. The conclusions of this paper open the question—what dataset can evaluate SNN ability to classify temporal data? [ABSTRACT FROM AUTHOR]

Spiking neural networks are energy efficient and biological interpretability, communicating through sparse, asynchronous spikes, which makes them suitable for neuromorphic hardware. However, due to the nature of binary weights and spike trains in time-coded binarized spiking neural networks, their forward propagation may cause neurons to not fire spikes, and their backward propagation has non-differentiable problems. Moreover, the current use of deep and complex network structures generates a large number of redundant parameters. Therefore, we need effective methods to improve energy efficiency without reducing accuracy. We propose a dynamic threshold model that can reduce the number of dead neurons. We combine the backpropagation algorithm and the spike timing dependent plasticity algorithm to avoid the non-differentiable problem. We propose a neuron pruning strategy based on adaptive firing time threshold. This pruning strategy prunes 267 neurons in a network of 600 neurons, reducing the network size and obtaining a more compact network structure. The energy efficiency is improved by 0.55 × , while the classification accuracy is lost by 1.1%. [ABSTRACT FROM AUTHOR]

Neural network approach to processing have been shown successful and efficient in numerous real world applications. The most successful of this approach are implemented in software but in order to achieve real-time processing similar to that of biological neural networks, hardware implementations of these networks need to be continually improved. This work presents a spiking neural network (SNN) implemented in digital CMOS. The SNN is constructed based on an indirect training algorithm that utilizes spike-timing dependent plasticity (STDP). The SNN is validated by using its outputs to control the motion of a virtual insect. The indirect training algorithm is used to train the SNN to navigate through a terrain with obstacles. The indirect approach is more appropriate for nanoscale CMOS implementation synaptic training since it is getting more difficult to perfectly control matching in CMOS circuits. [ABSTRACT FROM AUTHOR]

Abstract: Spike timing dependent plasticity (STDP) forms the basis of learning within neural networks. STDP allows for the modification of synaptic weights based upon the relative timing of pre- and post-synaptic spikes. A compact circuit is presented which can implement STDP, including the critical plasticity window, to determine synaptic modification. A physical model to predict the time window for plasticity to occur is formulated and the effects of process variations on the window is analyzed. The STDP circuit is implemented using two dedicated circuit blocks, one for potentiation and one for depression where each block consists of 4 transistors and a polysilicon capacitor. SpectreS simulations of the back-annotated layout of the circuit and experimental results indicate that STDP with biologically plausible critical timing windows over the range from 10µs to 100ms can be implemented. Also a floating gate weight storage capability, with drive circuits, is presented and a detailed analysis correlating weights changes with charging time is given. [Copyright &y& Elsevier]

Abstract: Neural development and differentiation are characterized by an overproduction of cells and a transient exuberant number of connections followed by cell death and selective synaptic pruning. We simulated large spiking neural networks (10,000 units at its maximum size) with and without an ontogenetic process corresponding to a brief initial phase of apoptosis driven by an excessive firing rate mimicking cell death due to glutamatergic neurotoxicity and glutamate-triggered apoptosis. This phase was followed by the onset of spike timing dependent synaptic plasticity (STDP), driven by spatiotemporal patterns of stimulation. Despite the reduction in cell counts the apoptosis tended to increase the excitatory/inhibitory ratio because the inhibitory cells were affected at first. Recurrent spatiotemporal firing patterns emerged in both developmental condition but they differed in dynamics. They were less numerous but repeated more often after apoptosis. The results suggest that initial cell death may be necessary for the emergence of stable cell assemblies, able to sustain and process temporal information, from the initially randomly connected networks. [Copyright &y& Elsevier]

Abstract: We investigate how different forms of plasticity shape the dynamics and computational properties of simple recurrent spiking neural networks. In particular, we study the effect of combining two forms of neuronal plasticity: spike timing dependent plasticity (STDP), which changes the synaptic strength, and intrinsic plasticity (IP), which changes the excitability of individual neurons to maintain homeostasis of their activity. We find that the interaction of these forms of plasticity gives rise to interesting network dynamics characterized by a comparatively large number of stable limit cycles. We study the response of such networks to external input and find that they exhibit a fading memory of recent inputs. We then demonstrate that the combination of STDP and IP shapes the network structure and dynamics in ways that allow the discovery of patterns in input time series and lead to good performance in time series prediction. Our results underscore the importance of studying the interaction of different forms of plasticity on network behavior. [Copyright &y& Elsevier]