Search

Spiking neural networks (SNNs) possess energy-efficient potential due to event-based computation. However, supervised training of SNNs remains a challenge as spike activities are non-differentiable. Previous SNNs training methods can be generally categorized into two basic classes, i.e., backpropagation-like training methods and plasticity-based learning methods. The former methods are dependent on energy-inefficient real-valued computation and non-local transmission, as also required in artificial neural networks (ANNs), whereas the latter are either considered to be biologically implausible or exhibit poor performance. Hence, biologically plausible (bio-plausible) high-performance supervised learning (SL) methods for SNNs remain deficient. In this paper, we proposed a novel bio-plausible SNN model for SL based on the symmetric spike-timing dependent plasticity (sym-STDP) rule found in neuroscience. By combining the sym-STDP rule with bio-plausible synaptic scaling and intrinsic plasticity of the dynamic threshold, our SNN model implemented SL well and achieved good performance in the benchmark recognition task (MNIST dataset). To reveal the underlying mechanism of our SL model, we visualized both layer-based activities and synaptic weights using the t-distributed stochastic neighbor embedding (t-SNE) method after training and found that they were well clustered, thereby demonstrating excellent classification ability. Furthermore, to verify the robustness of our model, we trained it on another more realistic dataset (Fashion-MNIST), which also showed good performance. As the learning rules were bio-plausible and based purely on local spike events, our model could be easily applied to neuromorphic hardware for online training and may be helpful for understanding SL information processing at the synaptic level in biological neural systems. [ABSTRACT FROM AUTHOR]

Spatially separated brain areas interact with each other to form networks with coordinated activities, supporting various brain functions. Interaction structures among brain areas have been widely investigated through pairwise measures. However, interactions among multiple (e.g., triple and quadruple) areas cannot be reduced to pairwise interactions. Such higher order interactions (HOIs), e.g., exclusive-or (XOR) operation, are widely implemented in computation systems and are crucial for effective information processing. However, it is currently unclear whether any HOIs are present in large-scale brain functional networks when subjects are executing specific tasks. Here we analyzed functional magnetic resonance imaging (fMRI) data collected from human subjects executing various perceptual, motor, and cognitive tasks. We found that HOI strength in the macroscopic functional networks was very weak for all tasks, suggesting that major brain activities do not rely on HOIs on the macroscopic level at the timescale of hundreds of milliseconds. These weak HOIs during tasks were further investigated with a neural network model activated by external inputs, which suggested that weak pairwise interactions among brain areas organized the system without involving HOIs. Taken together, these results demonstrated the dominance of pairwise interactions in organizing coordinated activities among different brain areas to support various brain functions. • The lack of HOIs at functional brain networks was revealed during various tasks. • The weakly coupled regime of macroscopic brain networks was suggested. • The lack of HOIs provides the methodological foundation for pairwise approaches. [ABSTRACT FROM AUTHOR]

Speech recognition (SR) has been improved significantly by artificial neural networks (ANNs), but ANNs have the drawbacks of biologically implausibility and excessive power consumption because of the nonlocal transfer of real-valued errors and weights. While spiking neural networks (SNNs) have the potential to solve these drawbacks of ANNs due to their efficient spike communication and their natural way to utilize kinds of synaptic plasticity rules found in brain for weight modification. However, existing SNN models for SR either had bad performance, or were trained in biologically implausible ways. In this paper, we present a biologically inspired convolutional SNN model for SR. The network adopts the time-to-first-spike coding scheme for fast and efficient information processing. A biological learning rule, spike-timing-dependent plasticity (STDP), is used to adjust the synaptic weights of convolutional neurons to form receptive fields in an unsupervised way. In the convolutional structure, the strategy of local weight sharing is introduced and could lead to better feature extraction of speech signals than global weight sharing. We first evaluated the SNN model with a linear support vector machine (SVM) on the TIDIGITS dataset and it got the performance of 97.5%, comparable to the best results of ANNs. Deep analysis on network outputs showed that, not only are the output data more linearly separable, but they also have fewer dimensions and become sparse. To further confirm the validity of our model, we trained it on a more difficult recognition task based on the TIMIT dataset, and it got a high performance of 93.8%. Moreover, a linear spike-based classifier—tempotron—can also achieve high accuracies very close to that of SVM on both the two tasks. These demonstrate that an STDP-based convolutional SNN model equipped with local weight sharing and temporal coding is capable of solving the SR task accurately and efficiently. [ABSTRACT FROM AUTHOR]

• We propose a joint training framework where the knowledge of ANN is consistently transferred to the SNN during training. • We factorize the parameters of both ANN and SNN and restrict them to have the same singular vectors while learning different singular values. • Extensive experiments are conducted on vision benchmarks, demonstrating that our method reaches SOTA. Emerged as a biology-inspired method, Spiking Neural Networks (SNNs) mimic the spiking nature of brain neurons and have received lots of research attention. SNNs deal with binary spikes as their activation and therefore derive extreme energy efficiency on hardware. However, it also leads to an intrinsic obstacle that training SNNs from scratch requires a re-definition of the firing function for computing gradient. Artificial Neural Networks (ANNs), however, are fully differentiable to be trained with gradient descent. In this paper, we propose a joint training framework of ANN and SNN, in which the ANN can guide the SNN's optimization. This joint framework contains two parts: First, the knowledge inside ANN is distilled to SNN by using multiple branches from the networks. Second, we restrict the parameters of ANN and SNN, where they share partial parameters and learn different singular weights. Extensive experiments over several widely used network structures show that our method consistently outperforms many other state-of-the-art training methods. For example, on the CIFAR100 classification task, the spiking ResNet-18 model trained by our method can reach to 77.39 % top-1 accuracy with only 4 time steps. [ABSTRACT FROM AUTHOR]