Search

Spiking neural networks (SNNs) possess energy-efficient potential due to event-based computation. However, supervised training of SNNs remains a challenge as spike activities are non-differentiable. Previous SNNs training methods can be generally categorized into two basic classes, i.e., backpropagation-like training methods and plasticity-based learning methods. The former methods are dependent on energy-inefficient real-valued computation and non-local transmission, as also required in artificial neural networks (ANNs), whereas the latter are either considered to be biologically implausible or exhibit poor performance. Hence, biologically plausible (bio-plausible) high-performance supervised learning (SL) methods for SNNs remain deficient. In this paper, we proposed a novel bio-plausible SNN model for SL based on the symmetric spike-timing dependent plasticity (sym-STDP) rule found in neuroscience. By combining the sym-STDP rule with bio-plausible synaptic scaling and intrinsic plasticity of the dynamic threshold, our SNN model implemented SL well and achieved good performance in the benchmark recognition task (MNIST dataset). To reveal the underlying mechanism of our SL model, we visualized both layer-based activities and synaptic weights using the t-distributed stochastic neighbor embedding (t-SNE) method after training and found that they were well clustered, thereby demonstrating excellent classification ability. Furthermore, to verify the robustness of our model, we trained it on another more realistic dataset (Fashion-MNIST), which also showed good performance. As the learning rules were bio-plausible and based purely on local spike events, our model could be easily applied to neuromorphic hardware for online training and may be helpful for understanding SL information processing at the synaptic level in biological neural systems. [ABSTRACT FROM AUTHOR]

Spatially separated brain areas interact with each other to form networks with coordinated activities, supporting various brain functions. Interaction structures among brain areas have been widely investigated through pairwise measures. However, interactions among multiple (e.g., triple and quadruple) areas cannot be reduced to pairwise interactions. Such higher order interactions (HOIs), e.g., exclusive-or (XOR) operation, are widely implemented in computation systems and are crucial for effective information processing. However, it is currently unclear whether any HOIs are present in large-scale brain functional networks when subjects are executing specific tasks. Here we analyzed functional magnetic resonance imaging (fMRI) data collected from human subjects executing various perceptual, motor, and cognitive tasks. We found that HOI strength in the macroscopic functional networks was very weak for all tasks, suggesting that major brain activities do not rely on HOIs on the macroscopic level at the timescale of hundreds of milliseconds. These weak HOIs during tasks were further investigated with a neural network model activated by external inputs, which suggested that weak pairwise interactions among brain areas organized the system without involving HOIs. Taken together, these results demonstrated the dominance of pairwise interactions in organizing coordinated activities among different brain areas to support various brain functions. • The lack of HOIs at functional brain networks was revealed during various tasks. • The weakly coupled regime of macroscopic brain networks was suggested. • The lack of HOIs provides the methodological foundation for pairwise approaches. [ABSTRACT FROM AUTHOR]

• We propose a joint training framework where the knowledge of ANN is consistently transferred to the SNN during training. • We factorize the parameters of both ANN and SNN and restrict them to have the same singular vectors while learning different singular values. • Extensive experiments are conducted on vision benchmarks, demonstrating that our method reaches SOTA. Emerged as a biology-inspired method, Spiking Neural Networks (SNNs) mimic the spiking nature of brain neurons and have received lots of research attention. SNNs deal with binary spikes as their activation and therefore derive extreme energy efficiency on hardware. However, it also leads to an intrinsic obstacle that training SNNs from scratch requires a re-definition of the firing function for computing gradient. Artificial Neural Networks (ANNs), however, are fully differentiable to be trained with gradient descent. In this paper, we propose a joint training framework of ANN and SNN, in which the ANN can guide the SNN's optimization. This joint framework contains two parts: First, the knowledge inside ANN is distilled to SNN by using multiple branches from the networks. Second, we restrict the parameters of ANN and SNN, where they share partial parameters and learn different singular weights. Extensive experiments over several widely used network structures show that our method consistently outperforms many other state-of-the-art training methods. For example, on the CIFAR100 classification task, the spiking ResNet-18 model trained by our method can reach to 77.39 % top-1 accuracy with only 4 time steps. [ABSTRACT FROM AUTHOR]