Search

Spiking neural networks (SNNs) have low power consumption and bio-interpretable characteristics, and are considered to have tremendous potential for energy-efficient computing. However, the exploration of SNNs on image generation tasks remains very limited, and a unified and effective structure for SNN-based generative models has yet to be proposed. In this paper, we explore a novel diffusion model architecture within spiking neural networks. We utilize transformer to replace the commonly used U-net structure in mainstream diffusion models. It can generate higher quality images with relatively lower computational cost and shorter sampling time. It aims to provide an empirical baseline for research of generative models based on SNNs. Experiments on MNIST, Fashion-MNIST, and CIFAR-10 datasets demonstrate that our work is highly competitive compared to existing SNN generative models.

Traditional end-to-end (E2E) training of deep networks necessitates storing intermediate activations for back-propagation, resulting in a large memory footprint on GPUs and restricted model parallelization. As an alternative, greedy local learning partitions the network into gradient-isolated modules and trains supervisely based on local preliminary losses, thereby providing asynchronous and parallel training methods that substantially reduce memory cost. However, empirical experiments reveal that as the number of segmentations of the gradient-isolated module increases, the performance of the local learning scheme degrades substantially, severely limiting its expansibility. To avoid this issue, we theoretically analyze the greedy local learning from the standpoint of information theory and propose a ContSup scheme, which incorporates context supply between isolated modules to compensate for information loss. Experiments on benchmark datasets (i.e. CIFAR, SVHN, STL-10) achieve SOTA results and indicate that our proposed method can significantly improve the performance of greedy local learning with minimal memory and computational overhead, allowing for the boost of the number of isolated modules. Our codes are available at https://github.com/Tab-ct/ContSup., Comment: 9 figures, 12 tables

Ultra-thin (UT) oxide semiconductors are promising candidates for back-end-of-line (BEOL) compatible transistors and monolithic three-dimensional integration. Experimentally, UT indium oxide (In$_2$O$_3$) field-effect transistors (FETs) with thicknesses down to 0.4 nm exhibits extremely high drain current (10000 $\mu$A/$\mu$m) and transconductance (4000 $\mu$S/$\mu$m). Here, we employ the ab initio quantum transport simulation to investigate the performance limit of sub-5-nm gate length (Lg) UT In$_2$O$_3$ FET. Based on the International Technology Roadmap for Semiconductors (ITRS) criteria for high-performance (HP) devices, the scaling limit of UT In$_2$O$_3$ FETs can reach 2 nm in terms of on-state current, delay time, and power dissipation. The wide bandgap nature of UT In$_2$O$_3$ (3.15 eV) renders it a suitable candidate for ITRS low-power (LP) electronics with Lg down to 3 nm. Both the HP and LP UT In$_2$O$_3$ FETs exhibit superior energy-delay products as compared to other common 2D semiconductors such as monolayer MoS2 and MoTe2. Our study unveils the immense promise of UT In$_2$O$_3$ for both HP and LP device applications., Comment: 16 pages, 7 figures

Gate-all-around (GAA) nanowire (NW) field-effect transistor (FET) is a promising device architecture due to its superior gate controllability than that of the conventional FinFET architecture. The significantly higher electron mobility of indium phosphide (InP) NW than silicon NW makes it particularly well-suited for high-performance (HP) electronics applications. In this work, we perform an ab initio quantum transport simulation to investigate the performance limit of sub-5-nm gate length (Lg) GAA InP NW FETs. The GAA InP NW FETs with Lg of 4 nm can meet the International Technology Roadmap for Semiconductors (ITRS) requirements for HP devices from the perspective of on-state current, delay time, and power dissipation. We also investigate the impact of strain on 3-nm-Lg GAA InP NW FETs. The application of tensile strain results in a remarkable increase of over 60% in the on-state current. These results highlight the potential of GAA InP NW FETs for HP applications in the sub-5-nm Lg region., Comment: 22 pages, 7 figures

Spiking Neural Networks (SNNs), as one of the algorithmic models in neuromorphic computing, have gained a great deal of research attention owing to temporal information processing capability, low power consumption, and high biological plausibility. The potential to efficiently extract spatio-temporal features makes it suitable for processing the event streams. However, existing synaptic structures in SNNs are almost full-connections or spatial 2D convolution, neither of which can extract temporal dependencies adequately. In this work, we take inspiration from biological synapses and propose a spatio-temporal synaptic connection SNN (STSC-SNN) model, to enhance the spatio-temporal receptive fields of synaptic connections, thereby establishing temporal dependencies across layers. Concretely, we incorporate temporal convolution and attention mechanisms to implement synaptic filtering and gating functions. We show that endowing synaptic models with temporal dependencies can improve the performance of SNNs on classification tasks. In addition, we investigate the impact of performance vias varied spatial-temporal receptive fields and reevaluate the temporal modules in SNNs. Our approach is tested on neuromorphic datasets, including DVS128 Gesture (gesture recognition), N-MNIST, CIFAR10-DVS (image classification), and SHD (speech digit recognition). The results show that the proposed model outperforms the state-of-the-art accuracy on nearly all datasets.

Spiking Neural Network (SNN) is considered more biologically realistic and power-efficient as it imitates the fundamental mechanism of the human brain. Recently, backpropagation (BP) based SNN learning algorithms that utilize deep learning frameworks have achieved good performance. However, bio-interpretability is partially neglected in those BP-based algorithms. Toward bio-plausible BP-based SNNs, we consider three properties in modeling spike activities: Multiplicity, Adaptability, and Plasticity (MAP). In terms of multiplicity, we propose a Multiple-Spike Pattern (MSP) with multiple spike transmission to strengthen model robustness in discrete time-iteration. To realize adaptability, we adopt Spike Frequency Adaption (SFA) under MSP to decrease spike activities for improved efficiency. For plasticity, we propose a trainable convolutional synapse that models spike response current to enhance the diversity of spiking neurons for temporal feature extraction. The proposed SNN model achieves competitive performances on neuromorphic datasets: N-MNIST and SHD. Furthermore, experimental results demonstrate that the proposed three aspects are significant to iterative robustness, spike efficiency, and temporal feature extraction capability of spike activities. In summary, this work proposes a feasible scheme for bio-inspired spike activities with MAP, offering a new neuromorphic perspective to embed biological characteristics into spiking neural networks.