Search

Communication by rare, binary spikes is a key factor for the energy efficiency of biological brains. However, it is harder to train biologically-inspired spiking neural networks than artificial neural networks. This is puzzling given that theoretical results provide exact mapping algorithms from artificial to spiking neural networks with time-to-first-spike coding. In this paper we analyze in theory and simulation the learning dynamics of time-to-first-spike-networks and identify a specific instance of the vanishing-or-exploding gradient problem. While two choices of spiking neural network mappings solve this problem at initialization, only the one with a constant slope of the neuron membrane potential at threshold guarantees the equivalence of the training trajectory between spiking and artificial neural networks with rectified linear units. For specific image classification architectures comprising feed-forward dense or convolutional layers, we demonstrate that deep spiking neural network models can be effectively trained from scratch on MNIST and Fashion-MNIST datasets, or fine-tuned on large-scale datasets, such as CIFAR10, CIFAR100 and PLACES365, to achieve the exact same performance as that of artificial neural networks, surpassing previous spiking neural networks. Our approach accomplishes high-performance classification with less than 0.3 spikes per neuron, lending itself for an energy-efficient implementation. We also show that fine-tuning spiking neural networks with our robust gradient descent algorithm enables their optimization for hardware implementations with low latency and resilience to noise and quantization. To address challenges of training spiking neural networks (SNNs) at scale, the authors propose a scalable, approximation-free training method for deep SNNs using time-to-first-spike coding. They demonstrate enhanced performance and energy efficiency for neuromorphic hardware. [ABSTRACT FROM AUTHOR]

Visual oddity task was conceived to study universal ethnic-independent analytic intelligence of humans from a perspective of comprehension of spatial concepts. Advancements in artificial intelligence led to important breakthroughs, yet excelling at such abstract tasks remains challenging. Current approaches typically resort to non-biologically-plausible architectures with ever-growing models consuming substantially more energy than the brain. Motivated by the brain’s efficiency and reasoning capabilities, we present a biologically inspired system that receives inputs from synthetic eye movements – reminiscent of saccades, and processes them with neuronal units incorporating dynamics of neocortical neurons. We introduce a procedurally generated visual oddity dataset to train an architecture extending conventional relational networks and our proposed system. We demonstrate that both approaches are capable of abstract problem-solving at high accuracy, and we uncover that both share the same essential underlying mechanism of reasoning in seemingly unrelated aspects of their architectures. Finally, we show that the biologically inspired network achieves superior accuracy, learns faster and requires fewer parameters than the conventional network.Visual oddity tasks delve into the visual analytic intelligence of humans, which remained challenging for artificial neural networks. The authors propose here a model with biologically inspired neural dynamics and synthetic saccadic eye movements with improved efficiency and accuracy in solving the visual oddity tasks. [ABSTRACT FROM AUTHOR]

Spiking neural networks (SNNs) are mimicking computationally powerful biologically inspired models in which neurons communicate through sequences of spikes, regarded here as sparse binary sequences of zeros and ones. In neuroscience it is conjectured that time encoding, where the information is carried by the temporal position of spikes, is playing a crucial role at least in some parts of the brain where estimation of the spiking rate with a large latency cannot take place. Motivated by the efficiency of temporal coding, compared with the widely used rate coding, the goal of this paper is to develop and train an energy-efficient time-coded deep spiking neural network system. To ensure that the similarity among input stimuli is translated into a correlation of the spike sequences, we introduce correlative temporal encoding and extended correlative temporal encoding techniques to map analog input information into input spike patterns. Importantly, we propose an implementation where all multiplications in the system are replaced with at most a few additions. As a more efficient alternative to both rate-coded SNNs and artificial neural networks, such system represents a preferable solution for the implementation of neuromorphic hardware. We consider data classification tasks where input spike patterns are presented to a feed-forward architecture with leaky-integrate-and-fire neurons. The SNN is trained by backpropagation through time with the objective to match sequences of output spikes with those of specifically designed target spike patterns, each corresponding to exactly one class. During inference the target spike pattern with the smallest van Rossum distance from the output spike pattern determines the class. Extensive simulations indicate that the proposed system achieves a classification accuracy at par with that of state-of-the-art machine learning models. [ABSTRACT FROM AUTHOR]

Traditional neural networks require enormous amounts of data to build their complex mappings during a slow training procedure that hinders their abilities for relearning and adapting to new data. Memory-augmented neural networks enhance neural networks with an explicit memory to overcome these issues. Access to this explicit memory, however, occurs via soft read and write operations involving every individual memory entry, resulting in a bottleneck when implemented using the conventional von Neumann computer architecture. To overcome this bottleneck, we propose a robust architecture that employs a computational memory unit as the explicit memory performing analog in-memory computation on high-dimensional (HD) vectors, while closely matching 32-bit software-equivalent accuracy. This is achieved by a content-based attention mechanism that represents unrelated items in the computational memory with uncorrelated HD vectors, whose real-valued components can be readily approximated by binary, or bipolar components. Experimental results demonstrate the efficacy of our approach on few-shot image classification tasks on the Omniglot dataset using more than 256,000 phase-change memory devices. Our approach effectively merges the richness of deep neural network representations with HD computing that paves the way for robust vector-symbolic manipulations applicable in reasoning, fusion, and compression. The implementation of memory-augmented neural networks using conventional computer architectures is challenging due to a large number of read and write operations. Here, Karunaratne, Schmuck et al. propose an architecture that enables analog in-memory computing on high-dimensional vectors at accuracy matching 32-bit software equivalent. [ABSTRACT FROM AUTHOR]

This paper is a contribution for validating a standard method for trace element monitoring based on transplants and analysis of aquatic bryophytes, in the framework of the EC Directive 2000/60. It presents the results of an experiment carried out to assess significant differences in the amount and variability of As, Cd, Co, Cr, Cu, Fe, Mn, Ni, Pb and Zn in three moss species ( Cinclidotus aquaticus, Fontinalis antipyretica, Platyhypnidium riparioides) and two different parts of the moss (whole plant vs apical tips). Mosses were caged in bags made of a plastic net and transplanted for 2 weeks to an irrigation canal impacted by a waste water treatment plant. Trace element concentrations were measured by inductively coupled plasma optical emission spectrometry (ICP-OES) and inductively coupled plasma mass spectrometry (ICP-MS) before and after exposure to the experimental and control sites in five samples. Enrichment factors >>2 were found for Cu, Ni, Mn, Pb and Zn in all moss species, lower in C. aquaticus, intermediate in F. antipyretica and higher in P. riparioides (the species we recommend to use). The analysis of apical tips after exposure instead of the whole plant led to (I) lower concentrations of As, Co, Cr, Fe and Zn in C. aquaticus (−7 to −30 %) and of Fe and Pb (−13, −18 %) in P. riparioides, (II) higher concentrations of Cu, Ni and Zn (+14 to +18 %) in P. riparioides, while (III) no significant difference ( p > 0.05) in F. antipyretica. Data variability after exposure was generally lower in apical tips, especially in C. aquaticus and in F. antipyretica, less in P. riparioides. In the aim of standardizing the moss-bag technique, the analysis of apical tips is recommended. [ABSTRACT FROM AUTHOR]

On tape media, longitudinal position (LPOS) information is normally encoded in dedicated servo bands to allow rapid and accurate positioning of the tape medium and correct data placement on the tape, which is typically several hundred meters long. Robust detection of the longitudinal position of the tape is important to guarantee reliable placement of data as well as rapid navigation to a target tape location. We introduce a method for improving the LPOS symbol detection reliability based on the weighted combining of soft information provided by two independent servo channels. The soft information for each servo channel is generated from metrics computed using a matched-filter detector for the two hypotheses corresponding to the binary LPOS symbols. The weights are derived from a signal-to-distortion ratio measure that is obtained at the output of each servo channel to optimally combine the available information from the two servo channels. [ABSTRACT FROM AUTHOR]