Search

Backpropagation (BP), a foundational algorithm for training artificial neural networks, predominates in contemporary deep learning. Although highly successful, it is often considered biologically implausible. A significant limitation arises from the need for precise symmetry between connections in the backward and forward pathways to backpropagate gradient signals accurately, which is not observed in biological brains. Researchers have proposed several algorithms to alleviate this symmetry constraint, such as feedback alignment and direct feedback alignment. However, their divergence from backpropagation dynamics presents challenges, particularly in deeper networks and convolutional layers. Here we introduce the Product Feedback Alignment (PFA) algorithm. Our findings demonstrate that PFA closely approximates BP and achieves comparable performance in deep convolutional networks while avoiding explicit weight symmetry. Our results offer a novel solution to the longstanding weight symmetry problem, leading to more biologically plausible learning in deep convolutional networks compared to earlier methods.

Understanding connections between artificial and biological intelligent systems can reveal fundamental principles of general intelligence. While many artificial intelligence models have a neuroscience counterpart, such connections are largely missing in Transformer models and the self-attention mechanism. Here, we examine the relationship between interacting attention heads and human episodic memory. We focus on induction heads, which contribute to in-context learning in Transformer-based large language models (LLMs). We demonstrate that induction heads are behaviorally, functionally, and mechanistically similar to the contextual maintenance and retrieval (CMR) model of human episodic memory. Our analyses of LLMs pre-trained on extensive text data show that CMR-like heads often emerge in the intermediate and late layers, qualitatively mirroring human memory biases. The ablation of CMR-like heads suggests their causal role in in-context learning. Our findings uncover a parallel between the computational mechanisms of LLMs and human memory, offering valuable insights into both research fields., Comment: NeurIPS 2024

In terms of energy efficiency and computational speed, neuromorphic electronics based on non-volatile memory devices is expected to be one of most promising hardware candidates for future artificial intelligence (AI). However, catastrophic forgetting, networks rapidly overwriting previously learned weights when learning new tasks, remains as a pivotal hurdle in either digital or analog AI chips for unleashing the true power of brain-like computing. To address catastrophic forgetting in the context of online memory storage, a complex synapse model (the Benna-Fusi model) has been proposed recently[1], whose synaptic weight and internal variables evolve following a diffusion dynamics. In this work, by designing a proton transistor with a series of charge-diffusion-controlled storage components, we have experimentally realized the Benna-Fusi artificial complex synapse. The memory consolidation from coupled storage components is revealed by both numerical simulations and experimental observations. Different memory timescales for the complex synapse are engineered by the diffusion length of charge carriers, the capacity and number of coupled storage components. The advantage of the demonstrated complex synapse in both memory capacity and memory consolidation is revealed by neural network simulations of face familiarity detection. Our experimental realization of the complex synapse suggests a promising approach to enhance memory capacity and to enable continual learning., Comment: 6 figures

This study aims to assess the properties and classification of 55 variable stars in Scutum, little studied since their discovery and reported in the Information Bulletin on Variable Stars (IBVS) 985 and update. Using data from previous studies and several astronomical databases, we performed our analysis mainly utilizing a period analysis software and comparing the photometric characteristics of the variables in a Colour-Absolute Magnitude Diagram. For all stars, the variability is confirmed. We discovered new significant results for the period and/or type of 17 variables and highlighted incorrect cross-reference names on astronomical databases for 3 stars. This assessment also identifies 12 cases for which the results from the ASAS-SN Catalog of Variable Stars are systematically not consistent with the original light curves., Comment: 53 Pages, 31 Figures, 7 Tables. arXiv admin note: substantial text overlap with arXiv:2101.06487, arXiv:2110.02354

This study aims to assess the properties and classification of 40 variable stars in Sagittarius, little studied since their discovery and reported in the Information Bulletin on Variable Stars (IBVS) 985 and update. Using data from previous studies and several astronomical databases, we performed our analysis mainly utilizing a period analysis software and comparing the photometric characteristics of the variables in a Colour-Absolute Magnitude Diagram. For all stars, the variability is confirmed. We discovered new significant results for the period and/or type of 15 variables and highlighted incorrect cross-reference names on astronomical databases for 5 stars. This assessment also identifies 9 cases for which results from the ASAS-SN Catalog of Variable Stars are systematically not consistent with the original light curves. A correct identification of NSV 10522 is provided., Comment: 38 Pages, 16 Figures, 7 Tables. arXiv admin note: substantial text overlap with arXiv:2101.06487

Data transmission accounts for significant energy consumption in wireless sensor networks where streaming data is generatedby the sensors. This impedes their use in many settings, including livestock monitoring over large pastures (which formsour target application). We present Ambrosia, a lightweight protocol that utilizes a window-based timeseries forecastingmechanism for data reduction. Ambrosia employs a configurable error threshold to ensure that the accuracy of end applicationsis unaffected by the data transfer reduction. Experimental evaluations using LoRa and BLE on a real livestock monitoringdeployment demonstrate 60% reduction in data transmission and a 2X increase in battery lifetime., Comment: 13 pages, 7 figures, Nature Scientific Reports

Lower atmospheric global dust storms affect the small- and large-scale weather and variability of the whole Martian atmosphere. Analysis of the CO$_2$ density data from the Neutral Gas and Ion Mass Spectrometer instrument (NGIMS) on board NASA's Mars Atmosphere Volatile EvolutioN (MAVEN) spacecraft show a remarkable increase of GW-induced density fluctuations in the thermosphere during the 2018 major dust storm with distinct latitude and local time variability. The mean thermospheric GW activity increases by a factor of two during the storm event. The magnitude of relative density perturbations is around 20% on average and 40% locally. One and a half months later, the GW activity gradually decreases. Enhanced temperature disturbances in the Martian thermosphere can facilitate atmospheric escape. For the first time, we estimate that, for a 20% and 40% GW-induced disturbances, the net increase of Jeans escape flux of hydrogen is a factor of 1.3 and 2, respectively., Comment: Accepted for publication in Geophysical Research Letters, 13 pages, 4 figures

SWELTO -- Space WEather Laboratory in Turin Observatory is a conceptual framework where new ideas for the analysis of space-based and ground-based data are developed and tested. The input data are (but not limited to) remote sensing observations (EUV images of the solar disk, Visible Light coronagraphic images, radio dynamic spectra, etc...), in situ plasma measurements (interplanetary plasma density, velocity, magnetic field, etc...), as well as measurements acquired by local sensors and detectors (radio antenna, fluxgate magnetometer, full-sky cameras, located in OATo). The output products are automatic identification, tracking, and monitoring of solar stationary and dynamic features near the Sun (coronal holes, active regions, coronal mass ejections, etc...), and in the interplanetary medium (shocks, plasmoids, corotating interaction regions, etc...), as well as reconstructions of the interplanetary medium where solar disturbances may propagate from the Sun to the Earth and beyond. These are based both on empirical models and numerical MHD simulations. The aim of SWELTO is not only to test new data analysis methods for future application for Space Weather monitoring and prediction purposes, but also to procure, test and deploy new ground-based instrumentation to monitor the ionospheric and geomagnetic responses to solar activity. Moreover, people involved in SWELTO are active in outreach to disseminate the topics related with Space Weather to students and the general public.

This study aims to assess the properties and classification of 62 variable stars in Cygnus, little studied since their discovery and originally reported in the Information Bulletin on Variable Stars (IBVS) 1302. Using data from previous studies and several astronomical databases, we performed our analysis mainly utilizing a period analysis software and comparing the photometric characteristics of the variables in a Colour-Absolute Magnitude Diagram. For all stars, the variability is confirmed. We discovered new significant results for the period and/or type of 17 variables and highlighted incorrect cross-reference names on astronomical databases for 23 stars. For 3 stars, whose original type was unknown, we propose a new type. We calculated an epoch of a minimum or a maximum for 24 stars; for 3 of them, the epoch has been defined for the first time. This assessment also identifies some cases for which results from the ASAS-SN Catalog of Variable Stars are not consistent with results from Gaia DR2 and/or our analysis., Comment: 59 Pages, 47 Figures, 9 Tables