Search

PURPOSE: This study uses deep neural network-generated rim-to-disc area ratio (RADAR) measurements and the disc damage likelihood scale (DDLS) to measure the rate of optic disc rim loss in a large cohort of glaucoma patients. METHODS: A deep neural network was used to calculate RADAR and DDLS for each optic disc photograph (ODP). Patient demographics, diagnosis, intraocular pressure (IOP), and mean deviation (MD) from perimetry were analyzed as risk factors for faster progression of RADAR. Receiver operating characteristic (ROC) curves were used to compare RADAR and DDLS in their utility to distinguish glaucoma from glaucoma suspect (GS) and for detecting glaucoma progression. RESULTS: A total of 13,679 ODPs with evidence of glaucomatous optic nerve damage from 4106 eyes of 2407 patients with glaucoma or GS were included. Of these eyes, 3264 (79.5%) had a diagnosis of glaucoma, and 842 (20.5%) eyes were GS. Mean ± SD baseline RADAR of GS and glaucoma were 0.67 ± 0.13 and 0.57 ± 0.18, respectively (P < 0.001). Older age, greater IOP fluctuation, baseline MD, right eye, and diagnosis of secondary open-angle glaucoma were associated with slope of RADAR. The mean baseline DDLS of GS and glaucoma were 3.78 and 4.39, respectively. Both RADAR and DDLS showed a less steep slope in advanced glaucoma. In glaucoma, the change of RADAR and DDLS correlated with the corresponding change in MD. RADAR and DDLS had a similar ability to discriminate glaucoma from GS and detect disease progression. Area under the ROC curve of RADAR and DDLS was 0.658 and 0.648. CONCLUSIONS: Automated calculation of RADAR and DDLS with a neural network can be used to evaluate the extent and long-term rate of optic disc rim loss and is further evidence of long-term nerve fiber loss in treated patients with glaucoma. TRANSLATIONAL RELEVANCE: Our study provides a large clinic-based experience for RADAR and DDLS measurements in GS and glaucoma with a neural network.

Abstract: A real-time capable core ICRF heating model on NSTX and WEST is developed. The model is based on two nonlinear regression algorithms, the random forest ensemble of decision trees and the multilayer perceptron neural network. The algorithms are trained on TORIC ICRF spectrum solver simulations of the expected flat-top operation scenarios in NSTX and WEST assuming Maxwellian plasmas. The surrogate models are shown to successfully capture the multi-species core ICRF power absorption predicted by the original model for the high harmonic fast wave and the ion cyclotron minority heating schemes while reducing the computational time by six orders of magnitude. Although these models can be expanded, the achieved regression scoring, computational efficiency and increased model robustness suggest these strategies can be implemented into integrated modeling frameworks for real-time control applications.

Objective. Physiological data are often low quality and thereby compromises the effectiveness of related health monitoring. The primary goal of this study is to develop a robust foundation model that can effectively handle low-quality issue in physiological data.Approach. We introduce SiamQuality, a self-supervised learning approach using convolutional neural networks (CNNs) as the backbone. SiamQuality learns to generate similar representations for both high and low quality photoplethysmography (PPG) signals that originate from similar physiological states. We leveraged a substantial dataset of PPG signals from hospitalized intensive care patients, comprised of over 36 million 30 s PPG pairs.Main results. After pre-training the SiamQuality model, it was fine-tuned and tested on six PPG downstream tasks focusing on cardiovascular monitoring. Notably, in tasks such as respiratory rate estimation and atrial fibrillation detection, the models performance exceeded the state-of-the-art by 75% and 5%, respectively. The results highlight the effectiveness of our model across all evaluated tasks, demonstrating significant improvements, especially in applications for heart monitoring on wearable devices.Significance. This study underscores the potential of CNNs as a robust backbone for foundation models tailored to physiological data, emphasizing their capability to maintain performance despite variations in data quality. The success of the SiamQuality model in handling real-world, variable-quality data opens new avenues for the development of more reliable and efficient healthcare monitoring technologies.

This literature review explores the role of attractor neural networks (ANNs) in modeling psychological processes in artificial and biological systems. By synthesizing research from dynamical systems theory, psychology, and computational neuroscience, the review provides an overview of the current understanding of ANN function in memory formation, memory reinforcement, retrieval, and forgetting. Key mathematical foundations of ANNs, including dynamical systems theory and energy functions, are discussed to explain the behavior and stability of these networks. The review also examines empirical applications of ANNs in cognitive processes such as semantic memory and episodic recall, as well as highlighting the hippocampus' role in pattern separation and completion. The review addresses challenges like catastrophic forgetting and noise effects on memory retrieval. By identifying gaps between theoretical models and empirical findings, it highlights the interdisciplinary nature of ANN research and suggests future areas for exploration.

A neural network was developed to detect and characterize bowel obstruction, a common cause of acute abdominal pain. In this retrospective study, 202 CT scans of 165 patients with bowel obstruction from March to June 2022 were included and partitioned into training and test data sets. A multi-channel neural network was trained to segment the gastrointestinal tract, and to predict the diameter and the longitudinal position (longitude) along the gastrointestinal tract using a novel embedding. Its performance was compared to manual segmentations using the Dice score, and to manual measurements of the diameter and longitude using intraclass correlation coefficients (ICC). ROC curves as well as sensitivity and specificity were calculated for diameters above a clinical threshold for obstruction, and for longitudes corresponding to small bowel. In the test data set, Dice score for segmentation of the gastrointestinal tract was 78 ± 8%. ICC between measured and predicted diameters was 0.72, indicating moderate agreement. ICC between measured and predicted longitude was 0.85, indicating good agreement. AUROC was 0.90 for detection of dilated bowel, and was 0.95 and 0.90 for differentiation of the proximal and distal gastrointestinal tract respectively. Overall sensitivity and specificity for dilated small bowel were 0.83 and 0.90. Since obstruction is diagnosed based on the diameter and longitude of the bowel, this neural network and embedding may enable detection and characterization of this important disease on CT.

Identifying and characterizing mobile genetic elements in sequencing data is essential for understanding their diversity, ecology, biotechnological applications and impact on public health. Here we introduce geNomad, a classification and annotation framework that combines information from gene content and a deep neural network to identify sequences of plasmids and viruses. geNomad uses a dataset of more than 200,000 marker protein profiles to provide functional gene annotation and taxonomic assignment of viral genomes. Using a conditional random field model, geNomad also detects proviruses integrated into host genomes with high precision. In benchmarks, geNomad achieved high classification performance for diverse plasmids and viruses (Matthews correlation coefficient of 77.8% and 95.3%, respectively), substantially outperforming other tools. Leveraging geNomad's speed and scalability, we processed over 2.7 trillion base pairs of sequencing data, leading to the discovery of millions of viruses and plasmids that are available through the IMG/VR and IMG/PR databases. geNomad is available at https://portal.nersc.gov/genomad .

Message passing neural networks (MPNNs) on molecular graphs generate continuous and differentiable encodings of small molecules with state-of-the-art performance on protein-ligand complex scoring tasks. Here, we describe the proximity graph network (PGN) package, an open-source toolkit that constructs ligand-receptor graphs based on atom proximity and allows users to rapidly apply and evaluate MPNN architectures for a broad range of tasks. We demonstrate the utility of PGN by introducing benchmarks for affinity and docking score prediction tasks. Graph networks generalize better than fingerprint-based models and perform strongly for the docking score prediction task. Overall, MPNNs with proximity graph data structures augment the prediction of ligand-receptor complex properties when ligand-receptor data are available.

OBJECTIVE: Increased subcutaneous and visceral adipose tissue (SAT/VAT) volume is associated with risk for cardiometabolic diseases. This work aimed to develop and evaluate automated abdominal SAT/VAT segmentation on longitudinal MRI in adults with overweight/obesity using attention-based competitive dense (ACD) 3D U-Net and 3D nnU-Net with full field-of-view volumetric multi-contrast inputs. MATERIALS AND METHODS: 920 adults with overweight/obesity were scanned twice at multiple 3 T MRI scanners and institutions. The first scan was divided into training/validation/testing sets (n = 646/92/182). The second scan from the subjects in the testing set was used to evaluate the generalizability for longitudinal analysis. Segmentation performance was assessed by measuring Dice scores (DICE-SAT, DICE-VAT), false negatives (FN), and false positives (FP). Volume agreement was assessed using the intraclass correlation coefficient (ICC). RESULTS: ACD 3D U-Net achieved rapid ( 0.997) in longitudinal analysis. DISCUSSION: ACD 3D U-Net and 3D nnU-Net can be automated tools to quantify abdominal SAT/VAT volume rapidly, accurately, and longitudinally in adults with overweight/obesity.

Deep learning-based mammographic evaluations could noninvasively assess response to breast cancer chemoprevention. We evaluated change in a convolutional neural network-based breast cancer risk model applied to mammograms among women enrolled in SWOG S0812, which randomly assigned 208 premenopausal high-risk women to receive oral vitamin D3 20 000 IU weekly or placebo for 12 months. We applied the convolutional neural network model to mammograms collected at baseline (n = 109), 12 months (n = 97), and 24 months (n = 67) and compared changes in convolutional neural network-based risk score between treatment groups. Change in convolutional neural network-based risk score was not statistically significantly different between vitamin D and placebo groups at 12 months (0.005 vs 0.002, P = .875) or at 24 months (0.020 vs 0.001, P = .563). The findings are consistent with the primary analysis of S0812, which did not demonstrate statistically significant changes in mammographic density with vitamin D supplementation compared with placebo. There is an ongoing need to evaluate biomarkers of response to novel breast cancer chemopreventive agents.

Adaptation is a universal aspect of neural systems that changes circuit computations to match prevailing inputs. These changes facilitate efficient encoding of sensory inputs while avoiding saturation. Conventional artificial neural networks (ANNs) have limited adaptive capabilities, hindering their ability to reliably predict neural output under dynamic input conditions. Can embedding neural adaptive mechanisms in ANNs improve their performance? To answer this question, we develop a new deep learning model of the retina that incorporates the biophysics of photoreceptor adaptation at the front-end of conventional convolutional neural networks (CNNs). These conventional CNNs build on 'Deep Retina,' a previously developed model of retinal ganglion cell (RGC) activity. CNNs that include this new photoreceptor layer outperform conventional CNN models at predicting male and female primate and rat RGC responses to naturalistic stimuli that include dynamic local intensity changes and large changes in the ambient illumination. These improved predictions result directly from adaptation within the phototransduction cascade. This research underscores the potential of embedding models of neural adaptation in ANNs and using them to determine how neural circuits manage the complexities of encoding natural inputs that are dynamic and span a large range of light levels.

PurposeHigh quality scan prescription that optimally covers the area of interest with scan planes aligned to relevant anatomical structures is crucial for error-free radiologic interpretation. The goal of this project was to develop a machine learning pipeline for oblique scan prescription that could be trained on localizer images and metadata from previously acquired MR exams.MethodsA novel Multislice Rotational Region-based Convolutional Neural Network (MS-R2CNN) architecture was developed. Based on this architecture, models for automated prescription sagittal lumbar spine acquisitions from axial, sagittal, and coronal localizer slices were trained. The automated prescription pipeline was integrated with the scanner console software and evaluated in experiments with healthy volunteers (N = 3) and patients with lower-back pain (N = 20).ResultsExperiments in healthy volunteers demonstrated high accuracy of automated prescription in all subjects. There was good agreement between alignment and coverage of manual and automated prescriptions, as well as consistent views of the lumbar spine at different positions of the subjects within the scanner bore. In patients with lower-back pain, the generated prescription was applied in 18 cases (90% of the total number). None of the cases required major adjustment, while in 11 cases (55%) there were minor manual adjustments to the generated prescription.ConclusionsThis study demonstrates the ability of oriented object detection-based models to be trained to prescribe oblique lumbar spine MRI acquisitions without the need of manual annotation or feature engineering and the feasibility of using machine learning-based pipelines on the scanner for automated prescription of MRI acquisitions.

BACKGROUND: Language in nonmedical data sets is known to transmit human-like biases when used in natural language processing (NLP) algorithms that can reinforce disparities. It is unclear if NLP algorithms of medical notes could lead to similar transmissions of biases. RESEARCH QUESTION: Can we identify implicit bias in clinical notes, and are biases stable across time and geography? STUDY DESIGN AND METHODS: To determine whether different racial and ethnic descriptors are similar contextually to stigmatizing language in ICU notes and whether these relationships are stable across time and geography, we identified notes on critically ill adults admitted to the University of California, San Francisco (UCSF), from 2012 through 2022 and to Beth Israel Deaconess Hospital (BIDMC) from 2001 through 2012. Because word meaning is derived largely from context, we trained unsupervised word-embedding algorithms to measure the similarity (cosine similarity) quantitatively of the context between a racial or ethnic descriptor (eg, African-American) and a stigmatizing target word (eg, nonco-operative) or group of words (violence, passivity, noncompliance, nonadherence). RESULTS: In UCSF notes, Black descriptors were less likely to be similar contextually to violent words compared with White descriptors. Contrastingly, in BIDMC notes, Black descriptors were more likely to be similar contextually to violent words compared with White descriptors. The UCSF data set also showed that Black descriptors were more similar contextually to passivity and noncompliance words compared with Latinx descriptors. INTERPRETATION: Implicit bias is identifiable in ICU notes. Racial and ethnic group descriptors carry different contextual relationships to stigmatizing words, depending on when and where notes were written. Because NLP models seem able to transmit implicit bias from training data, use of NLP algorithms in clinical prediction could reinforce disparities. Active debiasing strategies may be necessary to achieve algorithmic fairness when using language models in clinical research.

To maximize the greenhouse gas (GHG) emission reduction potential of Battery Electric Vehicles (BEVs), it is critical to develop EV dynamic charging management strategies. These strategies leverage the temporal variability in emissions associated with generated electricity to align EV charging with periods of low-carbon power generation. This study introduces a deep neural network tool to enable BEV drivers to make charging sessions align with the availability of cleaner energy resources. This study leverages a Long Short-Term Memory network to forecast individual BEV vehicle miles traveled (VMT) up to two days ahead, using a year-long dataset of driving and charging patterns from 66 California-based BEVs. Based on the predicted VMT, the model then estimates the vehicle's energy needs and the necessity of a charging session. This allows drivers to charge theirvehicles strategically, prioritizing low-carbon electricity periods without risking incomplete journeys. This framework empowers drivers to actively contribute to cleaner electricity consumption with minimal disruption to their daily routines. The tool developed in this project outperforms benchmark models such as recurrent neural networks and autoregressive integrated moving averages, demonstrating its predictive capabilities. To enhance the reliability of predictions, confidence intervals are integrated into the model, ensuring that the model does not disrupt drivers' daily routine trips when skipping non-critical charging events. The potential benefits of the tool are demonstrated by applying it to real-world EV data, finding that if drivers follow the tool’s predictive suggestion, they can reduce overall GHG emissions by 41% without changing their driving patterns. This study also found that even charging in regions with higher carbon-intensity electricity than California can achieve Californian emission levels for EV charging in the short term through strategic management of non-critical charging events. This findingreveals new possibilities for further emissions reduction from EV charging, even before the full transition to a carbon-neutral grid. View the NCST Project Webpage

Surface-enhanced Raman spectroscopy (SERS) is a powerful tool for elucidating the molecular makeup of materials. It possesses the unique characteristics of single-molecule sensitivity and extremely high specificity. However, the true potential of SERS, particularly in capturing the biochemical content of particles, remains underexplored. In this study, we harnessed transformer neural networks to interpret SERS spectra, aiming to discern the amino acid profiles within proteins. By training the network on the SERS profiles of 20 amino acids of human proteins, we explore the feasibility of predicting the predominant proteins within the µL-scale detection volume of SERS. Our results highlight a consistent alignment between the models predictions and the proteins known amino acid compositions, deepening our understanding of the inherent information contained within SERS spectra. For instance, the model achieved low root mean square error (RMSE) scores and minimal deviation in the prediction of amino acid compositions for proteins such as Bovine Serum Albumin (BSA), ACE2 protein, and CD63 antigen. This novel methodology offers a robust avenue not only for protein analytics but also sets a precedent for the broader realm of spectral analyses across diverse material categories. It represents a solid step forward to establishing SERS-based proteomics.

An important difference between brains and deep neural networks is the way they learn. Nervous systems learn online where a stream of noisy data points are presented in a non-independent, identically distributed way. Further, synaptic plasticity in the brain depends only on information local to synapses. Deep networks, on the other hand, typically use non-local learning algorithms and are trained in an offline, non-noisy, independent, identically distributed setting. Understanding how neural networks learn under the same constraints as the brain is an open problem for neuroscience and neuromorphic computing. A standard approach to this problem has yet to be established. In this paper, we propose that discrete graphical models that learn via an online maximum a posteriori learning algorithm could provide such an approach. We implement this kind of model in a neural network called the Sparse Quantized Hopfield Network. We show our model outperforms state-of-the-art neural networks on associative memory tasks, outperforms these networks in online, continual settings, learns efficiently with noisy inputs, and is better than baselines on an episodic memory task.

BackgroundWhole brain delineation (WBD) is utilized in neuroimaging analysis for data preprocessing and deriving whole brain image metrics. Current automated WBD techniques for analysis of preclinical brain MRI data show limited accuracy when images present with significant neuropathology and anatomical deformations, such as that resulting from organophosphate intoxication (OPI) and Alzheimer's Disease (AD), and inadequate generalizability.MethodsA modified 2D U-Net framework was employed for WBD of MRI rodent brains, consisting of 27 convolutional layers, batch normalization, two dropout layers and data augmentation, after training parameter optimization. A total of 265 T2-weighted 7.0 T MRI scans were utilized for the study, including 125 scans of an OPI rat model for neural network training. For testing and validation, 20 OPI rat scans and 120 scans of an AD rat model were utilized. U-Net performance was evaluated using Dice coefficients (DC) and Hausdorff distances (HD) between the U-Net-generated and manually segmented WBDs.ResultsThe U-Net achieved a DC (median[range]) of 0.984[0.936-0.990] and HD of 1.69[1.01-6.78] mm for OPI rat model scans, and a DC (mean[range]) of 0.975[0.898-0.991] and HD of 1.49[0.86-3.89] for the AD rat model scans.Comparison with existing methodsThe proposed approach is fully automated and robust across two rat strains and longitudinal brain changes with a computational speed of 8 seconds/scan, overcoming limitations of manual segmentation.ConclusionsThe modified 2D U-Net provided a fully automated, efficient, and generalizable segmentation approach that achieved high accuracy across two disparate rat models of neurological diseases.

PurposeThis study aimed to compare a new Artificial Intelligence (AI) method to conventional mathematical warping in accurately overlaying peripheral retinal vessels from two different imaging devices: confocal scanning laser ophthalmoscope (cSLO) wide-field images and SLO ultra-wide field images.MethodsImages were captured using the Heidelberg Spectralis 55-degree field-of-view and Optos ultra-wide field. The conventional mathematical warping was performed using Random Sample Consensus-Sample and Consensus sets (RANSAC-SC). This was compared to an AI alignment algorithm based on a one-way forward registration procedure consisting of full Convolutional Neural Networks (CNNs) with Outlier Rejection (OR CNN), as well as an iterative 3D camera pose optimization process (OR CNN + Distortion Correction [DC]). Images were provided in a checkerboard pattern, and peripheral vessels were graded in four quadrants based on alignment to the adjacent box.ResultsA total of 660 boxes were analysed from 55 eyes. Dice scores were compared between the three methods (RANSAC-SC/OR CNN/OR CNN + DC): 0.3341/0.4665/4784 for fold 1-2 and 0.3315/0.4494/4596 for fold 2-1 in composite images. The images composed using the OR CNN + DC have a median rating of 4 (out of 5) versus 2 using RANSAC-SC. The odds of getting a higher grading level are 4.8 times higher using our OR CNN + DC than RANSAC-SC (p

Objective.Artificial neural networks (ANNs) are state-of-the-art tools for modeling and decoding neural activity, but deploying them in closed-loop experiments with tight timing constraints is challenging due to their limited support in existing real-time frameworks. Researchers need a platform that fully supports high-level languages for running ANNs (e.g. Python and Julia) while maintaining support for languages that are critical for low-latency data acquisition and processing (e.g. C and C++).Approach.To address these needs, we introduce the Backend for Realtime Asynchronous Neural Decoding (BRAND). BRAND comprises Linux processes, termednodes, which communicate with each other in agraphvia streams of data. Its asynchronous design allows for acquisition, control, and analysis to be executed in parallel on streams of data that may operate at different timescales. BRAND uses Redis, an in-memory database, to send data between nodes, which enables fast inter-process communication and supports 54 different programming languages. Thus, developers can easily deploy existing ANN models in BRAND with minimal implementation changes.Main results.In our tests, BRAND achieved

This study introduces a neural network-based approach to predict dust emissions, specifically PM2.5 particles, during almond harvesting in California. Using a feedforward neural network (FNN), this research predicted PM2.5 emissions by analyzing key operational parameters of an advanced almond harvester. Preprocessing steps like outlier removal and normalization were employed to refine the dataset for training. The networks architecture was designed with two hidden layers and optimized using tanh activation and MSE loss functions through the Adam algorithm, striking a balance between model complexity and predictive accuracy. The model was trained on extensive field data from an almond pickup system, including variables like brush speed, angular velocity, and harvester forward speed. The results demonstrate a notable predictive accuracy of the FNN model, with a mean squared error (MSE) of 0.02 and a mean absolute error (MAE) of 0.01, indicating high precision in forecasting PM2.5 levels. By integrating machine learning with agricultural practices, this research provides a significant tool for environmental management in almond production, offering a method to reduce harmful emissions while maintaining operational efficiency. This model presents a solution for the almond industry and sets a precedent for applying predictive analytics in sustainable agriculture.

Traditional histochemical staining of post-mortem samples often confronts inferior staining quality due to autolysis caused by delayed fixation of cadaver tissue, and such chemical staining procedures covering large tissue areas demand substantial labor, cost and time. Here, we demonstrate virtual staining of autopsy tissue using a trained neural network to rapidly transform autofluorescence images of label-free autopsy tissue sections into brightfield equivalent images, matching hematoxylin and eosin (H&E) stained versions of the same samples. The trained model can effectively accentuate nuclear, cytoplasmic and extracellular features in new autopsy tissue samples that experienced severe autolysis, such as COVID-19 samples never seen before, where the traditional histochemical staining fails to provide consistent staining quality. This virtual autopsy staining technique provides a rapid and resource-efficient solution to generate artifact-free H&E stains despite severe autolysis and cell death, also reducing labor, cost and infrastructure requirements associated with the standard histochemical staining.

Recent developments in protein design rely on large neural networks with up to 100s of millions of parameters, yet it is unclear which residue dependencies are critical for determining protein function. Here, we show that amino acid preferences at individual residues-without accounting for mutation interactions-explain much and sometimes virtually all of the combinatorial mutation effects across 8 datasets (R2 ~ 78-98%). Hence, few observations (~100 times the number of mutated residues) enable accurate prediction of held-out variant effects (Pearson r > 0.80). We hypothesized that the local structural contexts around a residue could be sufficient to predict mutation preferences, and develop an unsupervised approach termed CoVES (Combinatorial Variant Effects from Structure). Our results suggest that CoVES outperforms not just model-free methods but also similarly to complex models for creating functional and diverse protein variants. CoVES offers an effective alternative to complicated models for identifying functional protein mutations.

Traditional systems for indoor pressure sensing and human activity recognition (HAR) rely on costly, high-resolution mats and computationally intensive neural network-based (NN-based) models that are prone to noise. In contrast, we design a cost-effective and noise-resilient pressure mat system for HAR, leveraging Velostat for intelligent pressure sensing and a novel hyperdimensional computing (HDC) classifier that is lightweight and highly noise resilient. To measure the performance of our system, we collected two datasets, capturing the static and continuous nature of human movements. Our HDC-based classification algorithm shows an accuracy of 93.19%, improving the accuracy by 9.47% over state-of-the-art CNNs, along with an 85% reduction in energy consumption. We propose a new HDC noise-resilient algorithm and analyze the performance of our proposed method in the presence of three different kinds of noise, including memory and communication, input, and sensor noise. Our system is more resilient across all three noise types. Specifically, in the presence of Gaussian noise, we achieve an accuracy of 92.15% (97.51% for static data), representing a 13.19% (8.77%) improvement compared to state-of-the-art CNNs.

The retina and primary visual cortex (V1) both exhibit diverse neural populations sensitive to diverse visual features. Yet it remains unclear how neural populations in each area partition stimulus space to span these features. One possibility is that neural populations are organized into discrete groups of neurons, with each group signaling a particular constellation of features. Alternatively, neurons could be continuously distributed across feature-encoding space. To distinguish these possibilities, we presented a battery of visual stimuli to the mouse retina and V1 while measuring neural responses with multi-electrode arrays. Using machine learning approaches, we developed a manifold embedding technique that captures how neural populations partition feature space and how visual responses correlate with physiological and anatomical properties of individual neurons. We show that retinal populations discretely encode features, while V1 populations provide a more continuous representation. Applying the same analysis approach to convolutional neural networks that model visual processing, we demonstrate that they partition features much more similarly to the retina, indicating they are more like big retinas than little brains.

Recent work in reinforcement learning has leveraged symmetries in the model to improve sample efficiency in training a policy. A commonly used simplifying assumption is that the dynamics and reward both exhibit the same symmetry; however, in many real-world environments, the dynamical model exhibits symmetry independent of the reward model. In this letter, we assume only the dynamics exhibit symmetry, extending the scope of problems in reinforcement learning and learning in control theory to which symmetry techniques can be applied. We use Cartan's moving frame method to introduce a technique for learning dynamics that, by construction, exhibit specified symmetries. Numerical experiments demonstrate that the proposed method learns a more accurate dynamical model.

The prediction of depression is a crucial area of research which makes it one of the top priorities in mental health research as it enables early intervention and can lead to higher success rates in treatment. Self-reported feelings by patients represent a valuable biomarker for predicting depression as they can be expressed in a lower-dimensional network form, offering an advantage in visualizing the interactive characteristics of depression-related feelings. Furthermore, the network form of data expresses high-dimensional data in a compact form, making the data easy to use as input for the machine learning processes. In this study, we applied the graph convolutional network (GCN) algorithm, an effective machine learning tool for handling network data, to predict depression-prone patients using the network form of self-reported log data as the input. We took a data augmentation step to expand the initially small dataset and fed the resulting data into the GCN algorithm, which achieved a high level of accuracy from 86-97% and an F1 (harmonic mean of precision and recall) score of 0.83-0.94 through three experimental cases. In these cases, the ratio of depressive cases varied, and high accuracy and F1 scores were observed in all three cases. This study not only demonstrates the potential for predicting depression-prone patients using self-reported logs as a biomarker in advance, but also shows promise in handling small data sets in the prediction, which is critical given the challenge of obtaining large datasets for biomarker research. The combination of self-reported logs and the GCN algorithm is a promising approach for predicting depression and warrants further investigation.

Computational cognitive models have been used extensively to formalize cognitive processes. Model parameters offer a simple way to quantify individual differences in how humans process information. Similarly, model comparison allows researchers to identify which theories, embedded in different models, provide the best accounts of the data. Cognitive modeling uses statistical tools to quantitatively relate models to data that often rely on computing/estimating the likelihood of the data under the model. However, this likelihood is computationally intractable for a substantial number of models. These relevant models may embody reasonable theories of cognition, but are often under-explored due to the limited range of tools available to relate them to data. We contribute to filling this gap in a simple way using artificial neural networks (ANNs) to map data directly onto model identity and parameters, bypassing the likelihood estimation. We test our instantiation of an ANN as a cognitive model fitting tool on classes of cognitive models with strong inter-trial dependencies (such as reinforcement learning models), which offer unique challenges to most methods. We show that we can adequately perform both parameter estimation and model identification using our ANN approach, including for models that cannot be fit using traditional likelihood-based methods. We further discuss our work in the context of the ongoing research leveraging simulation-based approaches to parameter estimation and model identification, and how these approaches broaden the class of cognitive models researchers can quantitatively investigate.

Passive acoustic monitoring is an essential tool for studying beaked whale populations. This approach can monitor elusive and pelagic species, but the volume of data it generates has overwhelmed researchers ability to quantify species occurrence for effective conservation and management efforts. Automation of data processing is crucial, and machine learning algorithms can rapidly identify species using their sounds. Beaked whale acoustic events, often infrequent and ephemeral, can be missed when co-occurring with signals of more abundant, and acoustically active species that dominate acoustic recordings. Prior efforts on large-scale classification of beaked whale signals with deep neural networks (DNNs) have approached the class as one of many classes, including other odontocete species and anthropogenic signals. That approach tends to miss ephemeral events in favor of more common and dominant classes. Here, we describe a DNN method for improved classification of beaked whale species using an extensive dataset from the western North Atlantic. We demonstrate that by training a DNN to focus on the taxonomic family of beaked whales, ephemeral events were correctly and efficiently identified to species, even with few echolocation clicks. By retrieving ephemeral events, this method can support improved estimation of beaked whale occurrence in regions of high odontocete acoustic activity.

The brain evolved as a sensory-motor machine that drives behavior while being linked to the world through sensors. Human cognition abstracts from these sensory-motor roots but retains intimate ties. The brain's structure reflects this history. How do neural processes at different distances from the sensory and motor surfaces integrate to achieve meaningful and grounded cognition? This is a challenge given the time-continuous and graded nature of sensory-motor processing, which enables continuous online updating. It is also a major challenge to understanding development and autonomous learning, in which the coupling across functional boundaries evolves under the influence of online activation patterns.

In-context learning refers to the ability of a neural network to learn from information presented in its context. While traditional learning in neural networks requires adjusting network weights for every new task, in-context learning operates purely by updating internal activations without needing any updates to network weights. The emergence of this ability in large language models has led to a paradigm shift in machine learning and has forced researchers to reconceptualize how they think about learning in neural networks. Looking beyond language models, we can find in-context learning in many computational models relevant to cognitive science, including those that emerge from meta-learning.

Deep Learning is a neural network approach where a network of multiple layers is trained to process complex data. Applications for deep learning include recognizing complex patterns in pictures, text, and sounds to, in some cases, produce insights into how human process information. Deep learning has garnered widescale interest, fostered by advances in, for instance, natural language processing and the release of innovative tools like Chat GPT. But does deep learning have implications for theory in the cognitive sciences; that is, is deep learning the answer for understanding human cognitive dynamics?

Neuroimaging, especially through Functional Magnetic Resonance Imaging (fMRI), plays a pivotal role in understanding brain activity by leveraging blood-oxygen level dependent (BOLD) signals to estimate neural activities across the brain. The interpretation of these signals through functional connectivity (FC) matrices facilitates the application of Graph Neural Networks (GNN) for analyzing brain network structures, offering insights into both normal and abnormal brain functions. Despite the potential of centralized learning methods in this domain, challenges related to data privacy and the feasibility of sharing sensitive medical datasets across institutions limit their application. This study introduces the Federated Graph Learning Framework for Brain Connectome (FGLBC), addressing these concerns. This novel approach enables the collaborative training of GNN models across multiple entities, such as hospitals, without compromising data privacy. The FGLBC framework implements a privacy-preserving local GNN training (PPGT) algorithm that incorporates Differential Privacy (DP) to safeguard sensitive information during model training. Furthermore, we introduce a unique similarity-weighted aggregation (SWA) algorithm that enhances the aggregation process, thereby boosting the global model's utility and performance. Our comprehensive evaluation across benchmark datasets demonstrates that the FGLBC not only preserves user privacy but also achieves or surpasses the performance of existing methods.

Electroencephalogram (EEG) signals are physiological indicators of brain activity, offering the advantage of high temporal resolution for capturing subtle emotional changes and providing rich information for emotion recognition. However, extracting effective features from EEG data with a low signal-to-noise ratio poses a significant challenge that hinders progress in this research field. To address this issue, we propose a multi-view time-frequency contrastive learning framework called MV-TFCL to enhance the information representation capability of EEG signals from multiple perspectives. Firstly, we introduce a recursive neural network based on multi-scale time-frequency consistency, which integrates global semantic information across different scales through gated units. To our knowledge, this is the first proposal of the theory of multi-scale time-frequency consistency applied in emotion recognition research. Subsequently, we design a tree-structured time-frequency encoder to capture local semantic information within the time-frequency domain. Finally, we incorporate semantic consistency constraints from both global and local perspectives to learn more generalizable and robust features. Extensive experimental results on two publicly available datasets demonstrate the effectiveness and superiority of our proposed method.

Here we used deep neural network + hidden Markov model (DNN+HMM) to provide a computational account for the relationship among holistic processing (HP), face scanning pattern and face recognition performance. The model accounted for the positive associations between HP and eyes-focused face scanning pattern/face recognition performance observed in the literature regardless of the version of the composite task used to measure HP. Interestingly, we observed a quadratic relationship between HP and face scanning pattern, where models being highly eyes-focused or highly nose-focused had lower HP. By inspecting fixation locations and associated attention window size in the model and XAI methods, we found that the eyes- and nose-focused models both developed local and holistic internal representations during training, and their difference was in the temporal dynamics of how these representations were used. Our findings demonstrated how computational modeling could unravel the mechanisms underlying cognition not readily observable in human data.

This paper introduces a novel perspective to significantly mitigate catastrophic forgetting in continuous learning (CL), which emphasizes models' capacity to preserve existing knowledge and assimilate new information. Current replay-based methods treat every task and data sample equally and thus can not fully exploit the potential of the replay buffer.In response, we propose COgnitive REplay (CORE), which draws inspiration from human cognitive review processes. CORE includes two key strategies: Adaptive Quantity Allocation and Quality-Focused Data Selection. The former adaptively modulates the replay buffer allocation for each task based on its forgetting rate, while the latter guarantees the inclusion of representative data that best encapsulates the characteristics of each task within the buffer. Our approach achieves an average accuracy of 37.95\% on split-CIFAR10, surpassing the best baseline method by 6.52\%. Additionally, it significantly enhances the accuracy of the poorest-performing task by 6.30\% compared to the top baseline. Code is available at https://github.com/sterzhang/CORE.

Temporal language grounding (TLG) aims to localize query-related events in videos, which explores how to cognize relationships of video content with language descriptions. According to selective visual attention mechanism in cognitive science, people's cognition and understanding of what happens often rely on dynamic foreground information in the video. Nonetheless, background usually predominates the scenes so that query-related visual features and irrelevant ones are confused. Thus, we propose a Foreground Enhanced Network (FEN) to diminish the background effect from two aspects.FEN at first in spatial dimension explicitly models the evolving foreground in video features by removing relatively unchanged background content. Besides, we propose a progressive contrastive sample generation module to gradually learn the differences between the predicted proposal and its elongated proposals that include the former as a portion, thereby distinguishing similar neighborhood frames. Experiments on two common-used datasets show the efficacy of our model.

Children learn powerful internal models of the world around them from a few years of egocentric visual experience. Can such internal models be learned from a child's visual experience with highly generic learning algorithms or do they require strong inductive biases? Recent advances in collecting large-scale, longitudinal, developmentally realistic video datasets and generic self-supervised learning (SSL) algorithms are allowing us to begin to tackle this nature vs. nurture question. However, existing work typically focuses on image-based SSL algorithms and visual capabilities that can be learned from static images (e.g. object recognition), thus ignoring temporal aspects of the world. To close this gap, here we train self-supervised video models on longitudinal, egocentric headcam recordings collected from a child over a two year period in their early development (6-31 months). The resulting models are highly effective at facilitating the learning of action concepts from a small number of labeled examples; they have favorable data size scaling properties; and they display emergent video interpolation capabilities. Video models also learn more robust object representations than image-based models trained with the exact same data. These results suggest that important temporal aspects of a child's internal model of the world may be learnable from their visual experience using highly generic learning algorithms and without strong inductive biases.

Reservoir computing (RC) offers distinct advantages in extracting spatiotemporal information with low training costs by separating recurrent neural networks into a fixed network with recurrent connections. The quality of the fixed network, known as the reservoir, plays a pivotal role in the performance of the RC system. Our work aims to provide a unified synaptic development framework for RC, constructing a more biologically plausible reservoir to model and understand the neural networks development within the human brain. In this paper, we propose an Autonomous Synaptic Development Reservoir Computing model (ASD-RC) based on an adaptive network of phase oscillators. The reservoir autonomously adjusts the distribution of connection weights in response to external stimuli, forming a task-specific structure. Through experiments and theoretical analyses, we demonstrate that ASD-RC can emulate various synaptic development rules of biological neural networks in \textit{vivo}, including the Hebbian rule and STDP. Furthermore, experiments reveal that combining different development rules can enhance performance on prediction tasks compared to using a single development rule, showcasing the emergence and effects of synergistic development that improve information processing capacity.

An important aspect of language comprehension is learning and generalizing complex lexical relations. For instance, having learned that the phrase preserve cucumbers predicts vinegar and that preserve berries predicts dehydrator, one should be able to infer that the novel phrase preserve peppers is more compatible with vinegar, because pepper is more similar to cucumber. We studied the ability to perform such (compositional) generalization in distributional models trained on an artificial corpus with strict semantic regularities. We found that word-encoding models failed to learn the multi-way lexical dependencies. Recurrent neural networks learned those dependencies but struggled to generalize to novel combinations. Only mini GPT-2, a minified version of the Transformer GPT-2, succeeded in both learning and generalization. Because successful generalization in our tasks requires capturing the relationship between a phrase and a word, we argue that mini GPT-2 acquired hierarchical representations that approximate phrase structure. Our results show that, compared to older models, Transformers are architecturally advantaged to perform compositional generalization.

During sentence processing, comprehenders have to maintain a mapping between lexical items and their position in the sentence (syntactic position). We propose a model of morpheme-position binding in working memory, based on models such as 'serial-order-in-a-box' and its SOB-complex-span version. Like those working memory models, our sentence processing version derives a range of attested memory interference effects from the process of item-position binding. We present simulation results capturing similarity-based interference and item-distortion. These two major classes of interference effects have not received a unified account before, and are not fully captured by cue-based retrieval models.

Dual language learners (DLLs) constitute a large portion of the population, but relatively little is known about the best ways in which to assess their vocabulary knowledge. Past research has used both conceptual vocabulary knowledge, assessing whether a child knows a word in either language, as well as total vocabulary knowledge, assessing what words a child knows in each language separately. The present work uses neural networks to predict specific word learning for individual Cantonese-English DLLs. As its input, The model utilizes word2vec embeddings that either represent children's' conceptual word knowledge or total word knowledge. We find that using total word knowledge results in higher predictive accuracy, suggesting that knowing what specific words DLLs know in each of their languages provides the most accurate picture of DLLs' vocabulary knowledge. The present work has many implications for both identification of at-risk individuals and the creation of learning materials for DLL populations.

Decoding naturalistic stimuli from neural recordings is a significant challenge in systems neuroscience, primarily due to the high-dimensional and nonlinear nature of stimulus-response interactions, and is further exacerbated by the limited availability and noisiness of neural data. While contemporary approaches that incorporate generative models, such as Generative Adversarial Networks (GANs), attempt to address these issues by mapping neural responses to latent representations, they do not fully overcome these obstacles. In this work, we present a novel paradigm of differential neural decoding (dicoding) that focuses on the relative changes in response patterns, which not only expands the neural training data quadratically but also inherently denoises it. To determine the corresponding stimulus changes, this method leverages the Euclidean and feature-disentangled properties of the underlying latents through vector arithmetic. As such, we not only effectively exploit the latent space but also achieve semantically meaningful latent offsets in the context of the stimuli, resulting in improved sample efficiency. We trained a decoder to predict changes in latent vectors based on the corresponding changes in neural responses. The absolute latent vector itself was derived by vector addition of the predicted latent change (indicative of stimulus shift) to a reference latent, which was fed to the generator for the reconstruction of the perceived stimulus. Our results show that this geometrically principled approach facilitates more effective reconstruction of naturalistic stimuli from noisy and limited neural data.

Advancements in deep neural networks have led to significant progress in computer vision and natural language processing. These networks, trained on real-world stimuli, develop high-level feature representations of stimuli. It is hypothesized that these representations, stemming from different inputs, should converge into similar conceptual systems, as they reflect various perspectives of the same underlying reality. This paper examines the degree to which different conceptual systems can be aligned in an unsupervised manner, using feature-based representations from deep neural networks. Our investigation centers on the alignment between the image and word representations produced by diverse neural networks, emphasizing those trained via self-supervised learning methods. Subsequently, to probe comparable alignment patterns in human learning, we extend this examination to models trained on developmental headcam data from children. Our findings reveal a more pronounced alignment in models trained through self-supervised learning compared to supervised learning, effectively uncovering higher-level structural connections among categories. However, this alignment was notably absent in models trained with limited developmental headcam data, suggesting more data, more inductive biases, or more supervision are needed to establish alignment from realistic input.

Humans preferentially recall items that are presented in close temporal proximity together -- a phenomenon known as the "temporal contiguity effect". In this study, we investigate whether this phenomenon emerges naturally when training a recurrent neural network with episodic memory on free recall tasks. The model learns to recall items in the order they were presented, consistent with the human contiguity effect. The strength of this effect predicts the performance of individual networks, mirroring experimental findings in humans where stronger contiguity effects predict higher recall performance. The contiguity effect in the model is supported by a neural representation of item index, resembling the `method of loci'. This differs from prominent computational models of human memory, which use a slow decay of past information to guide sequential retrieval. Our findings provide insights into the mechanisms underlying episodic memory and pave the way for future studies of its interactions with other cognitive processes.

Social rewards shape human behavior. During development, a caregiver guides a learner's behavior towards culturally aligned goals and values. How do these behaviors persist and generalize when the caregiver is no longer present, and the learner must continue autonomously? Here, we propose a model of value internalization where social feedback trains an internal social reward (ISR) model that generates internal rewards when social rewards are unavailable. Through empirical simulations, we show that an ISR model prevents agents from unlearning socialized behaviors and enables generalization in out-of-distribution tasks. Incomplete internalization, akin to "reward hacking" on the ISR, is observed when the model is undertrained. Finally, we show that our model internalizes prosocial behavior in a multi-agent environment. Our work provides a framework for understanding how humans acquire and generalize values and offers insights for aligning AI with human values.