Search

As of now, more than 60 years have passed since the first determination of protein structures through crystallography, and a significant portion of protein structures can be predicted by computers. This is due to the groundbreaking enhancement in protein structure prediction achieved through neural network training utilizing extensive sequence and structure data. However, substantial challenges persist in structure prediction due to limited data availability, with antibody structure prediction standing as one such challenge. In this paper, we propose a novel neural network architecture that effectively enables structure prediction by reflecting the inherent combinatorial nature involved in protein structure formation. The core idea of this neural network architecture is not solely to track and generate a single structure but rather to form a community of multiple structures and pursue accurate structure prediction by exchanging information among community members. Applying this concept to antibody CDR H3 loop structure prediction resulted in improved structure sampling. Such an approach could be applied in the structural and functional studies of proteins, particularly in exploring various physiological processes mediated by loops. Moreover, it holds potential in addressing various other types of combinatorial structure prediction and design problems. Author summary: In this paper, we propose a new architecture that aims to improve upon protein structure prediction algorithms like AlphaFold or RoseTTAFold by considering the combinatorial nature of protein structure formation. Such an architecture, reflecting the physical principles of nature, is expected to yield beneficial results, particularly in scenarios with limited structure and sequence information. Named ComMat, this architecture does not focus on a single structure but rather on a set of multiple structures—a community—simultaneously. In this process, combinatorial exploration of protein structure is encouraged through information exchange among community members. ComMat is an instance that integrates this idea within the structure module of AlphaFold. Applying ComMat to antibody CDR H3 loop structure prediction yielded outstanding results in structure sampling and prediction when tested on the IgFold set and compared with IgFold and AlphaFold-Multimer. It confirmed that improved structure sampling stems from effective structural exploration. The proposed concept here could potentially be used in the development of various other combinatorial protein structure prediction and protein design methods. [ABSTRACT FROM AUTHOR]

Using the CRISPR-Cas9 system to perform base substitutions at the target site is a typical technique for genome editing with the potential for applications in gene therapy and agricultural productivity. When the CRISPR-Cas9 system uses guide RNA to direct the Cas9 endonuclease to the target site, it may misdirect it to a potential off-target site, resulting in an unintended genome editing. Although several computational methods have been proposed to predict off-target effects, there is still room for improvement in the off-target effect prediction capability. In this paper, we present an effective approach called CRISPR-M with a new encoding scheme and a novel multi-view deep learning model to predict the sgRNA off-target effects for target sites containing indels and mismatches. CRISPR-M takes advantage of convolutional neural networks and bidirectional long short-term memory recurrent neural networks to construct a three-branch network towards multi-views. Compared with existing methods, CRISPR-M demonstrates significant performance advantages running on real-world datasets. Furthermore, experimental analysis of CRISPR-M under multiple metrics reveals its capability to extract features and validates its superiority on sgRNA off-target effect predictions. Author summary: Genome editing using the CRISPR-Cas9 system, particularly base substitutions directed by guide RNA, holds immense potential for applications in gene therapy and agricultural productivity. However, the risk of unintended off-target effects poses a challenge, as misdirection of the Cas9 endonuclease can lead to unintended genome alterations. While computational methods exist for predicting off-target effects, there remains a need for encoding methods with more representation space and deep learning models with generalization capability and the adaptability. This paper introduces CRISPR-M, an innovative approach addressing the limitations of existing methods in predicting off-target effects, especially for target sites with indels and mismatches. CRISPR-M employs a novel encoding scheme and a multi-view deep learning model, combining convolutional neural networks and bidirectional long short-term memory recurrent neural networks. The three-branch network structure enhances the prediction accuracy by considering multiple perspectives. Compared with previous representative methods, CRISPR-M exhibits remarkable performance advantages when applied to real-world datasets. The experimental evaluation of CRISPR-M, assessed by various metrics such as ROC, PRC, GC content and melting temperature, demonstrates its ability to extract meaningful features and establishes its superiority in predicting off-target effects of sgRNA. [ABSTRACT FROM AUTHOR]

In proteomics, a crucial aspect is to identify peptide sequences. De novo sequencing methods have been widely employed to identify peptide sequences, and numerous tools have been proposed over the past two decades. Recently, deep learning approaches have been introduced for de novo sequencing. Previous methods focused on encoding tandem mass spectra and predicting peptide sequences from the first amino acid onwards. However, when predicting peptides using tandem mass spectra, the peptide sequence can be predicted not only from the first amino acid but also from the last amino acid due to the coexistence of b-ion (or a- or c-ion) and y-ion (or x- or z-ion) fragments in the tandem mass spectra. Therefore, it is essential to predict peptide sequences bidirectionally. Our approach, called NovoB, utilizes a Transformer model to predict peptide sequences bidirectionally, starting with both the first and last amino acids. In comparison to Casanovo, our method achieved an improvement of the average peptide-level accuracy rate of approximately 9.8% across all species. Author summary: Understanding the characteristics of data is very important in deep learning methods. When predicting sentences, the transformer model naturally predicts from the first word. For this reason, previous methods predicted peptide sequences from the first amino acid. However, in tandem mass spectra, it is possible to predict peptide sequences bidirectionally. This method shows better results than previous approaches because it can better encode tandem mass spectra. We have demonstrated that good results can be achieved simply by understanding the characteristics of such data and using the model appropriately. We hope that this paper will help various readers improve the performance capabilities of their models. Furthermore, given that bidirectional peptide sequence prediction is crucial in de novo peptide sequence analysis, we hope that this approach will be applied to both existing and future methods utilizing deep learning techniques. [ABSTRACT FROM AUTHOR]

Deep learning methods, trained on the increasing set of available protein 3D structures and sequences, have substantially impacted the protein modeling and design field. These advancements have facilitated the creation of novel proteins, or the optimization of existing ones designed for specific functions, such as binding a target protein. Despite the demonstrated potential of such approaches in designing general protein binders, their application in designing immunotherapeutics remains relatively underexplored. A relevant application is the design of T cell receptors (TCRs). Given the crucial role of T cells in mediating immune responses, redirecting these cells to tumor or infected target cells through the engineering of TCRs has shown promising results in treating diseases, especially cancer. However, the computational design of TCR interactions presents challenges for current physics-based methods, particularly due to the unique natural characteristics of these interfaces, such as low affinity and cross-reactivity. For this reason, in this study, we explored the potential of two structure-based deep learning protein design methods, ProteinMPNN and ESM-IF1, in designing fixed-backbone TCRs for binding target antigenic peptides presented by the MHC through different design scenarios. To evaluate TCR designs, we employed a comprehensive set of sequence- and structure-based metrics, highlighting the benefits of these methods in comparison to classical physics-based design methods and identifying deficiencies for improvement. Author summary: Redirecting T cells to recognize and combat infected or altered cells through the engineering of T cell receptors (TCR) has shown promising results, particularly in immunotherapy against cancer. Leveraging breakthroughs in deep learning within the biosciences, this study explores whether deep learning architectures for general protein design, such as ProteinMPNN and ESM-IF1, can design TCRs to bind target antigenic peptides presented by the MHC (pMHC) based on fixed-backbone structures of the complexes. We utilized a comprehensive set of sequence-based and structure-based metrics, derived from deep learning or classical physics-based principles, to evaluate the generated TCR designs. Our findings highlight the benefits of these methods compared to traditional physics-based design approaches and pinpoint deficiencies for improvement. The results contribute to advancements in the field of TCR design, serving as a guiding framework for the structure-based design of TCRs using deep learning, thereby providing the basis for computational engineering of specific TCRs for T cell-based immunotherapy. [ABSTRACT FROM AUTHOR]

Advances in bioinformatics are primarily due to new algorithms for processing diverse biological data sources. While sophisticated alignment algorithms have been pivotal in analyzing biological sequences, deep learning has substantially transformed bioinformatics, addressing sequence, structure, and functional analyses. However, these methods are incredibly data-hungry, compute-intensive, and hard to interpret. Hyperdimensional computing (HDC) has recently emerged as an exciting alternative. The key idea is that random vectors of high dimensionality can represent concepts such as sequence identity or phylogeny. These vectors can then be combined using simple operators for learning, reasoning, or querying by exploiting the peculiar properties of high-dimensional spaces. Our work reviews and explores HDC's potential for bioinformatics, emphasizing its efficiency, interpretability, and adeptness in handling multimodal and structured data. HDC holds great potential for various omics data searching, biosignal analysis, and health applications. [ABSTRACT FROM AUTHOR]

Zebrafish have become an essential model organism in screening for developmental neurotoxic chemicals and their molecular targets. The success of zebrafish as a screening model is partially due to their physical characteristics including their relatively simple nervous system, rapid development, experimental tractability, and genetic diversity combined with technical advantages that allow for the generation of large amounts of high-dimensional behavioral data. These data are complex and require advanced machine learning and statistical techniques to comprehensively analyze and capture spatiotemporal responses. To accomplish this goal, we have trained semi-supervised deep autoencoders using behavior data from unexposed larval zebrafish to extract quintessential "normal" behavior. Following training, our network was evaluated using data from larvae shown to have significant changes in behavior (using a traditional statistical framework) following exposure to toxicants that include nanomaterials, aromatics, per- and polyfluoroalkyl substances (PFAS), and other environmental contaminants. Further, our model identified new chemicals (Perfluoro-n-octadecanoic acid, 8-Chloroperfluorooctylphosphonic acid, and Nonafluoropentanamide) as capable of inducing abnormal behavior at multiple chemical-concentrations pairs not captured using distance moved alone. Leveraging this deep learning model will allow for better characterization of the different exposure-induced behavioral phenotypes, facilitate improved genetic and neurobehavioral analysis in mechanistic determination studies and provide a robust framework for analyzing complex behaviors found in higher-order model systems. Author summary: We demonstrate that a deep autoencoder using raw behavioral tracking data from zebrafish toxicity screens outperforms conventional statistical methods, resulting in a comprehensive evaluation of behavioral data. Our models can accurately distinguish between normal and abnormal behavior with near-complete overlap with existing statistical approaches, with many chemicals detectable at lower concentrations than with conventional statistical tests; this is a crucial finding for the protection of public health as exposure can lead to a range of neurodevelopmental disorders, including cognitive and other behavioral deficits. Our deep learning models enable the identification of new substances capable of inducing aberrant behavior, and we generated new data to demonstrate the reproducibility of these results. Thus, neurodevelopmentally active chemicals identified by our deep autoencoder models may represent previously undetectable signals of subtle individual response differences. Our method elegantly accounts for the high degree of behavioral variability associated with the genetic diversity found in a highly outbred population, as is typical for zebrafish research, thereby making it applicable to multiple laboratories generating similar data. Utilizing the vast quantities of control data generated during high-throughput screening is one of the most innovative aspects of this study and to our knowledge is the first study to explicitly develop a deep autoencoder model for anomaly detection in large-scale toxicological behavior studies. [ABSTRACT FROM AUTHOR]

Our understanding of bird song, a model system for animal communication and the neurobiology of learning, depends critically on making reliable, validated comparisons between the complex multidimensional syllables that are used in songs. However, most assessments of song similarity are based on human inspection of spectrograms, or computational methods developed from human intuitions. Using a novel automated operant conditioning system, we collected a large corpus of zebra finches' (Taeniopygia guttata) decisions about song syllable similarity. We use this dataset to compare and externally validate similarity algorithms in widely-used publicly available software (Raven, Sound Analysis Pro, Luscinia). Although these methods all perform better than chance, they do not closely emulate the avian assessments. We then introduce a novel deep learning method that can produce perceptual similarity judgements trained on such avian decisions. We find that this new method outperforms the established methods in accuracy and more closely approaches the avian assessments. Inconsistent (hence ambiguous) decisions are a common occurrence in animal behavioural data; we show that a modification of the deep learning training that accommodates these leads to the strongest performance. We argue this approach is the best way to validate methods to compare song similarity, that our dataset can be used to validate novel methods, and that the general approach can easily be extended to other species. Author summary: How do birds hear the differences between their songs? This fascinating question carries implications, since the study of bird song, a model system for the neurobiology of learning and animal communication, depends critically on our ability to assess the similarity of songs. Traditionally, researchers compare sounds by human assessment, or use computational methods based on human intuitions about similarity. However, neither approach is connected to birds' own perception of sound similarity. Here, using a novel automated operant conditioning system, we recorded many thousands of acoustic judgments of similarity from zebra finches, and used this perceptual decision data for the first time to train a deep learning system. The trained system outperforms other computational methods for the task of making the same judgments as birds. This algorithm to compare song similarity, together with the potential of extending the general approach to other species, places the study of bird song on a firmer footing. [ABSTRACT FROM AUTHOR]

Melanoma showcases a complex interplay of genetic alterations and intra- and inter-cellular morphological changes during metastatic transformation. While pivotal, the role of specific mutations in dictating these changes still needs to be fully elucidated. Telomerase promoter mutations (TERTp mutations) significantly influence melanoma's progression, invasiveness, and resistance to various emerging treatments, including chemical inhibitors, telomerase inhibitors, targeted therapy, and immunotherapies. We aim to understand the morphological and phenotypic implications of the two dominant monoallelic TERTp mutations, C228T and C250T, enriched in melanoma metastasis. We developed isogenic clonal cell lines containing the TERTp mutations and utilized dual-color expression reporters steered by the endogenous Telomerase promoter, giving us allelic resolution. This approach allowed us to monitor morpholomic variations induced by these mutations. TERTp mutation-bearing cells exhibited significant morpholome differences from their wild-type counterparts, with increased allele expression patterns, augmented wound-healing rates, and unique spatiotemporal dynamics. Notably, the C250T mutation exerted more pronounced changes in the morpholome than C228T, suggesting a differential role in metastatic potential. Our findings underscore the distinct influence of TERTp mutations on melanoma's cellular architecture and behavior. The C250T mutation may offer a unique morpholomic and systems-driven advantage for metastasis. These insights provide a foundational understanding of how a non-coding mutation in melanoma metastasis affects the system, manifesting in cellular morpholome. Author summary: This work explores how specific genetic changes, known as telomerase promoter mutations, influence the shape and behavior of melanoma cells, a highly metastatic and deadly type of skin cancer. These mutations, C228T and C250T, are common in melanoma and are thought to make the cancer more aggressive and metastatic. Cell lines with these mutations were created, and advanced imaging and deep learning techniques were used to investigate them. We reveal that cells with the C250T mutation showed more significant cellular morphology and movement changes than cells with the C228T mutation or normal cells. This suggests that the C250T mutation might play a bigger role in helping melanoma spread throughout the body. Detailed changes due to these mutations were visualized and understood using cutting-edge technology to analyze the impact on single cells. This work is essential because it helps illustrate how tiny genetic changes can lead to significant differences in metastatic behavior. This could lead to better ways to improve early diagnoses and treatment of metastatic melanoma. The findings provide a clearer picture of how melanoma cells evolve and spread, which could guide future cancer research and therapy development. [ABSTRACT FROM AUTHOR]

All fields of science depend on mathematical models. Occam's razor refers to the principle that good models should exclude parameters beyond those minimally required to describe the systems they represent. This is because redundancy can lead to incorrect estimates of model parameters from data, and thus inaccurate or ambiguous conclusions. Here, we show how deep learning can be powerfully leveraged to apply Occam's razor to model parameters. Our method, FixFit, uses a feedforward deep neural network with a bottleneck layer to characterize and predict the behavior of a given model from its input parameters. FixFit has three major benefits. First, it provides a metric to quantify the original model's degree of complexity. Second, it allows for the unique fitting of data. Third, it provides an unbiased way to discriminate between experimental hypotheses that add value versus those that do not. In three use cases, we demonstrate the broad applicability of this method across scientific domains. To validate the method using a known system, we apply FixFit to recover known composite parameters for the Kepler orbit model and a dynamic model of blood glucose regulation. In the latter, we demonstrate the ability to fit the latent parameters to real data. To illustrate how the method can be applied to less well-established fields, we use it to identify parameters for a multi-scale brain model and reduce the search space for viable candidate mechanisms. Author summary: Mathematical modeling is a pillar of scientific inquiry, bridging the gap between theory and experimental observations. However, in complex systems such as those pervasive in biology (e.g., gene regulatory networks, multi-scale brain interactions, and drug pharmacokinetics), different mechanisms can yield equally plausible explanations for the data. This ambiguity is not due to data limitations but rather to the equations that govern these systems. Both the interpretation and estimation of the parameters of these models are hindered by these intrinsic degeneracies. For this reason, we present a general tool that harnesses the power of deep learning to automatically identify and rectify these ambiguities, allowing a broader range of models to be precisely determined by experimental data. We show, for example, that our method provides novel insights into the features of multi-scale brain dynamics that can be learned from functional neuroimaging. This represents a crucial initial step toward characterizing the mechanistic insights that different types of experiments can provide. [ABSTRACT FROM AUTHOR]

With recent methodological advances in the field of computational protein design, in particular those based on deep learning, there is an increasing need for frameworks that allow for coherent, direct integration of different models and objective functions into the generative design process. Here we demonstrate how evolutionary multiobjective optimization techniques can be adapted to provide such an approach. With the established Non-dominated Sorting Genetic Algorithm II (NSGA-II) as the optimization framework, we use AlphaFold2 and ProteinMPNN confidence metrics to define the objective space, and a mutation operator composed of ESM-1v and ProteinMPNN to rank and then redesign the least favorable positions. Using the two-state design problem of the foldswitching protein RfaH as an in-depth case study, and PapD and calmodulin as examples of higher-dimensional design problems, we show that the evolutionary multiobjective optimization approach leads to significant reduction in the bias and variance in RfaH native sequence recovery, compared to a direct application of ProteinMPNN. We suggest that this improvement is due to three factors: (i) the use of an informative mutation operator that accelerates the sequence space exploration, (ii) the parallel, iterative design process inherent to the genetic algorithm that improves upon the ProteinMPNN autoregressive sequence decoding scheme, and (iii) the explicit approximation of the Pareto front that leads to optimal design candidates representing diverse tradeoff conditions. We anticipate this approach to be readily adaptable to different models and broadly relevant for protein design tasks with complex specifications. Author summary: Proteins are the fundamental building blocks of life, and engineering them has broad applications in medicine and biotechnology. Computational methods that seek to model and predict the protein sequence-structure-function relationship have seen significant advancement from the recent development in deep learning. As more models become available, it remains an open question how to effectively combine them into a coherent computational design approach. One approach is to perform computational design with one model, and filter the design candidates with the others. In this work, we demonstrate how an optimization algorithm inspired by evolution can be adapted to provide an alternative framework that outperforms the post hoc filtering approach, especially for problems with multiple competing design specifications. Such a framework may enable researchers to more effectively integrate the strengths of different modeling approaches to tackle more complex design problems. [ABSTRACT FROM AUTHOR]

Interpreting transcriptome data is an important yet challenging aspect of bioinformatic analysis. While gene set enrichment analysis is a standard tool for interpreting regulatory changes, we utilize deep learning techniques, specifically autoencoder architectures, to learn latent variables that drive transcriptome signals. We investigate whether simple, variational autoencoder (VAE), and beta-weighted VAE are capable of learning reduced representations of transcriptomes that retain critical biological information. We propose a novel VAE that utilizes priors from biological data to direct the network to learn a representation of the transcriptome that is based on understandable biological concepts. After benchmarking five different autoencoder architectures, we found that each succeeded in reducing the transcriptomes to 50 latent dimensions, which captured enough variation for accurate reconstruction. The simple, fully connected autoencoder, performs best across the benchmarks, but lacks the characteristic of having directly interpretable latent dimensions. The beta-weighted, prior-informed VAE implementation is able to solve the benchmarking tasks, and provide semantically accurate latent features equating to biological pathways. This study opens a new direction for differential pathway analysis in transcriptomics with increased transparency and interpretability. Author summary: The ability to measure the human transcriptome has been a critical tool in studying health and disease. However, transcriptome datasets are too large and complex for direct human interpretation. Deep learning techniques such as autoencoders are capable of distilling high-level features from complex data. However, even if deep learning models find patterns, these patterns are not necessarily represented in a way that humans can easily understand. By bringing in the prior knowledge of biological pathways, we have trained the model to "speak the language" of the biologist, and represent complex transcrtomes, in simpler concepts that are already familiar to biologists. We can then apply the tool to compare for example samples from lung cancer cells to healthy cells, and show which biological processes are perturbed. [ABSTRACT FROM AUTHOR]

Amortized simulation-based neural posterior estimation provides a novel machine learning based approach for solving parameter estimation problems. It has been shown to be computationally efficient and able to handle complex models and data sets. Yet, the available approach cannot handle the in experimental studies ubiquitous case of missing data, and might provide incorrect posterior estimates. In this work, we discuss various ways of encoding missing data and integrate them into the training and inference process. We implement the approaches in the BayesFlow methodology, an amortized estimation framework based on invertible neural networks, and evaluate their performance on multiple test problems. We find that an approach in which the data vector is augmented with binary indicators of presence or absence of values performs the most robustly. Indeed, it improved the performance also for the simpler problem of data sets with variable length. Accordingly, we demonstrate that amortized simulation-based inference approaches are applicable even with missing data, and we provide a guideline for their handling, which is relevant for a broad spectrum of applications. Author summary: In biomedical research, mechanistic models describe dynamic processes, yet inferring their underlying parameters can often be challenging. Bayesian statistics provides an established framework for this by integrating prior knowledge with observed data, and naturally enables uncertainty quantification as a distribution of parameter values is returned. However, classical case-based methods for Bayesian inference can be computationally expensive, particularly when the same model needs to be fitted to different data sets. Recently, deep-learning-based approaches have been developed to streamline the inference procedure, allowing the upfront training cost to amortize when applied to multiple data sets. In this manuscript, we explore approaches to extend the setup to data sets with missing data. In summary, an encoding scheme which exploits data augmentation with binary indicators of presence or absence performs the most robustly across different test problems. [ABSTRACT FROM AUTHOR]

Responses to natural stimuli in area V4—a mid-level area of the visual ventral stream—are well predicted by features from convolutional neural networks (CNNs) trained on image classification. This result has been taken as evidence for the functional role of V4 in object classification. However, we currently do not know if and to what extent V4 plays a role in solving other computational objectives. Here, we investigated normative accounts of V4 (and V1 for comparison) by predicting macaque single-neuron responses to natural images from the representations extracted by 23 CNNs trained on different computer vision tasks including semantic, geometric, 2D, and 3D types of tasks. We found that V4 was best predicted by semantic classification features and exhibited high task selectivity, while the choice of task was less consequential to V1 performance. Consistent with traditional characterizations of V4 function that show its high-dimensional tuning to various 2D and 3D stimulus directions, we found that diverse non-semantic tasks explained aspects of V4 function that are not captured by individual semantic tasks. Nevertheless, jointly considering the features of a pair of semantic classification tasks was sufficient to yield one of our top V4 models, solidifying V4's main functional role in semantic processing and suggesting that V4's selectivity to 2D or 3D stimulus properties found by electrophysiologists can result from semantic functional goals. Author summary: The functional role of area V4 in the primate visual cortex has been traditionally studied by measuring tuning properties to simple, parametric stimuli, potentially overlooking other important aspects that could be revealed with richer, natural stimuli. Here, we combine single-cell recordings of macaque V1 and V4 responses to natural images, and deep learning models trained on multiple computer vision tasks. We found that V4 responses are best predicted by representations that are critical to solve semantic tasks like object and scene classification. Moreover, our results suggest that that V4's affinity to different 2D and 3D stimulus properties likely stems from its involvement in semantic processing. Overall, our diverse task-driven modeling approach enriches our understanding of the functional role of visual areas in the brain. [ABSTRACT FROM AUTHOR]

The activities of most enzymes and drugs depend on interactions between proteins and small molecules. Accurate prediction of these interactions could greatly accelerate pharmaceutical and biotechnological research. Current machine learning models designed for this task have a limited ability to generalize beyond the proteins used for training. This limitation is likely due to a lack of information exchange between the protein and the small molecule during the generation of the required numerical representations. Here, we introduce ProSmith, a machine learning framework that employs a multimodal Transformer Network to simultaneously process protein amino acid sequences and small molecule strings in the same input. This approach facilitates the exchange of all relevant information between the two molecule types during the computation of their numerical representations, allowing the model to account for their structural and functional interactions. Our final model combines gradient boosting predictions based on the resulting multimodal Transformer Network with independent predictions based on separate deep learning representations of the proteins and small molecules. The resulting predictions outperform recently published state-of-the-art models for predicting protein-small molecule interactions across three diverse tasks: predicting kinase inhibitions; inferring potential substrates for enzymes; and predicting Michaelis constants KM. The Python code provided can be used to easily implement and improve machine learning predictions involving arbitrary protein-small molecule interactions. Author summary: Understanding how proteins interact with small molecules, such as drugs, is critical to advancing medical, biological, and biotechnological research. Our work introduces ProSmith, a machine learning framework that improves the prediction of protein-small molecule interactions. Protein-small molecule interactions can be predicted by using numerical representations of proteins and small molecules as input to machine learning prediction models. Previous methods typically generated separate numerical representations for the proteins and small molecules without considering their interactions. ProSmith, however, combines both protein sequence and small molecule structural information in the input of a single multimodal Transformer Network to generate a joint numerical representation. Unlike previous methods, this allows for a comprehensive exchange of information between protein and small molecule, capturing the complex relationships and interactions between these two types of molecules. ProSmith successfully predicts several biological interactions, including kinase inhibitions, potential enzyme-substrate pairs, and enzyme kinetic parameters KM. We provide Python code that can be easily adapted to improve predictions for any protein-small molecule interaction. [ABSTRACT FROM AUTHOR]

Tracking body parts in behaving animals, extracting fluorescence signals from cells embedded in deforming tissue, and analyzing cell migration patterns during development all require tracking objects with partially correlated motion. As dataset sizes increase, manual tracking of objects becomes prohibitively inefficient and slow, necessitating automated and semi-automated computational tools. Unfortunately, existing methods for multiple object tracking (MOT) are either developed for specific datasets and hence do not generalize well to other datasets, or require large amounts of training data that are not readily available. This is further exacerbated when tracking fluorescent sources in moving and deforming tissues, where the lack of unique features and sparsely populated images create a challenging environment, especially for modern deep learning techniques. By leveraging technology recently developed for spatial transformer networks, we propose ZephIR, an image registration framework for semi-supervised MOT in 2D and 3D videos. ZephIR can generalize to a wide range of biological systems by incorporating adjustable parameters that encode spatial (sparsity, texture, rigidity) and temporal priors of a given data class. We demonstrate the accuracy and versatility of our approach in a variety of applications, including tracking the body parts of a behaving mouse and neurons in the brain of a freely moving C. elegans. We provide an open-source package along with a web-based graphical user interface that allows users to provide small numbers of annotations to interactively improve tracking results. Author summary: While deep learning with convolutional neural networks has been successfully applied to many multiple object tracking problems, these advances do not immediately generalize to videos of fluorescence reported dynamics in living tissue, where the combination of sparse global distributions and locally dense, homogeneous peaks present a challenging instance of a multiple object tracking problem. Imaging such sparse fluorescent signals is a standard tool for observing neuronal activity in genetically engineered animals, and performing imaging in naturally behaving animals to place that activity in the context of behavior only increases the difficulty of the problem. Thus, this step is typically a significant bottleneck in efforts to understand the relationship between neuronal activity and naturalistic behavior. We build upon recent advances in spatial transformers and differentiable grid sampling to develop a new registration-based approach: ZephIR, a semi-supervised multiple object tracking algorithm with a novel cost function that can incorporate a diverse set of spatio-temporal constraints that can change dynamically during optimization. Local registration of image features enables tracking of keypoints even in sparse imaging conditions, such as fluorescent cellular data, while a spring network incorporates a flexible motion model of the neighboring keypoints without the need for a highly specialized skeletal model. Feature detection can help fine-tune tracking results to match a nearby detected feature in the image or even recover good tracking accuracy in cases where registration clearly fails to produce good gradients. In addition, any amount of new manual labor, whether simply verifying correct results or fixing incorrect ones, can dramatically improve ZephIR's accuracy. Through this workflow, ZephIR achieves state-of-the-art accuracy with minimal manual labor, even on a freely behaving C. elegans, where large deformations present a particularly challenging tracking problem. [ABSTRACT FROM AUTHOR]

In the last few years, Micro-C has shown itself as an improved alternative to Hi-C. It replaced the restriction enzymes in Hi-C assays with micrococcal nuclease (MNase), resulting in capturing nucleosome resolution chromatin interactions. The signal-to-noise improvement of Micro-C allows it to detect more chromatin loops than high-resolution Hi-C. However, compared with massive Hi-C datasets available in the literature, there are only a limited number of Micro-C datasets. To take full advantage of these Hi-C datasets, we present HiC2MicroC, a computational method learning and then predicting Micro-C from Hi-C based on the denoising diffusion probabilistic models (DDPM). We trained our DDPM and other regression models in human foreskin fibroblast (HFFc6) cell line and evaluated these methods in six different cell types at 5-kb and 1-kb resolution. Our evaluations demonstrate that both HiC2MicroC and regression methods can markedly improve Hi-C towards Micro-C, and our DDPM-based HiC2MicroC outperforms regression in various terms. First, HiC2MicroC successfully recovers most of the Micro-C loops even those not detected in Hi-C maps. Second, a majority of the HiC2MicroC-recovered loops anchor CTCF binding sites in a convergent orientation. Third, HiC2MicroC loops share genomic and epigenetic properties with Micro-C loops, including linking promoters and enhancers, and their anchors are enriched for structural proteins (CTCF and cohesin) and histone modifications. Lastly, we find our recovered loops are also consistent with the loops identified from promoter capture Micro-C (PCMicro-C) and Chromatin Interaction Analysis by Paired-End Tag Sequencing (ChIA-PET). Overall, HiC2MicroC is an effective tool for further studying Hi-C data with Micro-C as a template. HiC2MicroC is publicly available at https://github.com/zwang-bioinformatics/HiC2MicroC/. Author summary: The Micro-C assay, an improved variant of Hi-C, can capture genome-wide DNA interactions at nucleosome resolution, which makes it possible to detect more chromatin loops that relate to regulatory elements. However, there are not enough experimental Micro-C data publicly available compared with widely performed Hi-C experiments. Here we use deep learning methods to predict Micro-C from Hi-C. Our evaluation indicates that our method can successfully recover the loops that are detected in Micro-C but not identified in Hi-C. Our method provides an effective way to further take advantage of Hi-C. [ABSTRACT FROM AUTHOR]

The use of deep learning (DL) is steadily gaining traction in scientific challenges such as cancer research. Advances in enhanced data generation, machine learning algorithms, and compute infrastructure have led to an acceleration in the use of deep learning in various domains of cancer research such as drug response problems. In our study, we explored tree-based models to improve the accuracy of a single drug response model and demonstrate that tree-based models such as XGBoost (eXtreme Gradient Boosting) have advantages over deep learning models, such as a convolutional neural network (CNN), for single drug response problems. However, comparing models is not a trivial task. To make training and comparing CNNs and XGBoost more accessible to users, we developed an open-source library called UNNT (A novel Utility for comparing Neural Net and Tree-based models). The case studies, in this manuscript, focus on cancer drug response datasets however the application can be used on datasets from other domains, such as chemistry. Author summary: Advancement in data science, machine learning (ML), and artificial intelligence (AI) methods has enabled extraction of meaningful information from large and complex datasets that has assisted in better understanding, diagnosing, and treating cancer. The understanding of the drug response domain in cancer research has been accelerated with developing ML models to aid in predicting the effectiveness of the drugs based on a specific genomic molecular feature. In this study we developed a novel robust framework called UNNT (A novel Utility for comparing Neural Net and Tree-based models) that trains and compares deep learning method such as CNN and tree-based method such as XGBoost on the user input dataset. We applied this software to single drug response problem in cancer to identify the best performing ML method based on the National Cancer Institute 60 (NCI60) dataset. In addition, we studied the computational aspects of training each of these models where our results show that neither is evidently superior on both CPUs and GPUs while training. This shows that when both models have similar error rates for a dataset the hardware available determines the model choice for training. [ABSTRACT FROM AUTHOR]

Interpretation of extracellular recordings can be challenging due to the long range of electric field. This challenge can be mitigated by estimating the current source density (CSD). Here we introduce kCSD-python, an open Python package implementing Kernel Current Source Density (kCSD) method and related tools to facilitate CSD analysis of experimental data and the interpretation of results. We show how to counter the limitations imposed by noise and assumptions in the method itself. kCSD-python allows CSD estimation for an arbitrary distribution of electrodes in 1D, 2D, and 3D, assuming distributions of sources in tissue, a slice, or in a single cell, and includes a range of diagnostic aids. We demonstrate its features in a Jupyter Notebook tutorial which illustrates a typical analytical workflow and main functionalities useful in validating analysis results. Author summary: Recording electric potential with wires inserted in the brain is a standard method in neuroscientist's arsenal for a century. This extracellular potential reflects electric activity of billions of neural cells processing incoming information and our deepest thoughts. It is difficult to interpret because the effect of every single cell activity propagates in space like a wave in a lake after a stone is thrown. Therefore, every electric recording is a sum of billions of stronger and weaker contributions from close and remote cells, respectively. It is easier to understand the local activity of the brain when we reconstruct the distribution of the sources contributing to the measurements. This is similar to reconstructing the information on where and when stones where thrown into the lake from wiggling of multiple buoys placed on the water surface. The techniques for source reconstruction are called the Current Source Density methods. Early techniques required distributions of electrodes on regular grids. Our kernel Current Source Density method can reconstruct this activity from arbitrary distribution of electrodes and can take into account measurement noise. Here we present our software toolbox, kCSD-python, which implements this method. It comes with a number of tools for efficient estimation, for testing reliability of the results, and with several tutorials showing how to use it. [ABSTRACT FROM AUTHOR]

Inclusion at academic events is facing increased scrutiny as the communities these events serve raise their expectations for who can practically attend. Active efforts in recent years to bring more diversity to academic events have brought progress and created momentum. However, we must reflect on these efforts and determine which underrepresented groups are being disadvantaged. Inclusion at academic events is important to ensure diversity of discourse and opinion, to help build networks, and to avoid academic siloing. All of these contribute to the development of a robust and resilient academic field. We have developed these Ten Simple Rules both to amplify the voices that have been speaking out and to celebrate the progress of many Equity, Diversity, and Inclusivity practices that continue to drive the organisation of academic events. The Rules aim to raise awareness as well as provide actionable suggestions and tools to support these initiatives further. This aims to support academic organisations such as the Deep Learning Indaba, Neuromatch Academy, the IBRO-Simons Computational Neuroscience Imbizo, Biodiversity Information Standards (TDWG), Arabs in Neuroscience, FAIRPoints, and OLS (formerly Open Life Science). This article is a call to action for organisers to reevaluate the impact and reach of their inclusive practices. [ABSTRACT FROM AUTHOR]

Dendritic spines are the seat of most excitatory synapses in the brain, and a cellular structure considered central to learning, memory, and activity-dependent plasticity. The quantification of dendritic spines from light microscopy data is usually performed by humans in a painstaking and error-prone process. We found that human-to-human variability is substantial (inter-rater reliability 82.2±6.4%), raising concerns about the reproducibility of experiments and the validity of using human-annotated 'ground truth' as an evaluation method for computational approaches of spine identification. To address this, we present DeepD3, an open deep learning-based framework to robustly quantify dendritic spines in microscopy data in a fully automated fashion. DeepD3's neural networks have been trained on data from different sources and experimental conditions, annotated and segmented by multiple experts and they offer precise quantification of dendrites and dendritic spines. Importantly, these networks were validated in a number of datasets on varying acquisition modalities, species, anatomical locations and fluorescent indicators. The entire DeepD3 open framework, including the fully segmented training data, a benchmark that multiple experts have annotated, and the DeepD3 model zoo is fully available, addressing the lack of openly available datasets of dendritic spines while offering a ready-to-use, flexible, transparent, and reproducible spine quantification method. Author summary: Our study approaches the automated quantification of a crucial structure in the brain: dendritic spines, critical to many functions of the brain such as learning and memory. Traditionally, scientists manually analyze these spines from microscope images, a process that is slow and prone to errors. We discovered that even among experts results vary substantially, thereby casting doubt on the reliability of such manual analyses. To overcome these challenges, we introduce DeepD3, an open-source tool that uses deep learning to automatically and accurately quantify dendritic spines. DeepD3 is unique because it is trained on a diverse range of data, annotated by multiple experts under various experimental conditions. This ensures its effectiveness across different species, brain regions, and imaging methods. By making DeepD3 freely available, we are providing researchers with a powerful tool that is not only efficient and reliable but also transparent and adaptable to different research needs. With DeepD3, we aim to improve consistency and reproducibility of neuroscience research involving dendritic spines to help disentangle the many complex functions of the brain, such as learning and memory. [ABSTRACT FROM AUTHOR]

Spatial transcriptomic (ST) clustering employs spatial and transcription information to group spots spatially coherent and transcriptionally similar together into the same spatial domain. Graph convolution network (GCN) and graph attention network (GAT), fed with spatial coordinates derived adjacency and transcription profile derived feature matrix are often used to solve the problem. Our proposed method STGIC (spatial transcriptomic clustering with graph and image convolution) is designed for techniques with regular lattices on chips. It utilizes an adaptive graph convolution (AGC) to get high quality pseudo-labels and then resorts to dilated convolution framework (DCF) for virtual image converted from gene expression information and spatial coordinates of spots. The dilation rates and kernel sizes are set appropriately and updating of weight values in the kernels is made to be subject to the spatial distance from the position of corresponding elements to kernel centers so that feature extraction of each spot is better guided by spatial distance to neighbor spots. Self-supervision realized by Kullback–Leibler (KL) divergence, spatial continuity loss and cross entropy calculated among spots with high confidence pseudo-labels make up the training objective of DCF. STGIC attains state-of-the-art (SOTA) clustering performance on the benchmark dataset of 10x Visium human dorsolateral prefrontal cortex (DLPFC). Besides, it's capable of depicting fine structures of other tissues from other species as well as guiding the identification of marker genes. Also, STGIC is expandable to Stereo-seq data with high spatial resolution. Author summary: Spatial transcriptomics detect gene transcription profile with high spatial resolution even to sub-cellular level, which is very helpful to characterize organization and architecture of tissues and find marker genes corresponding to sub-structures. Therefore, it is much more informative than traditional transcriptomics for interpreting the biological processes underlying tissue development and disease progression. Clustering to clarify each spatial domain of tissues in which cells present similar transcription profile and histology is a primary task for analyzing spatial transcriptomics data. To solve the important problem, we propose the method STGIC combining deep learning skills in both graph and image, since we notice that image methods tend to display good performances on some samples in benchmark datasets which existing graph methods are not good at, and vice versa. Besides, we substitute the vanilla form of graph convolution frequently used in this field with the one which can find the optimal order of neighbor involved in feature aggregation adaptively for different graphs. STGIC displays high clustering performance and identifies spatial domains which can be used to depict fine-grained structure and help ascertain marker genes of different tissues from different species sequenced with 10x Visum and Stereo-seq techniques. [ABSTRACT FROM AUTHOR]

Biophysically detailed neural models are a powerful technique to study neural dynamics in health and disease with a growing number of established and openly available models. A major challenge in the use of such models is that parameter inference is an inherently difficult and unsolved problem. Identifying unique parameter distributions that can account for observed neural dynamics, and differences across experimental conditions, is essential to their meaningful use. Recently, simulation based inference (SBI) has been proposed as an approach to perform Bayesian inference to estimate parameters in detailed neural models. SBI overcomes the challenge of not having access to a likelihood function, which has severely limited inference methods in such models, by leveraging advances in deep learning to perform density estimation. While the substantial methodological advancements offered by SBI are promising, their use in large scale biophysically detailed models is challenging and methods for doing so have not been established, particularly when inferring parameters that can account for time series waveforms. We provide guidelines and considerations on how SBI can be applied to estimate time series waveforms in biophysically detailed neural models starting with a simplified example and extending to specific applications to common MEG/EEG waveforms using the the large scale neural modeling framework of the Human Neocortical Neurosolver. Specifically, we describe how to estimate and compare results from example oscillatory and event related potential simulations. We also describe how diagnostics can be used to assess the quality and uniqueness of the posterior estimates. The methods described provide a principled foundation to guide future applications of SBI in a wide variety of applications that use detailed models to study neural dynamics. Author summary: A central problem in computational neural modeling is estimating model parameters that can account for observed activity patterns. While several techniques exist to perform parameter inference in special classes of abstract neural models, there are comparatively few approaches for large scale biophysically detailed neural models. In this work, we describe challenges and solutions in applying a deep learning based statistical framework to estimate parameters in a biophysically detailed large scale neural model, and emphasize the particular difficulties in estimating parameters for time series data. Our example uses a multi-scale model designed to connect human MEG/EEG recordings to the underlying cell and circuit level generators. Our approach allows for crucial insight into how cell-level properties interact to produce measured neural activity, and provides guidelines for diagnosing the quality of the estimate and uniqueness of predictions for different MEG/EEG biomarkers. [ABSTRACT FROM AUTHOR]

Despite decades of research, much is still unknown about the computations carried out in the human face processing network. Recently, deep networks have been proposed as a computational account of human visual processing, but while they provide a good match to neural data throughout visual cortex, they lack interpretability. We introduce a method for interpreting brain activity using a new class of deep generative models, disentangled representation learning models, which learn a low-dimensional latent space that "disentangles" different semantically meaningful dimensions of faces, such as rotation, lighting, or hairstyle, in an unsupervised manner by enforcing statistical independence between dimensions. We find that the majority of our model's learned latent dimensions are interpretable by human raters. Further, these latent dimensions serve as a good encoding model for human fMRI data. We next investigate the representation of different latent dimensions across face-selective voxels. We find that low- and high-level face features are represented in posterior and anterior face-selective regions, respectively, corroborating prior models of human face recognition. Interestingly, though, we find identity-relevant and irrelevant face features across the face processing network. Finally, we provide new insight into the few "entangled" (uninterpretable) dimensions in our model by showing that they match responses in the ventral stream and carry information about facial identity. Disentangled face encoding models provide an exciting alternative to standard "black box" deep learning approaches for modeling and interpreting human brain data. Author summary: We use a class of interpretable deep neural network models, disentangled variational autoencoders (dVAEs), to analyze human fMRI data. We find that a dVAE learns human interpretable dimensions of faces, such as lighting, expression, and hairstyle, and provides as good a match to human fMRI data as matched, non-disentangled models. Our disentangled encoding approach allows us to map different disentangled features to ROI and voxel activity. A decoding analysis confirms that the model separates identity relevant and irrelevant information and reveals that the remaining entangled dimensions contain identity-relevant information. Together these results highlight the use of disentangled models for more interpretable fMRI encoding than standard deep learning models. [ABSTRACT FROM AUTHOR]

Recent advances in computer vision have led to significant progress in the generation of realistic image data, with denoising diffusion probabilistic models proving to be a particularly effective method. In this study, we demonstrate that diffusion models can effectively generate fully-annotated microscopy image data sets through an unsupervised and intuitive approach, using rough sketches of desired structures as the starting point. The proposed pipeline helps to reduce the reliance on manual annotations when training deep learning-based segmentation approaches and enables the segmentation of diverse datasets without the need for human annotations. We demonstrate that segmentation models trained with a small set of synthetic image data reach accuracy levels comparable to those of generalist models trained with a large and diverse collection of manually annotated image data, thereby offering a streamlined and specialized application of segmentation models. Author summary: Modern generative techniques have unlocked the potential to create realistic image data of high quality, prompting the possibility of substituting real image data in segmentation training workflows. Our study highlights the capacity of denoising diffusion probabilistic models to generate high-quality microscopy image data. With adjustments to the generation process, these models can produce realistic fully-annotated image datasets through an intuitive and unsupervised approach. The parameters of the generative pipeline undergo optimization through various evaluations, resulting in synthetic image data that exhibits high PSNR scores. Our practical experiments encompass multiple scenarios, including manual annotations, initial segmentations, and simulations as starting points, demonstrating the versatility of our approach. Importantly, we compare the performance of segmentation models trained on a limited set of synthetic image data with those trained on a vast and diverse collection of manually annotated data, demonstrating the potential of our pipeline to alleviate the reliance on extensive manually annotated datasets. Our approach lays the groundwork for similar applications, thereby promoting the much-needed availability of publicly accessible fully-annotated image datasets and advancing the goal of annotation-free segmentation. [ABSTRACT FROM AUTHOR]

The pathogenic, tropical Leishmania flagellates belong to an early-branching eukaryotic lineage (Kinetoplastida) with several unique features. Unfortunately, they are poorly understood from a molecular biology perspective, making development of mechanistically novel and selective drugs difficult. Here, we explore three functionally critical targeting short linear motif systems as well as their receptors in depth, using a combination of structural modeling, evolutionary sequence divergence and deep learning. Secretory signal peptides, endoplasmic reticulum (ER) retention motifs (KDEL motifs), and autophagy signals (motifs interacting with ATG8 family members) are ancient and essential components of cellular life. Although expected to be conserved amongst the kinetoplastids, we observe that all three systems show a varying degree of divergence from their better studied equivalents in animals, plants, or fungi. We not only describe their behaviour, but also build models that allow the prediction of localization and potential functions for several uncharacterized Leishmania proteins. The unusually Ala/Val-rich secretory signal peptides, endoplasmic reticulum resident proteins ending in Asp-Leu-COOH and atypical ATG8-like proteins are all unique molecular features of kinetoplastid parasites. Several of their critical protein-protein interactions could serve as targets of selective antimicrobial agents against Leishmaniasis due to their systematic divergence from the host. Author summary: Leishmaniasis is a detrimental disease which causes significant decrease in quality of life and in severe cases can be lethal. The disease is caused by the pathogen Leishmania spread by sandfly vectors with vertebrates (including humans) being the primary hosts. Leishmaniasis affects over 10 million people worldwide, with ongoing environmental warming being considered to be helping the disease to spread to new regions. Leishmania pathogenesis is currently poorly understood and to combat the infection and to better control the disease further insights into molecular details of the pathogen are needed. Here we describe three protein sorting systems critical for Leishmania cellular housekeeping and infection processes that are greatly diverged from their host counterparts: secretory signal peptides, endoplasmic reticulum retention motifs, and autophagy signals. We provide state-of-the-art computational models to illuminate these systems. Exploring these major cellular sorting motifs may provide new insights into Leishmania cell biology and aid in developing novel therapeutics. [ABSTRACT FROM AUTHOR]

Learning from past experience is an important adaptation and theoretical models may help to understand its evolution. Many of the existing models study simple phenotypes and do not consider the mechanisms underlying learning while the more complex neural network models often make biologically unrealistic assumptions and rarely consider evolutionary questions. Here, we present a novel way of modelling learning using small neural networks and a simple, biology-inspired learning algorithm. Learning affects only part of the network, and it is governed by the difference between expectations and reality. We use this model to study the evolution of learning under various environmental conditions and different scenarios for the trade-off between exploration (learning) and exploitation (foraging). Efficient learning readily evolves in our individual-based simulations. However, in line with previous studies, the evolution of learning is less likely in relatively constant environments, where genetic adaptation alone can lead to efficient foraging, or in short-lived organisms that cannot afford to spend much of their lifetime on exploration. Once learning does evolve, the characteristics of the learning strategy (i.e. the duration of the learning period and the learning rate) and the average performance after learning are surprisingly little affected by the frequency and/or magnitude of environmental change. In contrast, an organism's lifespan and the distribution of resources in the environment have a strong effect on the evolved learning strategy: a shorter lifespan and a narrow resource distribution reduce the likelihood of learning to evolve. Interestingly, a longer learning period does not always lead to better performance, indicating that the evolved neural networks differ in the effectiveness of learning. Overall, however, we show that a biologically inspired, yet relatively simple, learning mechanism can evolve to lead to an efficient adaptation in a changing environment. Author summary: The ability to learn from experience is an important adaptation. However, it is still unclear how learning is shaped by natural selection. Here, we present a novel way of modelling the evolution of learning using small neural networks and a simple, biology-inspired learning mechanism. Computer simulations reveal that efficient learning readily evolves in this model. However, the evolution of learning is less likely in relatively constant environments (where evolved inborn preferences can guide animal behaviour) and in short-lived organisms (that cannot afford to spend much of their lifetime on learning). If learning does evolve, the evolved learning strategy is strongly affected by the lifespan and environmental richness but surprisingly little by the rate and degree of environmental change. In summary, we show that a simple and biologically plausible mechanism can help understand the evolution of learning and the structure of the evolved learning strategies. [ABSTRACT FROM AUTHOR]

One of the challenges in a viral pandemic is the emergence of novel variants with different phenotypical characteristics. An ability to forecast future viral individuals at the sequence level enables advance preparation by characterizing the sequences and closing vulnerabilities in current preventative and therapeutic methods. In this article, we explore, in the context of a viral pandemic, the problem of generating complete instances of undiscovered viral protein sequences, which have a high likelihood of being discovered in the future using protein language models. Current approaches to training these models fit model parameters to a known sequence set, which does not suit pandemic forecasting as future sequences differ from known sequences in some respects. To address this, we develop a novel method, called PandoGen, to train protein language models towards the pandemic protein forecasting task. PandoGen combines techniques such as synthetic data generation, conditional sequence generation, and reward-based learning, enabling the model to forecast future sequences, with a high propensity to spread. Applying our method to modeling the SARS-CoV-2 Spike protein sequence, we find empirically that our model forecasts twice as many novel sequences with five times the case counts compared to a model that is 30× larger. Our method forecasts unseen lineages months in advance, whereas models 4× and 30× larger forecast almost no new lineages. When trained on data available up to a month before the onset of important Variants of Concern, our method consistently forecasts sequences belonging to those variants within tight sequence budgets. Author summary: Viral protein sequences play a pivotal role in the spread of a pandemic. As the virus evolves, so do the viral proteins, increasing the potency of the virus. Knowledge of future viral protein sequences can be invaluable because it allows us to test the efficacy of preventative and treatment methods against future changes to the virus, and tailor them to such changes early. We attempt to forecast viral proteins ahead of time. Making such predictions is very challenging and complex because the prediction target is a sequence with thousands of positions, and a single mis-predicted sequence position may invalidate the entire prediction. Also, as the virus continues to evolve, the data available to train models becomes obsolete. Addressing these challenges, we create a novel approach to train models of the SARS-CoV-2 Spike protein, that are especially tailored to forecasting future sequences. Models trained using this approach outperform existing approaches in their effectiveness. In addition, our method can train models to forecast important pandemic variants ahead of time. [ABSTRACT FROM AUTHOR]