Search

Artificial Intelligence models encoding biology and chemistry are opening new routes to high-throughput and high-quality in-silico drug development. However, their training increasingly relies on computational scale, with recent protein language models (pLM) training on hundreds of graphical processing units (GPUs). We introduce the BioNeMo Framework to facilitate the training of computational biology and chemistry AI models across hundreds of GPUs. Its modular design allows the integration of individual components, such as data loaders, into existing workflows and is open to community contributions. We detail technical features of the BioNeMo Framework through use cases such as pLM pre-training and fine-tuning. On 256 NVIDIA A100s, BioNeMo Framework trains a three billion parameter BERT-based pLM on over one trillion tokens in 4.2 days. The BioNeMo Framework is open-source and free for everyone to use.

Fault-tolerant protocols enable large and precise quantum algorithms. Many such protocols rely on a feed-forward processing of data, enabled by a hybrid of quantum and classical logic. Representing the control structure of such programs can be a challenge. Here we explore two such fault-tolerant subroutines and analyze the performance of the subroutines using Quantum Intermediate Representation (QIR) as their underlying intermediate representation. First, we look at QIR's ability to leverage the LLVM compiler toolchain to unroll the quantum iteration logic required to perform magic state distillation on the $[[5,1,3]]$ quantum error-correcting code as originally introduced by Bravyi and Kitaev [Phys. Rev. A 71, 022316 (2005)]. This allows us to not only realize the first implementation of a real-time magic state distillation protocol on quantum hardware, but also demonstrate QIR's ability to optimize complex program structures without degrading machine performance. Next, we investigate a different fault-tolerant protocol that was first introduced by Paetznick and Svore [arXiv:1311.1074 (2013)], that reduces the amount of non-Clifford gates needed for a particular algorithm. We look at four different implementations of this two-stage repeat-until-success algorithm to analyze the performance changes as the results of programming choices. We find the QIR offers a viable representation for a compiled high-level program that performs nearly as well as a hand-optimized version written directly in quantum assembly. Both of these results demonstrate QIR's ability to accurately and efficiently expand the complexity of fault-tolerant protocols that can be realized today on quantum hardware.

Many variables of interest in clinical medicine, like disease severity, are recorded using discrete ordinal categories such as normal/mild/moderate/severe. These labels are used to train and evaluate disease severity prediction models. However, ordinal categories represent a simplification of an underlying continuous severity spectrum. Using continuous scores instead of ordinal categories is more sensitive to detecting small changes in disease severity over time. Here, we present a generalized framework that accurately predicts continuously valued variables using only discrete ordinal labels during model development. We found that for three clinical prediction tasks, models that take the ordinal relationship of the training labels into account outperformed conventional multi-class classification models. Particularly the continuous scores generated by ordinal classification and regression models showed a significantly higher correlation with expert rankings of disease severity and lower mean squared errors compared to the multi-class classification models. Furthermore, the use of MC dropout significantly improved the ability of all evaluated deep learning approaches to predict continuously valued scores that truthfully reflect the underlying continuous target variable. We showed that accurate continuously valued predictions can be generated even if the model development only involves discrete ordinal labels. The novel framework has been validated on three different clinical prediction tasks and has proven to bridge the gap between discrete ordinal labels and the underlying continuously valued variables.

(MnNiSi)1-(Fe2Ge)x composition (x=0.34) alloy was prepared by arc melting, crushed, and sieved to approximately <32 microns. They were utilized in examining the possible magnetic and structural changes when exposed to a dosage of a continuous sweeping rate of ~>120 Gy/min and an absorbed dose of 35 kGy of X-ray radiation. This study reports observable trends in X-ray diffraction and magnetic measurements. Magnetization in the magnetization vs. temperature (both heating and cooling) measurements showed an increase from 2.72 emu/g to 4.01 emu/g in the irradiated sample. The Magnetization vs. magnetic field loops exhibited irradiation-induced magnetic hysteresis. The irradiated sample also exhibited an observable change in its coercivity of {\Delta}Hc = 14.7% at 200 K. A maximum entropy change {\Delta}Smag of ~ 11.139 J/kg*K and a Tave peak of 317.5 K was achieved for the pristine sample in comparison to {\Delta}Smag of ~ 11.349 J/kg*K and a Tave peak of 312.5 K for the irradiated sample. These presented results provided deeper insights into tuning the effect of irradiation to the magnetic properties of (MnNiSi)1-(Fe2Ge)x for composition (x=0.34) that can be utilized in a wide range of magnetocaloric applications in high-energy radiation environments. The irradiation applied to the sample did not induce any structural or magnetic phase changes in the selected composition but modified the magnetic properties marginally.

This paper reports on Team Northeastern's Avatar system for telepresence, and our holistic approach to meet the ANA Avatar XPRIZE Final testing task requirements. The system features a dual-arm configuration with hydraulically actuated glove-gripper pair for haptic force feedback. Our proposed Avatar system was evaluated in the ANA Avatar XPRIZE Finals and completed all 10 tasks, scored 14.5 points out of 15.0, and received the 3rd Place Award. We provide the details of improvements over our first generation Avatar, covering manipulation, perception, locomotion, power, network, and controller design. We also extensively discuss the major lessons learned during our participation in the competition., Comment: 7 pages, submitted to IROS 2023

A long-standing goal of machine-learning-based protein engineering is to accelerate the discovery of novel mutations that improve the function of a known protein. We introduce a sampling framework for evolving proteins in silico that supports mixing and matching a variety of unsupervised models, such as protein language models, and supervised models that predict protein function from sequence. By composing these models, we aim to improve our ability to evaluate unseen mutations and constrain search to regions of sequence space likely to contain functional proteins. Our framework achieves this without any model fine-tuning or re-training by constructing a product of experts distribution directly in discrete protein space. Instead of resorting to brute force search or random sampling, which is typical of classic directed evolution, we introduce a fast MCMC sampler that uses gradients to propose promising mutations. We conduct in silico directed evolution experiments on wide fitness landscapes and across a range of different pre-trained unsupervised models, including a 650M parameter protein language model. Our results demonstrate an ability to efficiently discover variants with high evolutionary likelihood as well as estimated activity multiple mutations away from a wild type protein, suggesting our sampler provides a practical and effective new paradigm for machine-learning-based protein engineering., Comment: 31 pages, 8 figures. To appear in the Machine Learning: Science & Technology (ML:S&T) journal. Code is available at https://github.com/pemami4911/ppde. A short version of this work appeared at the NeurIPS 2022 Machine Learning in Structural Biology Workshop

The integration of artificial intelligence into clinical workflows requires reliable and robust models. Repeatability is a key attribute of model robustness. Repeatable models output predictions with low variation during independent tests carried out under similar conditions. During model development and evaluation, much attention is given to classification performance while model repeatability is rarely assessed, leading to the development of models that are unusable in clinical practice. In this work, we evaluate the repeatability of four model types (binary classification, multi-class classification, ordinal classification, and regression) on images that were acquired from the same patient during the same visit. We study the performance of binary, multi-class, ordinal, and regression models on four medical image classification tasks from public and private datasets: knee osteoarthritis, cervical cancer screening, breast density estimation, and retinopathy of prematurity. Repeatability is measured and compared on ResNet and DenseNet architectures. Moreover, we assess the impact of sampling Monte Carlo dropout predictions at test time on classification performance and repeatability. Leveraging Monte Carlo predictions significantly increased repeatability for all tasks on the binary, multi-class, and ordinal models leading to an average reduction of the 95\% limits of agreement by 16% points and of the disagreement rate by 7% points. The classification accuracy improved in most settings along with the repeatability. Our results suggest that beyond about 20 Monte Carlo iterations, there is no further gain in repeatability. In addition to the higher test-retest agreement, Monte Carlo predictions were better calibrated which leads to output probabilities reflecting more accurately the true likelihood of being correctly classified., Comment: arXiv admin note: text overlap with arXiv:2111.06754

The integration of artificial intelligence into clinical workflows requires reliable and robust models. Among the main features of robustness is repeatability. Much attention is given to classification performance without assessing the model repeatability, leading to the development of models that turn out to be unusable in practice. In this work, we evaluate the repeatability of four model types on images from the same patient that were acquired during the same visit. We study the performance of binary, multi-class, ordinal, and regression models on three medical image analysis tasks: cervical cancer screening, breast density estimation, and retinopathy of prematurity classification. Moreover, we assess the impact of sampling Monte Carlo dropout predictions at test time on classification performance and repeatability. Leveraging Monte Carlo predictions significantly increased repeatability for all tasks on the binary, multi-class, and ordinal models leading to an average reduction of the 95% limits of agreement by 17% points., Comment: Machine Learning for Health (ML4H) at NeurIPS 2021 - Extended Abstract