Search

Large-scale cell microscopy screens are used in drug discovery and molecular biology research to study the effects of millions of chemical and genetic perturbations on cells. To use these images in downstream analysis, we need models that can map each image into a feature space that represents diverse biological phenotypes consistently, in the sense that perturbations with similar biological effects have similar representations. In this work, we present the largest foundation model for cell microscopy data to date, a new 1.9 billion-parameter ViT-G/8 MAE trained on over 8 billion microscopy image crops. Compared to a previous published ViT-L/8 MAE, our new model achieves a 60% improvement in linear separability of genetic perturbations and obtains the best overall performance on whole-genome biological relationship recall and replicate consistency benchmarks. Beyond scaling, we developed two key methods that improve performance: (1) training on a curated and diverse dataset; and, (2) using biologically motivated linear probing tasks to search across each transformer block for the best candidate representation of whole-genome screens. We find that many self-supervised vision transformers, pretrained on either natural or microscopy images, yield significantly more biologically meaningful representations of microscopy images in their intermediate blocks than in their typically used final blocks. More broadly, our approach and results provide insights toward a general strategy for successfully building foundation models for large-scale biological data., Comment: NeurIPS 2024 Foundation Models for Science Workshop (38th Conference on Neural Information Processing Systems). 18 pages, 7 figures

Featurizing microscopy images for use in biological research remains a significant challenge, especially for large-scale experiments spanning millions of images. This work explores the scaling properties of weakly supervised classifiers and self-supervised masked autoencoders (MAEs) when training with increasingly larger model backbones and microscopy datasets. Our results show that ViT-based MAEs outperform weakly supervised classifiers on a variety of tasks, achieving as much as a 11.5% relative improvement when recalling known biological relationships curated from public databases. Additionally, we develop a new channel-agnostic MAE architecture (CA-MAE) that allows for inputting images of different numbers and orders of channels at inference time. We demonstrate that CA-MAEs effectively generalize by inferring and evaluating on a microscopy image dataset (JUMP-CP) generated under different experimental conditions with a different channel structure than our pretraining data (RPI-93M). Our findings motivate continued research into scaling self-supervised learning on microscopy data in order to create powerful foundation models of cellular biology that have the potential to catalyze advancements in drug discovery and beyond., Comment: CVPR 2024 Highlight. arXiv admin note: text overlap with arXiv:2309.16064

Inferring biological relationships from cellular phenotypes in high-content microscopy screens provides significant opportunity and challenge in biological research. Prior results have shown that deep vision models can capture biological signal better than hand-crafted features. This work explores how weakly supervised and self-supervised deep learning approaches scale when training larger models on larger datasets. Our results show that both CNN- and ViT-based masked autoencoders significantly outperform weakly supervised models. At the high-end of our scale, a ViT-L/8 trained on over 3.5-billion unique crops sampled from 95-million microscopy images achieves relative improvements as high as 28% over our best weakly supervised models at inferring known biological relationships curated from public databases., Comment: 4 pages, 4 figures

Obtaining information from a quantum system through a measurement typically disturbs its state. The postmeasurement states for a given measurement, however, are not unique and highly rely on the chosen measurement model, complicating the puzzle of information-disturbance. Two distinct questions are then in order. Firstly, what is the minimum disturbance a measurement may induce? Secondly, when a fixed disturbance occurs, how informative is the possible measurement in the best-case scenario? Here, we propose various approaches to tackle these questions and provide explicit solutions for the set of unbiased binary qubit measurements and postmeasurement state spaces that are equivalent to the image of a unital qubit channel. In particular, we show there are different tradeoff relations between the sharpness of this measurement and the average fidelity of the premeasurement and postmeasurement state spaces as well as the sharpness and quantum resources preserved in the postmeasurement states in terms of coherence and discord-like correlation once the measurement is applied locally., Comment: 10 pages, 2 figures

High-throughput screening techniques are commonly used to obtain large quantities of data in many fields of biology. It is well known that artifacts arising from variability in the technical execution of different experimental batches within such screens confound these observations and can lead to invalid biological conclusions. It is therefore necessary to account for these batch effects when analyzing outcomes. In this paper we describe RxRx1, a biological dataset designed specifically for the systematic study of batch effect correction methods. The dataset consists of 125,510 high-resolution fluorescence microscopy images of human cells under 1,138 genetic perturbations in 51 experimental batches across 4 cell types. Visual inspection of the images alone clearly demonstrates significant batch effects. We propose a classification task designed to evaluate the effectiveness of experimental batch correction methods on these images and examine the performance of a number of correction methods on this task. Our goal in releasing RxRx1 is to encourage the development of effective experimental batch correction methods that generalize well to unseen experimental batches. The dataset can be downloaded at https://rxrx.ai.

Drug repurposing can accelerate the identification of effective compounds for clinical use against SARS-CoV-2, with the advantage of pre-existing clinical safety data and an established supply chain. RNA viruses such as SARS-CoV-2 manipulate cellular pathways and induce reorganization of subcellular structures to support their life cycle. These morphological changes can be quantified using bioimaging techniques. In this work, we developed DEEMD: a computational pipeline using deep neural network models within a multiple instance learning framework, to identify putative treatments effective against SARS-CoV-2 based on morphological analysis of the publicly available RxRx19a dataset. This dataset consists of fluorescence microscopy images of SARS-CoV-2 non-infected cells and infected cells, with and without drug treatment. DEEMD first extracts discriminative morphological features to generate cell morphological profiles from the non-infected and infected cells. These morphological profiles are then used in a statistical model to estimate the applied treatment efficacy on infected cells based on similarities to non-infected cells. DEEMD is capable of localizing infected cells via weak supervision without any expensive pixel-level annotations. DEEMD identifies known SARS-CoV-2 inhibitors, such as Remdesivir and Aloxistatin, supporting the validity of our approach. DEEMD can be explored for use on other emerging viruses and datasets to rapidly identify candidate antiviral treatments in the future}. Our implementation is available online at https://www.github.com/Sadegh-Saberian/DEEMD

Image hash codes are produced by binarizing the embeddings of convolutional neural networks (CNN) trained for either classification or retrieval. While proxy embeddings achieve good performance on both tasks, they are non-trivial to binarize, due to a rotational ambiguity that encourages non-binary embeddings. The use of a fixed set of proxies (weights of the CNN classification layer) is proposed to eliminate this ambiguity, and a procedure to design proxy sets that are nearly optimal for both classification and hashing is introduced. The resulting hash-consistent large margin (HCLM) proxies are shown to encourage saturation of hashing units, thus guaranteeing a small binarization error, while producing highly discriminative hash-codes. A semantic extension (sHCLM), aimed to improve hashing performance in a transfer scenario, is also proposed. Extensive experiments show that sHCLM embeddings achieve significant improvements over state-of-the-art hashing procedures on several small and large datasets, both within and beyond the set of training classes., Comment: Accepted at International Journal of Computer Vision

In this paper we propose a method to build a neural network that is similar to an ensemble of decision trees. We first illustrate how to convert a learned ensemble of decision trees to a single neural network with one hidden layer and an input transformation. We then relax some properties of this network such as thresholds and activation functions to train an approximately equivalent decision tree ensemble. The final model, Hammock, is surprisingly simple: a fully connected two layers neural network where the input is quantized and one-hot encoded. Experiments on large and small datasets show this simple method can achieve performance similar to that of Gradient Boosted Decision Trees.

Internet traffic classification has become more important with rapid growth of current Internet network and online applications. There have been numerous studies on this topic which have led to many different approaches. Most of these approaches use predefined features extracted by an expert in order to classify network traffic. In contrast, in this study, we propose a \emph{deep learning} based approach which integrates both feature extraction and classification phases into one system. Our proposed scheme, called "Deep Packet," can handle both \emph{traffic characterization} in which the network traffic is categorized into major classes (\eg, FTP and P2P) and application identification in which end-user applications (\eg, BitTorrent and Skype) identification is desired. Contrary to most of the current methods, Deep Packet can identify encrypted traffic and also distinguishes between VPN and non-VPN network traffic. After an initial pre-processing phase on data, packets are fed into Deep Packet framework that embeds stacked autoencoder and convolution neural network in order to classify network traffic. Deep packet with CNN as its classification model achieved recall of $0.98$ in application identification task and $0.94$ in traffic categorization task. To the best of our knowledge, Deep Packet outperforms all of the proposed classification methods on UNB ISCX VPN-nonVPN dataset.

The design of complexity-aware cascaded detectors, combining features of very different complexities, is considered. A new cascade design procedure is introduced, by formulating cascade learning as the Lagrangian optimization of a risk that accounts for both accuracy and complexity. A boosting algorithm, denoted as complexity aware cascade training (CompACT), is then derived to solve this optimization. CompACT cascades are shown to seek an optimal trade-off between accuracy and complexity by pushing features of higher complexity to the later cascade stages, where only a few difficult candidate patches remain to be classified. This enables the use of features of vastly different complexities in a single detector. In result, the feature pool can be expanded to features previously impractical for cascade design, such as the responses of a deep convolutional neural network (CNN). This is demonstrated through the design of a pedestrian detector with a pool of features whose complexities span orders of magnitude. The resulting cascade generalizes the combination of a CNN with an object proposal mechanism: rather than a pre-processing stage, CompACT cascades seamlessly integrate CNNs in their stages. This enables state of the art performance on the Caltech and KITTI datasets, at fairly fast speeds.

In this paper we consider the problem of multi-view face detection. While there has been significant research on this problem, current state-of-the-art approaches for this task require annotation of facial landmarks, e.g. TSM [25], or annotation of face poses [28, 22]. They also require training dozens of models to fully capture faces in all orientations, e.g. 22 models in HeadHunter method [22]. In this paper we propose Deep Dense Face Detector (DDFD), a method that does not require pose/landmark annotation and is able to detect faces in a wide range of orientations using a single model based on deep convolutional neural networks. The proposed method has minimal complexity; unlike other recent deep learning object detection methods [9], it does not require additional components such as segmentation, bounding-box regression, or SVM classifiers. Furthermore, we analyzed scores of the proposed face detector for faces in different orientations and found that 1) the proposed method is able to detect faces from different angles and can handle occlusion to some extent, 2) there seems to be a correlation between dis- tribution of positive examples in the training set and scores of the proposed face detector. The latter suggests that the proposed methods performance can be further improved by using better sampling strategies and more sophisticated data augmentation techniques. Evaluations on popular face detection benchmark datasets show that our single-model face detector algorithm has similar or better performance compared to the previous methods, which are more complex and require annotations of either different poses or facial landmarks., Comment: in International Conference on Multimedia Retrieval 2015 (ICMR)

The low-frequency acoustic-like modes in a pair plasma (electron-positron or pair-ion) is studied by employing a kinetic theory model based on the Vlasov and Poisson's equation with emphasizing the Tsallis's nonextensive statistics. The possibility of the acoustic-like modes and their properties in both fully symmetric and temperature-asymmetric cases are examined by studying the dispersion relation, Landau damping and instability of modes. The resultant dispersion relation in this study is compatible with the acoustic branch of the experimental data [W. Oohara, D. Date, and R. Hatakeyama, Phys. Rev. Lett. 95, 175003 (2005)], in which the electrostatic waves have been examined in a pure pair-ion plasma. Particularly, our study reveals that the occurrence of growing or damped acoustic-like modes depends strongly on the nonextensivity of the system as a measure for describing the long-range Coulombic interactions and correlations in the plasma. The mechanism that leads to the unstable modes lies in the heart of the nonextensive formalism yet, the mechanism of damping is the same developed by Landau. Furthermore, the solutions of acoustic-like waves in an equilibrium Maxwellian pair plasma are recovered in the extensive limit ($q\rightarrow1$), where the acoustic modes have only the Landau damping and no growth., Comment: Accepted for publication in Astrophysics and Space Science

Propagation of ion-acoustic solitary waves (IASWs) is studied using the hydrodynamic equations coupled with the Poisson equation in a warm plasma consisting of adiabatic ions and superthermal (Kappa distributed) electrons in presence of an electron-beam component. In the linear limit, the dispersion relation for ion-acoustic (IA) waves is obtained by linearizing of basic equations. On the other hand, in the nonlinear analysis, an energy-balance like equation involving Sagdeev's pseudo-potential is derived in order to investigate arbitrary amplitude IA solitons. The Mach number range is determined in which, propagation and characteristics of IA solitons are analyzed both parametrically and numerically. The variation of amplitude and width of electrostatic (ES) excitations as a result of superthermality (via) and also the physical parameters (ion temperature, soliton speed, electron-beam density and electron-beam velocity) are examined. A typical interaction between IASWs and the electron-beam in plasma is confirmed. It is also found that ES solitons with both negative polarity (rarefactive solitons) and positive polarity (compressive solitons) can coexist in plasma. Furthermore, the small but finite amplitude limit of IASWs is investigated and it is showed that this limit is restricted to negative potential pulses., Comment: 27 pages, 12 figures

Propagation of large amplitude ion-acoustic solitary waves (IASWs) in a fully relativistic plasma consisting of cold ions and ultrarelativistic hot electrons and positrons is investigated using the Sagdeev's pseudopotential method in a relativistic hydrodynamics model. Effects of streaming speed of plasma fluid, thermal energy, positron density and positron temperature on large amplitude IASWs are studied by analysis of the pseudopotential structure. It is found that in regions that the streaming speed of plasma fluid is larger than that of solitary wave, by increasing the streaming speed of plasma fluid the depth and width of potential well increases and resulting in narrower solitons with larger amplitude. This behavior is opposite for the case where the streaming speed of plasma fluid is smaller than that of solitary wave. On the other hand, increase of the thermal energy results in wider solitons with smaller amplitude, because the depth and width of potential well decreases in that case. Additionally, the maximum soliton amplitude increases and the width becomes narrower as a result of increase in positron density. It is shown that varying the positron temperature does not have considerable effect on the width and amplitude of IASWs. The existence of stationary soliton-like arbitary amplitude waves is also predicted in fully relativistic electron-positron-ion (EPI) plasmas. Effects of streaming speed of plasma fluid, thermal energy, positron density and positron temperature on these kinds of solitons are the same as that for large amplitude IASWs., Comment: 17 pages, 14 figures, Accepted: Canadian Journal of Physics, Manuscript Number 10-486

Arbitrary amplitude ion-acoustic solitary waves (IASWs) are studied using Sagdeev-Potential approach in electron-positron-ion plasma with ultra-relativistic or non-relativistic degenerate electrons and positrons and the matching criteria of existence of such solitary waves are numerically investigated. It has been shown that the relativistic degeneracy of electrons and positrons has significant effects on the amplitude and the Mach-number range of IASWs. Also it is remarked that only compressive IASWs can propagate in both non-relativistic and ultra-relativistic degenerate plasmas., Comment: Published in Journal Plasma and Fusion Research