Search

The analysis of live-cell single-molecule imaging experiments can reveal valuable information about the heterogeneity of transport processes and interactions between cell components. These characteristics are seen as motion changes in the particle trajectories. Despite the existence of multiple approaches to carry out this type of analysis, no objective assessment of these methods has been performed so far. Here, we have designed a competition to characterize and rank the performance of these methods when analyzing the dynamic behavior of single molecules. To run this competition, we have implemented a software library to simulate realistic data corresponding to widespread diffusion and interaction models, both in the form of trajectories and videos obtained in typical experimental conditions. The competition will constitute the first assessment of these methods, provide insights into the current limits of the field, foster the development of new approaches, and guide researchers to identify optimal tools for analyzing their experiments., Comment: 19 pages, 4 figure, 2 tables. Stage 1 registered report, accepted in principle in Nature Communications (https://springernature.figshare.com/articles/journal_contribution/Quantitative_evaluation_of_methods_to_analyze_motion_changes_in_single-particle_experiments_Registered_Report_Stage_1_Protocol_/24771687)

Through digital imaging, microscopy has evolved from primarily being a means for visual observation of life at the micro- and nano-scale, to a quantitative tool with ever-increasing resolution and throughput. Artificial intelligence, deep neural networks, and machine learning are all niche terms describing computational methods that have gained a pivotal role in microscopy-based research over the past decade. This Roadmap is written collectively by prominent researchers and encompasses selected aspects of how machine learning is applied to microscopy image data, with the aim of gaining scientific knowledge by improved image quality, automated detection, segmentation, classification and tracking of objects, and efficient merging of information from multiple imaging modalities. We aim to give the reader an overview of the key developments and an understanding of possibilities and limitations of machine learning for microscopy. It will be of interest to a wide cross-disciplinary audience in the physical sciences and life sciences.

Soft-granular media, such as dense emulsions, foams or tissues, exhibit either fluid- or solid-like properties depending on the applied external stresses. Whereas bulk rheology of such materials has been thoroughly investigated, the internal structural mechanics of finite soft-granular structures with free interfaces is still poorly understood. Here, we report the spontaneous `crystallization' and `melting' inside a model soft granular cluster -- a densely packed aggregate of $N\sim 30-40$ droplets engulfed by a fluid film -- subject to a varying external flow. We develop new machine learning tools to track the internal rearrangements in the quasi-2D cluster as it transits a sequence of constrictions. As the cluster relaxes from a state of strong mechanical deformations, we find differences in the dynamics of the grains within the interior of the cluster and those at its rim, with the latter experiencing larger deformations and less frequent rearrangements, effectively acting as an elastic membrane around a fluid-like core. We conclude that the observed structural-dynamical heterogeneity results from an interplay of the topological constrains, due to the presence of a closed interface, and the internal solid-fluid transitions. We discuss universality of such behavior in various types of finite soft granular structures, including biological tissues., Comment: 16 pages, 9 figures

Specular microscopy assessment of the human corneal endothelium (CE) in Fuchs' dystrophy is challenging due to the presence of dark image regions called guttae. This paper proposes a UNet-based segmentation approach that requires minimal post-processing and achieves reliable CE morphometric assessment and guttae identification across all degrees of Fuchs' dystrophy. We cast the segmentation problem as a regression task of the cell and gutta signed distance maps instead of a pixel-level classification task as typically done with UNets. Compared to the conventional UNet classification approach, the distance-map regression approach converges faster in clinically relevant parameters. It also produces morphometric parameters that agree with the manually-segmented ground-truth data, namely the average cell density difference of -41.9 cells/mm2 (95% confidence interval (CI) [-306.2, 222.5]) and the average difference of mean cell area of 14.8 um2 (95% CI [-41.9, 71.5]). These results suggest a promising alternative for CE assessment.

Particle tracking is a fundamental task in digital microscopy. Recently, machine-learning approaches have made great strides in overcoming the limitations of more classical approaches. The training of state-of-the-art machine-learning methods almost universally relies on either vast amounts of labeled experimental data or the ability to numerically simulate realistic datasets. However, the data produced by experiments are often challenging to label and cannot be easily reproduced numerically. Here, we propose a novel deep-learning method, named LodeSTAR (Low-shot deep Symmetric Tracking And Regression), that learns to tracks objects with sub-pixel accuracy from a single unlabeled experimental image. This is made possible by exploiting the inherent roto-translational symmetries of the data. We demonstrate that LodeSTAR outperforms traditional methods in terms of accuracy. Furthermore, we analyze challenging experimental data containing densely packed cells or noisy backgrounds. We also exploit additional symmetries to extend the measurable particle properties to the particle's vertical position by propagating the signal in Fourier space and its polarizability by scaling the signal strength. Thanks to the ability to train deep-learning models with a single unlabeled image, LodeSTAR can accelerate the development of high-quality microscopic analysis pipelines for engineering, biology, and medicine., Comment: 19 pages, 4 figures

The characterization of dynamical processes in living systems provides important clues for their mechanistic interpretation and link to biological functions. Thanks to recent advances in microscopy techniques, it is now possible to routinely record the motion of cells, organelles, and individual molecules at multiple spatiotemporal scales in physiological conditions. However, the automated analysis of dynamics occurring in crowded and complex environments still lags behind the acquisition of microscopic image sequences. Here, we present a framework based on geometric deep learning that achieves the accurate estimation of dynamical properties in various biologically-relevant scenarios. This deep-learning approach relies on a graph neural network enhanced by attention-based components. By processing object features with geometric priors, the network is capable of performing multiple tasks, from linking coordinates into trajectories to inferring local and global dynamic properties. We demonstrate the flexibility and reliability of this approach by applying it to real and simulated data corresponding to a broad range of biological experiments., Comment: 17 pages, 5 figure, 2 supplementary figures

Active matter comprises self-driven units, such as bacteria and synthetic microswimmers, that can spontaneously form complex patterns and assemble into functional microdevices. These processes are possible thanks to the out-of-equilibrium nature of active-matter systems, fueled by a one-way free-energy flow from the environment into the system. Here, we take the next step in the evolution of active matter by realizing a two-way coupling between active particles and their environment, where active particles act back on the environment giving rise to the formation of superstructures. In experiments and simulations we observe that, under light-illumination, colloidal particles and their near-critical environment create mutually-coupled co-evolving structures. These structures unify in the form of active superstructures featuring a droplet shape and a colloidal engine inducing self-propulsion. We call them active droploids -- a portmanteau of droplet and colloids. Our results provide a pathway to create active superstructures through environmental feedback.

Quantitative analysis of cell structures is essential for biomedical and pharmaceutical research. The standard imaging approach relies on fluorescence microscopy, where cell structures of interest are labeled by chemical staining techniques. However, these techniques are often invasive and sometimes even toxic to the cells, in addition to being time-consuming, labor-intensive, and expensive. Here, we introduce an alternative deep-learning-powered approach based on the analysis of brightfield images by a conditional generative adversarial neural network (cGAN). We show that this approach can extract information from the brightfield images to generate virtually-stained images, which can be used in subsequent downstream quantitative analyses of cell structures. Specifically, we train a cGAN to virtually stain lipid droplets, cytoplasm, and nuclei using brightfield images of human stem-cell-derived fat cells (adipocytes), which are of particular interest for nanomedicine and vaccine development. Subsequently, we use these virtually-stained images to extract quantitative measures about these cell structures. Generating virtually-stained fluorescence images is less invasive, less expensive, and more reproducible than standard chemical staining; furthermore, it frees up the fluorescence microscopy channels for other analytical probes, thus increasing the amount of information that can be extracted from each cell., Comment: 12 pages, 4 figures

Video microscopy has a long history of providing insights and breakthroughs for a broad range of disciplines, from physics to biology. Image analysis to extract quantitative information from video microscopy data has traditionally relied on algorithmic approaches, which are often difficult to implement, time consuming, and computationally expensive. Recently, alternative data-driven approaches using deep learning have greatly improved quantitative digital microscopy, potentially offering automatized, accurate, and fast image analysis. However, the combination of deep learning and video microscopy remains underutilized primarily due to the steep learning curve involved in developing custom deep-learning solutions. To overcome this issue, we introduce a software, DeepTrack 2.0, to design, train and validate deep-learning solutions for digital microscopy. We use it to exemplify how deep learning can be employed for a broad range of applications, from particle localization, tracking and characterization to cell counting and classification. Thanks to its user-friendly graphical interface, DeepTrack 2.0 can be easily customized for user-specific applications, and, thanks to its open-source object-oriented programming, it can be easily expanded to add features and functionalities, potentially introducing deep-learning-enhanced video microscopy to a far wider audience., Comment: 25 pages; 13 figures

This paper explores the use of a Bayesian non-parametric topic modeling technique for the purpose of anomaly detection in video data. We present results from two experiments. The first experiment shows that the proposed technique is automatically able characterize the underlying terrain, and detect anomalous flora in image data collected by an underwater robot. The second experiment shows that the same technique can be used on images from a static camera in a dynamic unstructured environment. In the second dataset, consisting of video data from a static seafloor camera capturing images of a busy coral reef, the proposed technique was able to detect all three instances of an underwater vehicle passing in front of the camera, amongst many other observations of fishes, debris, lighting changes due to surface waves, and benthic flora., Comment: 6 pages, ICRA 2016

In this work we discuss the elements required for the construction of the operator algebra for the space of paths over a simply laced $SU(3)$ graph. These operators are an important step in the construction of the bialgebra required to find the partition functions of some modular invariant CFTs. We define the cup and cap operators associated with back-and-forth sequences and add them to the creation and annihilation operators in the operator algebra as they are required for the calculation of the full space of essential paths prescribed by the fusion algebra. These operators require collapsed triangular cells that had not been found in previous works; here we provide explicit values for these cells and show their importance in order for the cell system to fulfill the Kuperberg relations for $SU(3)$ tangles. We also find that demanding that our operators satisfy the Temperley-Lieb algebra leads one naturally to consider operators that create and annihilate closed triangular sequences, which in turn provides an alternative the cup and cap operators as they allow one to replace back-and-forth sequences with closed triangular ones. We finally show that the essential paths obtained by using closed triangles are equivalent to those obtained originally using back-and-forth sequences., Comment: 10 pages

For simply laced $SU(3)$ graphs we offer a geometric understanding of the path creation and annihilation operators for $SU(3)$ in terms of creation and annihilation of sequences of three vertices forming triangular cells or collapsed triangular cells. We prove that the space of paths of a given length can be decomposed as a direct sum of orthogonal sub-spaces constructed by recurrent applications of the path creation operator on subspaces of essential paths of shorter length., Comment: 12 pages, 4 figures, v2: revised length with a more formal mathematical structure. Some partial results have been removed for clarity

We study a topological Abelian gauge theory that generalizes the Abelian Chern-Simons one, and that leads in a natural way to the Milnor's link invariant $\bar{\mu}(1,2,3)$ when the classical action on-shell is calculated., Comment: 4 pages; corrected equation