DeepNeurite™: Identification of neurites from non‐specific binding of fluorescence probes through deep learning

Abstract The nervous system plays an important role in human health and disease, and the unique morphologies of the neurons underlie its ability to interface with tissues and organs throughout the entire body. In vitro, neurons can be grown alone or with other cell types to gain insight into how they communicate with other cell types in a more controlled experimental setup. To measure neuron growth and to study neuronal connectivity in vitro, neurite identification is an essential readout. However, non‐specific binding of fluorescence probes, a fundamental issue of fluorescence imaging, impairs neurite identification through conventional mathematical morphology‐based methods, especially in neuron and other cell type co‐culture imaging conditions. Here, we utilized a deep learning algorithm and developed a computational tool called DeepNeurite™, to overcome this challenge. We demonstrated that DeepNeurite™ can accurately identify neurite structure in images acquired from microfluidic compartmentalized chambers where neurons were co‐cultured, such as with a human prostate cancer cell line, PC3. We further validated that the model can be generalized to handle a direct co‐culture in which neurons and lung cancer cells (DMS273) are grown intermingled in the same well. Using this method, we observed more neurite growth into PC3 containing chambers in microfluidic compartmentalized chambers, which could be blocked by an NGF antibody. Finally, we applied DeepNeurite™ coupled with functional calcium imaging to study the communication of primary sensory neurons and cancer cells. We showed that the cancer cells closer to neurites exhibit greater calcium activity in response to neuronal stimulation. This method opens lots of opportunities to study the effect of neurons on various other cell types. This model could further tackle the off‐target labeling of the fluorescence probe in other subcellular structures or cell types.