Search

In a complex disease such as tuberculosis, the evidence for the disease and its evolution may be present in multiple modalities such as clinical, genomic, or imaging data. Effective patient-tailored outcome prediction and therapeutic guidance will require fusing evidence from these modalities. Such multimodal fusion is difficult since the evidence for the disease may not be uniform across all modalities, not all modality features may be relevant, or not all modalities may be present for all patients. All these nuances make simple methods of early, late, or intermediate fusion of features inadequate for outcome prediction. In this paper, we present a novel fusion framework using multiplexed graphs and derive a new graph neural network for learning from such graphs. Specifically, the framework allows modalities to be represented through their targeted encodings, and models their relationship explicitly via multiplexed graphs derived from salient features in a combined latent space. We present results that show that our proposed method outperforms state-of-the-art methods of fusing modalities for multi-outcome prediction on a large Tuberculosis (TB) dataset., Comment: Accepted into MICCAI 2022

Deep learning-based methods are gaining traction in digital pathology, with an increasing number of publications and challenges that aim at easing the work of systematically and exhaustively analyzing tissue slides. These methods often achieve very high accuracies, at the cost of requiring large annotated datasets to train. This requirement is especially difficult to fulfill in the medical field, where expert knowledge is essential. In this paper we focus on nuclei segmentation, which generally requires experienced pathologists to annotate the nuclear areas in gigapixel histological images. We propose an algorithm for instance segmentation that uses pseudo-label segmentations generated automatically from point annotations, as a method to reduce the burden for pathologists. With the generated segmentation masks, the proposed method trains a modified version of HoVer-Net model to achieve instance segmentation. Experimental results show that the proposed method is robust to inaccuracies in point annotations and comparison with Hover-Net trained with fully annotated instance masks shows that a degradation in segmentation performance does not always imply a degradation in higher order tasks such as tissue classification.

Mitotic counting is a vital prognostic marker of tumor proliferation in breast cancer. Deep learning-based mitotic detection is on par with pathologists, but it requires large labeled data for training. We propose a deep classification framework for enhancing mitosis detection by leveraging class label information, via softmax loss, and spatial distribution information among samples, via distance metric learning. We also investigate strategies towards steadily providing informative samples to boost the learning. The efficacy of the proposed framework is established through evaluation on ICPR 2012 and AMIDA 2013 mitotic data. Our framework significantly improves the detection with small training data and achieves on par or superior performance compared to state-of-the-art methods for using the entire training data., Comment: 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI)