Search

The accurate classification of kidney tumors necessitates the utilization of various diagnostic techniques. Within the domain of medical imaging tests, the integration of multi-modal medical imaging represents an innovative approach to enhance diagnostic precision. However, the integration of multi-modal medical imaging faces a critical challenge: the insufficiency of correspondingly paired data across modalities, resulting in a paucity of training samples for neural networks. To mitigate this limitation, generative artificial intelligence, specifically generative models capable of generating additional data, thereby addressing the gap in the multi-modal medical imaging record. In our work, our primary objective is to improve the classification results that outperform an existing method of using single-modal medical images. To achieve this, we harness the wealth of information of multi-modal data derived from Contrast-Enhanced Computed Tomography (CECT) and Magnetic Resonance Imaging (MRI), along with their respective subtypes, to determine which specific modality or pair contributes the most effectively to classification. Our work introduces the comprehensive comparison between the different multi-modal fusion techniques, in which the Area Under the Curve (AUC) serves as the benchmark for performance evaluation. Moreover, this work tackles the problem of the unavailability of kidney tumor data by partially recovering from the available data using Conditional CycleGAN, which is part of the image-to-image translation that maps between two different image domains. Through the employment of multi-modal fusion techniques and our proposed recovery of missing data, our research has yielded superior classification results than single-modal classification approaches.

In this research, we proposed a two-stage pipeline for segmenting urinary stones. The first stage U-Net generated the map localizing the urinary organs in full abdominal x-ray images. Then, this map was used for creating partitioned images input to the second stage U-Net to reduce class imbalance and was also used in stone-embedding augmentation to increase a number of training data. The U-Net model was trained with the combination of real stone-contained images and synthesized stone-embedded images to segment urinary stones on the partitioned input images. In addition, we proposed to use an inverse weighting method in the focal Tversky loss function in order to rebalance lesion size. The U-Net model using our proposed pipeline produced a 71.28% pixel-wise $F_{2}$ score and a 69.82% region-wise $F_{2}$ score, which were 2.88% and 7.63%, respectively, higher than those of a baseline method. Experimental results showed that the proposed method improved urinary stone segmentation results, especially for small stones and stones in uncommon locations.

A urinary stone is a type of abnormality that occurs frequently in the urinary system. An automated segmentation of urinary stones is important for assisting medical doctors in early diagnosis and further treatment. While deep learning techniques are effective for image segmentation, they require a large number of datasets to achieve high accuracy. We proposed a GAN-based augmentation technique for creating synthetic images based on stone and non-stone mask inputs in order to improve the segmentation network’s performance by increasing the number and diversity of training data. The synthetic training images were generated from stone-contained images and stone-free images using existing stone ground truth and corresponding stone location maps, respectively. To segment urinary stones from full abdominal x-ray images, we trained the MultiResUnet model using both original stone-contained and our proposed synthetic samples. The proposed method obtained a 69.59% pixel-wise $F_{1}$ score and a 68.14% region-wise $F_{1}$ score, which achieved an improvement of 2.12% and 2.13%, respectively, over a model trained with only the original stone-contained dataset.