Search

BACKGROUND: The emergence of deep learning (DL) techniques has revolutionized tumor detection and classification in medical imaging, with multimodal medical imaging (MMI) gaining recognition for its precision in diagnosis, treatment, and progression tracking. OBJECTIVE: This review comprehensively examines DL methods in transforming tumor detection and classification across MMI modalities, aiming to provide insights into advancements, limitations, and key challenges for further progress. METHODS: Systematic literature analysis identifies DL studies for tumor detection and classification, outlining methodologies including convolutional neural networks (CNNs), recurrent neural networks (RNNs), and their variants. Integration of multimodality imaging enhances accuracy and robustness. RESULTS: Recent advancements in DL-based MMI evaluation methods are surveyed, focusing on tumor detection and classification tasks. Various DL approaches, including CNNs, YOLO, Siamese Networks, Fusion-Based Models, Attention-Based Models, and Generative Adversarial Networks, are discussed with emphasis on PET-MRI, PET-CT, and SPECT-CT. FUTURE DIRECTIONS: The review outlines emerging trends and future directions in DL-based tumor analysis, aiming to guide researchers and clinicians toward more effective diagnosis and prognosis. Continued innovation and collaboration are stressed in this rapidly evolving domain. CONCLUSION: Conclusions drawn from literature analysis underscore the efficacy of DL approaches in tumor detection and classification, highlighting their potential to address challenges in MMI analysis and their implications for clinical practice. [ABSTRACT FROM AUTHOR]

The analysis of body motion is a valuable tool in the assessment and diagnosis of gait impairments, particularly those related to neurological disorders. In this study, we propose a novel automated system leveraging artificial intelligence for efficiently analyzing gait impairment from video-recorded images. The proposed methodology encompasses three key aspects. First, we generate a novel one-dimensional representation of each silhouette image, termed a silhouette sinogram, by computing the distance and angle between the centroid and each detected boundary points. This process enables us to effectively utilize relative variations in motion at different angles to detect gait patterns. Second, a one-dimensional convolutional neural network (1D CNN) model is developed and trained by incorporating the consecutive silhouette sinogram signals of silhouette frames to capture spatiotemporal information via assisted knowledge learning. This process allows the network to capture a broader context and temporal dependencies within the gait cycle, enabling a more accurate diagnosis of gait abnormalities. This study conducts training and an evaluation utilizing the publicly accessible INIT GAIT database. Finally, two evaluation schemes are employed: one leveraging individual silhouette frames and the other operating at the subject level, utilizing a majority voting technique. The outcomes of the proposed method showed superior enhancements in gait impairment recognition, with overall F1-scores of 100%, 90.62%, and 77.32% when evaluated based on sinogram signals, and 100%, 100%, and 83.33% when evaluated based on the subject level, for cases involving two, four, and six gait abnormalities, respectively. In conclusion, by comparing the observed locomotor function to a conventional gait pattern often seen in healthy individuals, the recommended approach allows for a quantitative and non-invasive evaluation of locomotion. [ABSTRACT FROM AUTHOR]

Medical image segmentation of tissue abnormalities, key organs, or blood vascular system is of great significance for any computerized diagnostic system. However, automatic segmentation in medical image analysis is a challenging task since it requires sophisticated knowledge of the target organ anatomy. This paper develops an end-to-end deep learning segmentation method called Contextual Multi-Scale Multi-Level Network (CMM-Net). The main idea is to fuse the global contextual features of multiple spatial scales at every contracting convolutional network level in the U-Net. Also, we re-exploit the dilated convolution module that enables an expansion of the receptive field with different rates depending on the size of feature maps throughout the networks. In addition, an augmented testing scheme referred to as Inversion Recovery (IR) which uses logical "OR" and "AND" operators is developed. The proposed segmentation network is evaluated on three medical imaging datasets, namely ISIC 2017 for skin lesions segmentation from dermoscopy images, DRIVE for retinal blood vessels segmentation from fundus images, and BraTS 2018 for brain gliomas segmentation from MR scans. The experimental results showed superior state-of-the-art performance with overall dice similarity coefficients of 85.78%, 80.27%, and 88.96% on the segmentation of skin lesions, retinal blood vessels, and brain tumors, respectively. The proposed CMM-Net is inherently general and could be efficiently applied as a robust tool for various medical image segmentations. [ABSTRACT FROM AUTHOR]

• We develop a single-stage knowledge distillation framework that successively trains the teacher and student networks in the same training iteration, which can be used for brain tumor segmentation on limited MR modality. • As a practical verification of the feasibility of the proposed segmentation approach to be generalized to clinical usage, we further test our model using brain dataset collected from the local hospital. • Experimental results of proposed model are better than other benchmark methods on both the BraTS dataset and our local dataset. Precisely segmenting brain tumors using multimodal Magnetic Resonance Imaging (MRI) is an essential task for early diagnosis, disease monitoring, and surgical planning. Unfortunately, the complete four image modalities utilized in the well-known BraTS benchmark dataset: T1, T2, Fluid-Attenuated Inversion Recovery (FLAIR), and T1 Contrast-Enhanced (T1CE) are not regularly acquired in clinical practice due to the high cost and long acquisition time. Rather, it is common to utilize limited image modalities for brain tumor segmentation. In this paper, we propose a single stage learning of knowledge distillation algorithm that derives information from the missing modalities for better segmentation of brain tumors. Unlike the previous works that adopted a two-stage framework to distill the knowledge from a pre-trained network into a student network, where the latter network is trained on limited image modality, we train both models simultaneously using a single-stage knowledge distillation algorithm. We transfer the information by reducing the redundancy from a teacher network trained on full image modalities to the student network using Barlow Twins loss on a latent-space level. To distill the knowledge on the pixel level, we further employ a deep supervision idea that trains the backbone networks of both teacher and student paths using Cross-Entropy loss. We demonstrate that the proposed single-stage knowledge distillation approach enables improving the performance of the student network in each tumor category with overall dice scores of 91.11% for Tumor Core, 89.70% for Enhancing Tumor, and 92.20% for Whole Tumor in the case of only using the FLAIR and T1CE images, outperforming the state-of-the-art segmentation methods. The outcomes of this work prove the feasibility of exploiting the knowledge distillation in segmenting brain tumors using limited image modalities and hence make it closer to clinical practices. [Display omitted] [ABSTRACT FROM AUTHOR]

A computer-aided diagnosis (CAD) system requires detection, segmentation, and classification in one framework to assist radiologists efficiently in an accurate diagnosis. In this paper, a completely integrated CAD system is proposed to screen digital X-ray mammograms involving detection, segmentation, and classification of breast masses via deep learning methodologies. In this work, to detect breast mass from entire mammograms, You-Only-Look-Once (YOLO), a regional deep learning approach, is used. To segment the mass, full resolution convolutional network (FrCN), a new deep network model, is proposed and utilized. Finally, a deep convolutional neural network (CNN) is used to recognize the mass and classify it as either benign or malignant. To evaluate the proposed integrated CAD system in terms of the accuracies of detection, segmentation, and classification, the publicly available and annotated INbreast database was utilized. The evaluation results of the proposed CAD system via four-fold cross-validation tests show that a mass detection accuracy of 98.96%, Matthews correlation coefficient (MCC) of 97.62%, and F1-score of 99.24% are achieved with the INbreast dataset. Moreover, the mass segmentation results via FrCN produced an overall accuracy of 92.97%, MCC of 85.93%, and Dice (F1-score) of 92.69% and Jaccard similarity coefficient metrics of 86.37%, respectively. The detected and segmented masses were classified via CNN and achieved an overall accuracy of 95.64%, AUC of 94.78%, MCC of 89.91%, and F1-score of 96.84%, respectively. Our results demonstrate that the proposed CAD system, through all stages of detection, segmentation, and classification, outperforms the latest conventional deep learning methodologies. Our proposed CAD system could be used to assist radiologists in all stages of detection, segmentation, and classification of breast masses. [ABSTRACT FROM AUTHOR]

Background and objective Automatic segmentation of skin lesions in dermoscopy images is still a challenging task due to the large shape variations and indistinct boundaries of the lesions. Accurate segmentation of skin lesions is a key prerequisite step for any computer-aided diagnostic system to recognize skin melanoma. Methods In this paper, we propose a novel segmentation methodology via full resolution convolutional networks (FrCN). The proposed FrCN method directly learns the full resolution features of each individual pixel of the input data without the need for pre- or post-processing operations such as artifact removal, low contrast adjustment, or further enhancement of the segmented skin lesion boundaries. We evaluated the proposed method using two publicly available databases, the IEEE International Symposium on Biomedical Imaging (ISBI) 2017 Challenge and PH2 datasets. To evaluate the proposed method, we compared the segmentation performance with the latest deep learning segmentation approaches such as the fully convolutional network (FCN), U-Net, and SegNet. Results Our results showed that the proposed FrCN method segmented the skin lesions with an average Jaccard index of 77.11% and an overall segmentation accuracy of 94.03% for the ISBI 2017 test dataset and 84.79% and 95.08%, respectively, for the PH2 dataset. In comparison to FCN, U-Net, and SegNet, the proposed FrCN outperformed them by 4.94%, 15.47%, and 7.48% for the Jaccard index and 1.31%, 3.89%, and 2.27% for the segmentation accuracy, respectively. Furthermore, the proposed FrCN achieved a segmentation accuracy of 95.62% for some representative clinical benign cases, 90.78% for the melanoma cases, and 91.29% for the seborrheic keratosis cases in the ISBI 2017 test dataset, exhibiting better performance than those of FCN, U-Net, and SegNet. Conclusions We conclude that using the full spatial resolutions of the input image could enable to learn better specific and prominent features, leading to an improvement in the segmentation performance. [ABSTRACT FROM AUTHOR]

BACKGROUND: In general, the image quality of high and low energy images of dual energy X-ray absorptiometry (DXA) suffers from noise due to the use of a small amount of X-rays. Denoising of DXA images could be a key process to improve a bone mineral density map, which is derived from a pair of high and low energy images. This could further improve the accuracy of diagnosis of bone fractures and osteoporosis. OBJECTIVE: This study aims to develop and test a new technology to improve the quality, remove the noise, and preserve the edges and fine details of real DXA images. METHODS: In this study, a denoising technique for high and low energy DXA images using a non-local mean filter (NLM) was presented. The source and detector noises of a DXA system were modeled for both high and low DXA images. Then, the optimized parameters of the NLM filter were derived utilizing the experimental data from CIRS-BFP phantoms. After that, the optimized NLM was tested and verified using the DXA images of the phantoms and real human spine and femur. RESULTS: Quantitative evaluation of the results showed average 24.22% and 34.43% improvement of the signal-to-noise ratio for real high and low spine images, respectively, while the improvements were about 15.26% and 13.55% for the high and low images of the femur. The qualitative visual observations of both phantom and real structures also showed significantly improved quality and reduced noise while preserving the edges in both high and low energy images. Our results demonstrate that the proposed NLM outperforms the conventional method using an anisotropic diffusion filter (ADF) and median techniques for all phantom and real human DXA images. CONCLUSIONS: Our work suggests that denoising via NLM could be a key preprocessing method for clinical DXA imaging. [ABSTRACT FROM AUTHOR]

Background and objective Automatic detection and classification of the masses in mammograms are still a big challenge and play a crucial role to assist radiologists for accurate diagnosis. In this paper, we propose a novel Computer-Aided Diagnosis (CAD) system based on one of the regional deep learning techniques, a ROI-based Convolutional Neural Network (CNN) which is called You Only Look Once (YOLO). Although most previous studies only deal with classification of masses, our proposed YOLO-based CAD system can handle detection and classification simultaneously in one framework. Methods The proposed CAD system contains four main stages: preprocessing of mammograms, feature extraction utilizing deep convolutional networks, mass detection with confidence, and finally mass classification using Fully Connected Neural Networks (FC-NNs). In this study, we utilized original 600 mammograms from Digital Database for Screening Mammography (DDSM) and their augmented mammograms of 2,400 with the information of the masses and their types in training and testing our CAD. The trained YOLO-based CAD system detects the masses and then classifies their types into benign or malignant. Results Our results with five-fold cross validation tests show that the proposed CAD system detects the mass location with an overall accuracy of 99.7%. The system also distinguishes between benign and malignant lesions with an overall accuracy of 97%. Conclusions Our proposed system even works on some challenging breast cancer cases where the masses exist over the pectoral muscles or dense regions. [ABSTRACT FROM AUTHOR]

Magnetic Resonance Imaging (MRI) is sensitive to motion caused by patient movement due to the relatively long data acquisition time. This could cause severe degradation of image quality and therefore affect the overall diagnosis. In this paper, we develop an efficient retrospective 2D deep learning method called stacked U-Nets with self-assisted priors to address the problem of rigid motion artifacts in 3D brain MRI. The proposed work exploits the usage of additional knowledge priors from the corrupted images themselves without the need for additional contrast data. The proposed network learns the missed structural details through sharing auxiliary information from the contiguous slices of the same distorted subject. We further design a refinement stacked U-Nets that facilitates preserving the spatial image details and improves the pixel-to-pixel dependency. To perform network training, simulation of MRI motion artifacts is inevitable. The proposed network is optimized by minimizing the loss of structural similarity (SSIM) using the synthesized motion-corrupted images from 83 real motion-free subjects. We present an intensive analysis using various types of image priors: the proposed self-assisted priors and priors from other image contrast of the same subject. The experimental analysis proves the effectiveness and feasibility of our self-assisted priors since it does not require any further data scans. The overall image quality of the motion-corrected images via the proposed motion correction network significantly improves SSIM from 71.66% to 95.03% and declines the mean square error from 99.25 to 29.76. These results indicate the high similarity of the brain's anatomical structure in the corrected images compared to the motion-free data. The motion-corrected results of both the simulated and real motion data showed the potential of the proposed motion correction network to be feasible and applicable in clinical practices. [Display omitted] [ABSTRACT FROM AUTHOR]

Electroretinography (ERG) is a non-invasive method of assessing retinal function by recording the retina's response to a brief flash of light. This study focused on optimizing the ERG waveform signal classification by utilizing Short-Time Fourier Transform (STFT) spectrogram preprocessing with a machine learning (ML) decision system. Several window functions of different sizes and window overlaps were compared to enhance feature extraction concerning specific ML algorithms. The obtained spectrograms were employed to train deep learning models alongside manual feature extraction for more classical ML models. Our findings demonstrated the superiority of utilizing the Visual Transformer architecture with a Hamming window function, showcasing its advantage in ERG signal classification. Also, as a result, we recommend the RF algorithm for scenarios necessitating manual feature extraction, particularly with the Boxcar (rectangular) or Bartlett window functions. By elucidating the optimal methodologies for feature extraction and classification, this study contributes to advancing the diagnostic capabilities of ERG analysis in clinical settings. [ABSTRACT FROM AUTHOR]

• An integrated two-stage diagnostic model for skin lesion boundary segmentation and multiple skin diseases classification is proposed. • Segmentation of skin lesion boundaries was performed using a full resolution convolutional network (FrCN) segmentator. • Four well-established classifiers are evaluated: Inception-v3, ResNet-50, Inception-ResNet-v2, and DenseNet-201, to distinguish between different skin diseases. • The proposed work was trained and evaluated using three different datasets: two classes of ISIC 2016; three classes of ISIC 2017; and seven classes of ISIC 2018. • Proper rebalancing, segmentation, and augmentation of the datasets are investigated. Computer automated diagnosis of various skin lesions through medical dermoscopy images remains a challenging task. In this work, we propose an integrated diagnostic framework that combines a skin lesion boundary segmentation stage and a multiple skin lesions classification stage. Firstly, we segment the skin lesion boundaries from the entire dermoscopy images using deep learning full resolution convolutional network (FrCN). Then, a convolutional neural network classifier (i.e., Inception-v3, ResNet-50, Inception-ResNet-v2, and DenseNet-201) is applied on the segmented skin lesions for classification. The former stage is a critical prerequisite step for skin lesion diagnosis since it extracts prominent features of various types of skin lesions. A promising classifier is selected by testing well-established classification convolutional neural networks. The proposed integrated deep learning model has been evaluated using three independent datasets (i.e., International Skin Imaging Collaboration (ISIC) 2016, 2017, and 2018, which contain two, three, and seven types of skin lesions, respectively) with proper balancing, segmentation, and augmentation. In the integrated diagnostic system, segmented lesions improve the classification performance of Inception-ResNet-v2 by 2.72% and 4.71% in terms of the F1-score for benign and malignant cases of the ISIC 2016 test dataset, respectively. The classifiers of Inception-v3, ResNet-50, Inception-ResNet-v2, and DenseNet-201 exhibit their capability with overall weighted prediction accuracies of 77.04%, 79.95%, 81.79%, and 81.27% for two classes of ISIC 2016, 81.29%, 81.57%, 81.34%, and 73.44% for three classes of ISIC 2017, and 88.05%, 89.28%, 87.74%, and 88.70% for seven classes of ISIC 2018, respectively, demonstrating the superior performance of ResNet-50. The proposed integrated diagnostic networks could be used to support and aid dermatologists for further improvement in skin cancer diagnosis. Image, graphical abstract [ABSTRACT FROM AUTHOR]

Refactoring, a widely adopted technique, has proven effective in facilitating and reducing maintenance activities and costs. Nonetheless, the effects of applying refactoring techniques on software quality exhibit inconsistencies and contradictions, leading to conflicting evidence on their overall benefit. Consequently, software developers face challenges in leveraging these techniques to improve software quality. Moreover, the absence of a categorization model hampers developers' ability to decide the most suitable refactoring techniques for improving software quality, considering specific design goals. Thus, this study aims to propose a novel refactoring categorization model that categorizes techniques based on their measurable impacts on internal quality attributes. Initially, the most common refactoring techniques used by software practitioners were identified. Subsequently, an experimental study was conducted using five case studies to measure the impacts of refactoring techniques on internal quality attributes. A subsequent multi-case analysis was conducted to analyze these effects across the case studies. The proposed model was developed based on the experimental study results and the subsequent multi-case analysis. The model categorizes refactoring techniques into green, yellow, and red categories. The proposed model, by acting as a guideline, assists developers in understanding the effects of each refactoring technique on quality attributes, allowing them to select appropriate techniques to improve specific quality attributes. Compared to existing studies, the proposed model emerges superior by offering a more granular categorization (green, yellow, and red categories), and its range is wide (including ten refactoring techniques and eleven internal quality attributes). Such granularity not only equips developers with an in-depth understanding of each technique's impact but also fosters informed decision-making. In addition, the proposed model outperforms current studies and offers a more nuanced understanding, explicitly highlighting areas of strength and concern for each refactoring technique. This enhancement aids developers in better grasping the implications of each refactoring technique on quality attributes. As a result, the model simplifies the decision-making process for developers, saving time and effort that would otherwise be spent weighing the benefits and drawbacks of various refactoring techniques. Furthermore, it has the potential to help reduce maintenance activities and associated costs. [ABSTRACT FROM AUTHOR]

Magnetic resonance electrical properties tomography (MR‐EPT) is a non‐invasive measurement technique that derives the electrical properties (EPs, e.g., conductivity or permittivity) of tissues in the radiofrequency range (64 MHz for 1.5 T and 128 MHz for 3 T MR systems). Clinical studies have shown the potential of tissue conductivity as a biomarker. To date, model‐based conductivity reconstructions rely on numerical assumptions and approximations, leading to inaccuracies in the reconstructed maps. To address such limitations, we propose an artificial neural network (ANN)‐based non‐linear conductivity estimator trained on simulated data for conductivity brain imaging. Network training was performed on 201 synthesized T2‐weighted spin‐echo (SE) data obtained from the finite‐difference time‐domain (FDTD) electromagnetic (EM) simulation. The dataset was composed of an approximated T2‐w SE magnitude and transceive phase information. The proposed method was tested three in‐silico and in‐vivo on two volunteers and three patients' data. For comparison purposes, various conventional phase‐based EPT reconstruction methods were used that ignore B1+ magnitude information, such as Savitzky–Golay kernel combined with Gaussian filter (S‐G Kernel), phase‐based convection‐reaction EPT (cr‐EPT), magnitude‐weighted polynomial‐fitting phase‐based EPT (Poly‐Fit), and integral‐based phase‐based EPT (Integral‐based). From the in‐silico experiments, quantitative analysis showed that the proposed method provides more accurate and improved quality (e.g., high structural preservation) conductivity maps compared to conventional reconstruction methods. Representatively, in the healthy brain in‐silico phantom experiment, the proposed method yielded mean conductivity values of 1.97 ± 0.20 S/m for CSF, 0.33 ± 0.04 S/m for WM, and 0.52 ± 0.08 S/m for GM, which were closer to the ground‐truth conductivity (2.00, 0.30, 0.50 S/m) than the integral‐based method (2.56 ± 2.31, 0.39 ± 0.12, 0.68 ± 0.33 S/m). In‐vivo ANN‐based conductivity reconstructions were also of improved quality compared to conventional reconstructions and demonstrated network generalizability and robustness to in‐vivo data and pathologies. The reported in‐vivo brain conductivity values were in agreement with literatures. In addition, the proposed method was observed for various SNR levels (SNR levels = 10, 20, 40, and 58) and repeatability conditions (the eight acquisitions with the number of signal averages = 1). The preliminary investigations on brain tumor patient datasets suggest that the network trained on simulated dataset can generalize to unforeseen in‐vivo pathologies, thus demonstrating its potential for clinical applications. [ABSTRACT FROM AUTHOR]

BACKGROUND: Accurate measurement of bone mineral density (BMD) in dual-energy X-ray absorptiometry (DXA) is essential for proper diagnosis of osteoporosis. Calculation of BMD requires precise bone segmentation and subtraction of soft tissue absorption. Femur segmentation remains a challenge as many existing methods fail to correctly distinguish femur from soft tissue. Reasons for this failure include low contrast and noise in DXA images, bone shape variability, and inconsistent X-ray beam penetration and attenuation, which cause shadowing effects and person-to-person variation. OBJECTIVE: To present a new method namely, a Pixel Label Decision Tree (PLDT), and test whether it can achieve higher accurate performance in femur segmentation in DXA imaging. METHODS: PLDT involves mainly feature extraction and selection. Unlike photographic images, X-ray images include features on the surface and inside an object. In order to reveal hidden patterns in DXA images, PLDT generates seven new feature maps from existing high energy (HE) and low energy (LE) X-ray features and determines the best feature set for the model. The performance of PLDT in femur segmentation is compared with that of three widely used medical image segmentation algorithms, the Global Threshold (GT), Region Growing Threshold (RGT), and artificial neural networks (ANN). RESULTS: PLDT achieved a higher accuracy of femur segmentation in DXA imaging (91.4%) than either GT (68.4%), RGT (76%) or ANN (84.4%). CONCLUSIONS: The study demonstrated that PLDT outperformed other conventional segmentation techniques in segmenting DXA images. Improved segmentation should help accurate computation of BMD which later improves clinical diagnosis of osteoporosis. [ABSTRACT FROM AUTHOR]

Most current surgical navigation methods rely on optical navigators with images displayed on an external screen. However, minimizing distractions during surgery is critical and the spatial information displayed in this arrangement is non-intuitive. Previous studies have proposed combining optical navigation systems with augmented reality (AR) to provide surgeons with intuitive imaging during surgery, through the use of planar and three-dimensional imagery. However, these studies have mainly focused on visual aids and have paid relatively little attention to real surgical guidance aids. Moreover, the use of augmented reality reduces system stability and accuracy, and optical navigation systems are costly. Therefore, this paper proposed an augmented reality surgical navigation system based on image positioning that achieves the desired system advantages with low cost, high stability, and high accuracy. This system also provides intuitive guidance for the surgical target point, entry point, and trajectory. Once the surgeon uses the navigation stick to indicate the position of the surgical entry point, the connection between the surgical target and the surgical entry point is immediately displayed on the AR device (tablet or HoloLens glasses), and a dynamic auxiliary line is shown to assist with incision angle and depth. Clinical trials were conducted for EVD (extra-ventricular drainage) surgery, and surgeons confirmed the system's overall benefit. A "virtual object automatic scanning" method is proposed to achieve a high accuracy of 1 ± 0.1 mm for the AR-based system. Furthermore, a deep learning-based U-Net segmentation network is incorporated to enable automatic identification of the hydrocephalus location by the system. The system achieves improved recognition accuracy, sensitivity, and specificity of 99.93%, 93.85%, and 95.73%, respectively, representing a significant improvement from previous studies. [ABSTRACT FROM AUTHOR]