Search

Purpose: Although traditional craniotomy (TC) surgery has failed to show benefits for the functional outcome of intracerebral hemorrhage (ICH). However, a minimally invasive hematoma removal plan to avoid white matter fiber damage may be a safer and more feasible surgical approach, which may improve the prognosis of ICH. We conducted a historical cohort study on the use of multimodal image fusion-assisted neuroendoscopic surgery (MINS) for the treatment of ICH, and compared its safety and effectiveness with traditional methods. Methods: This is a historical cohort study involving 241 patients with cerebral hemorrhage. Divided into MINS group and TC group based on surgical methods. Multimodal images (CT skull, CT angiography, and white matter fiber of MRI diffusion-tensor imaging) were fused into 3 dimensional images for preoperative planning and intraoperative guidance of endoscopic hematoma removal in the MINS group. Clinical features, operative efficiency, perioperative complications, and prognoses between 2 groups were compared. Normally distributed data were analyzed using t-test of 2 independent samples, Non-normally distributed data were compared using the Kruskal-Wallis test. Meanwhile categorical data were analyzed via the Chi-square test or Fisher’s exact test. All statistical tests were two-sided, and p < 0.05 was considered statistically significant. Results: A total of 42 patients with ICH were enrolled, who underwent TC surgery or MINS. Patients who underwent MINS had shorter operative time (p

Feature representation is a crucial issue in multimodal image registration. The handcrafted features extracted by traditional methods are highly sensitive to nonlinear radiation differences, while supervised learning methods are limited by deficient labeled samples in the remote sensing field. Therefore, this article proposes a consistency feature representation learning method for multimodal image registration, which involves mapping data into a common feature space to realize the accurate alignment of remote sensing images. First, a contrastive network with a spatial attention mechanism is driven to enhance the capability to highlight high-level features of images. Second, a positive sample augmentation strategy is implemented with contrastive loss, which helps the model learn the inherent features better, and imposes constraints on the sample similarity to optimize the feature projection. Finally, a multimodal image registration framework is proposed to enhance the stability of feature matching. The proposed framework achieves accurate feature extraction and consistency feature description for multimodal images, ensuring robustness against nonlinear radiometric differences. Experimental results demonstrate that the proposed method obtains more reliable registration results on the SEN1-2 dataset. Furthermore, the proposed algorithm achieves superior performance on data from other modalities, indicating strong generalization ability.

Glioma, the most prevalent primary malignant tumor of the adult central nervous system, is highly malignant. Despite the current clinical treatment options including surgical resection, radiotherapy and chemotherapy, the survival period of patients is still relatively short, which poses a serious threat to patients' lives and health. In recent years, the research on glioma has made great progress, and the study on the mechanism of glioma development and the characteristics of the immune microenvironment has also deepened. At the same time, researches on molecular pathological typing of glioma, comprehensive clinical diagnostic and therapeutic programs, novel drug delivery systems and big data clinical translation are also steadily underway. This article reviews the research progress in the field of glioma in China in 2022, with the hope of providing reference for clinical diagnosis and treatment.

Feature matching for multimodal images is an important task in image processing. However, most methods perform image feature detection, description, and matching sequentially, resulting in a large loss, low matching accuracy, and slow performance. To tackle these challenges, we propose a detector-free method called FeMIP for feature matching of multimodal images. We design coarse matching and fine regression modules to implement accurate multimodal image feature matches in a coarse-to-fine manner. Furthermore, we add a novel data augmentation method enabling FeMIP to achieve feature matching faster and more accurately. The coarse-to-fine module automatically generates pixel-level labels on the original image, enabling FeMIP to perform pixel-level matching on data with only image-level labels. In addition, we use the principle of reinforcement learning to design a policy gradient method to improve the solution to the problem of discreteness in matching. Extensive experiments show that FeMIP has good generalization and achieves excellent matching performances. The code will be released at: https://github.com/LiaoYun0x0/FeMIP. [ABSTRACT FROM AUTHOR]

Although coronavirus disease 2019 (COVID-19) primarily affects the respiratory system, this disease can affect multiple systems, including the musculoskeletal system. However, studies regarding the effect of COVID-19 on musculoskeletal system are limited. This study analyzed the bone mineral density (BMD) and body composition of 33 patients with COVID-19 using their chest CT scans at admission, discharge, and one-year follow-up. Among the patients, 2 were asymptomatic, 8 had mild symptoms, 19 had ordinary symptoms, and 4 had severe symptoms, and no critical patients were included. The mean length of hospital stay was (29.5±9.6) days, and the mean duration to follow-up was (423.0±53.6) days. The study found no significant changes in BMD of segmental or total thoracic vertebrae at admission, discharge, and one-year follow-up. No significant changes were observed in the subcutaneous adipose tissue (SAT) area, visceral adipose tissue (VAT) area, and total adipose tissue (TAT) of the mid-slice of the T12~L1 intervertebral disc. However, the asymptomatic and mild group had a higher average BMD of the T1~T12 than that of the common and severe group at admission and one-year follow-up. The common and severe group had higher VAT and TAT area than those of the asymptomatic and mild group at admission and one-year follow-up. This study provides valuable insights into the effects of COVID-19 on the musculoskeletal system and suggests that COVID-19 does not significantly affect thoracic BMD and abdominal adipose tissue after one year of infection.

Matching remotely sensed multimodal images is a crucial process that poses significant challenges due to nonlinear radiometric differences and substantial image noise. To overcome these difficulties, this study presents a novel and practical template-matching algorithm specifically designed for this purpose. Unlike traditional approaches that rely on image intensity, the proposed algorithm focuses on matching multimodal images based on their geometric structure information. This approach enables the method to effectively adapt to variations in grayscale caused by radiometric differences. To enhance the matching performance, principal component analysis calculation based on the log-Gabor filter is proposed to estimate the structural feature of the image. The proposed method can estimate the structure feature accurately even under severe noise distortion. In addition, a learnable matching network is proposed for similarity measuring to adapt to the gradient reversal caused by the radiometric difference among remotely sensed multimodal images. Infrared, visible light, and synthetic aperture radar images are adopted for the evaluation, to verify the performance of the proposed algorithm. Based on the results, the proposed algorithm has a distinct advantage over other state-of-the-art template-matching algorithms. [ABSTRACT FROM AUTHOR]

Establishing feature correspondences between multimodal remote sensing images is an essential task for realizing diverse applications. Conventional matching methods, which employ gradient or phase congruency (PC) for feature detection and description, produce limited performance when images suffer from strong noises and intensity differences. In this study, we propose a novel modality-invariant structural feature representation (MISFR) method for multimodal remote sensing image matching. First, a maximal/minimal enhanced PC moment (EPCM) representation is designed by incorporating the PC with a multiscale relative total variation (RTV) model for feature detection and description. The EPCM integrates the advantages of these two models to exploit intrinsic structural features while providing robustness against modality variations. Then, to improve the feature stability and repeatability, an aggregation structural feature detector (ASFD) is proposed, in which the corner and edge points are separately extracted on the minimal and maximum EPCMs. Moreover, an adaptive binning-based log-polar descriptor is constructed on the maximal EPCM, named enhanced features of orientated PC (EOPC), to robustly characterize feature points. Various experiments on a series of multimodal remote sensing images demonstrate that our MISFR significantly improves the matching performance compared with four state-of-the-art approaches. [ABSTRACT FROM AUTHOR]

Multispectral and multimodal images are of important usage in the field of multi-source visual information fusion. Due to the alternation or movement of image devices, the acquired multispectral and multimodal images are usually misaligned, and hence image registration is pre-requisite. Different from the registration of common images, the registration of multispectral or multimodal images is a challenging problem due to the nonlinear variation of intensity and gradient. To cope with this challenge, we propose the phase congruency network (PCNet) to enhance the structure similarity of multispectral or multimodal images. The images can then be aligned using the similarity-enhanced feature maps produced by the network. PCNet is constructed under the inspiration of the well-known phase congruency. The network embeds the phase congruency prior into two simple trainable layers and series of modified learnable Gabor kernels. Thanks to the prior knowledge, once trained, PCNet is applicable on a variety of multispectral and multimodal data such as flash/no-flash and RGB/NIR images without additional further tuning. The prior also makes the network lightweight. The trainable parameters of PCNet is 2400× and 1500×less than the deep-learning registration method deep homography network (DHN) and unsupervised deep homography network (UDHN), while its registration performance surpasses them. Experimental results validate that PCNet outperforms current state-of-the-art conventional multimodal registration algorithms. Besides, PCNet can act as a complementary part of the deep-learning registration methods, which significantly boosts their registration accuracy. On the Columbia imaging and vision laboratory (CAVE) multispectral dataset, the percentage of the number of images under 1 pixel average corner error (ACE) of UDHN is raised from 0.1% to 82.5% after the processing of PCNet. • Using phase congruency prior to design a trainable light-weight network. • The network can enhance the similarity of multispectral/multimodal images. • Achieving high accuracy registration on various multispectral/multimodal datasets. • Boosting the registration results of deep-learning methods significantly. [ABSTRACT FROM AUTHOR]

Copyright of CT Theory & Applications is the property of Editorial Department of CT Theory & Applications and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Copyright of Journal of Xinyang Normal University Natural Science Edition is the property of Journal of Xinyang Normal University Editorial Office and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use. This abstract may be abridged. No warranty is given about the accuracy of the copy. Users should refer to the original published version of the material for the full abstract. (Copyright applies to all Abstracts.)

Multimodal image fusion aims to retain valid information from different modalities, remove redundant information to highlight critical targets, and maintain rich texture details in the fused image. However, current image fusion networks only use simple convolutional layers to extract features, ignoring global dependencies and channel contexts. This paper proposes GRPAFusion, a multimodal image fusion framework based on gradient residual and pyramid attention. The framework uses multiscale gradient residual blocks to extract multiscale structural features and multigranularity detail features from the source image. The depth features from different modalities were adaptively corrected for inter-channel responses using a pyramid split attention module to generate high-quality fused images. Experimental results on public datasets indicated that GRPAFusion outperforms the current fusion methods in subjective and objective evaluations. [ABSTRACT FROM AUTHOR]

Alzheimer's disease (AD), one of the major neurodegenerative diseases, has become the most common cause of dementia problems. Up to now, there is a lack of effective targeted therapeutic drugs and effective treatment modalities to stop the progression of the disease. With the continuous development of computer technology, the use of computer-aided diagnostic technology tools for AD early classification studies will provide clinicians with important assistance. Deep learning-based Alzheimer's disease (AD) imaging classification has become a current research hotspot. In this paper, we first describe the commonly used publicly available datasets in the AD imaging classification task; then introduce the commonly used deep learning classification models for AD diagnosis; secondly, we compare the studies that target different biomarkers of the subjects and the use of unimodal or a combination of different modalities for the early classification of AD; and finally, The challenges of AD classification are summarized and future research directions are proposed. [ABSTRACT FROM AUTHOR]

We reported a 74 year old Asian female with a uveal tumor with subretinal fluid (SRF) and cystoid macula edema. Since she declined biopsy, multimodal imaging study was performed, including fundus photography, ultrasonography, optical coherence tomography, fluorescein angiography, and magnetic resonance imaging. Uveal melanoma was diagnosed. However, despite aggressive treatment with TTT and three adjuvant intravitreal bevacizumab doses, SRF, and cystoid macula edema were persistent. Therefore, aflibercept was administered, resulting in anatomical and functional improvement within 1 month, which lasted for 12 months. Aflibercept offered great efficacy in improving refractory macular edema in this case of primary uveal melanoma. Multimodal imaging can provide us with more diagnostic clues in differentiating the nature of intraocular tumors.

A fast and robust template matching scheme, called Matching by Slice Transform Matrix Mapping (MSTMM), is proposed for the matching difficulty under occluded scenarios caused by nonlinear intensity differences and structure differences between visible and thermal infrared images. The first step in the MSTMM scheme was to extract information about the distribution of pixels with the same gray level through the developed Expanded Slice Transform with Adaptive Gray Level (EST-AGL). After completing the construction of the EST-AGL matrix for all image patches, different EST-AGL matrices were mapped to different integers or floats through the traditional special integer mapping mechanism or the neural network mapping mechanism. Finally, template matching between visible and thermal infrared images was achieved by evaluating the similarity of correlation mapping surface images through the Normalized Cross Correlation (NCC) algorithm. The proposed EST-AGL method can overcome the nonlinear intensity differences between visible and thermal infrared images by extracting the structural features of the image. The mapping mechanism of the MSTMM scheme can reduce the structural differences between the normal template image and the query image under an occluded scenario by increasing the similarity between the normal image patches and the image patches with occlusion. The proper mapping mechanism ensures the high performance of the MSTMM scheme by using only the simple NCC algorithm instead of other time-consuming anti-occlusion dense feature algorithms in the similarity evaluation stage. The three main experimental results of the MSTMM scheme are as follows: (1) the scheme of MSTMM can achieve template matching in only 0.015 seconds when a $64\times 64$ template image slides on a $256\times 256$ query image on a hardware platform with limited resources; (2) the matching success rate of the MSTMM scheme can reach up to 75% among 2107 experimental samples; and (3) the neural network training in the neural network mapping mechanism only takes at least 104.4 seconds on the CPU.

Image registration of multimodal remote sensing images plays a vital role in remote sensing image analysis. However, there are significant nonlinear intensity differences between multimodal remote sensing image pairs, making it difficult for most traditional image registration algorithms to meet the registration requirements. In this paper, we propose a novel edge descriptor utilizing edge information, which has not only affine invariance but is also insensitive to nonlinear intensity differences. Moreover, we utilize the proposed descriptor to design a multimodal image registration algorithm. We use several different multimodal image pairs to evaluate the proposed algorithm. The experimental results show that the proposed algorithm holds a stable performance and can still achieve accurate spatial alignment even with the huge nonlinear intensity differences. [ABSTRACT FROM AUTHOR]

The identification of melanoma involves an integrated analysis of skin lesion images acquired using clinical and dermoscopy modalities. Dermoscopic images provide a detailed view of the subsurface visual structures that supplement the macroscopic details from clinical images. Visual melanoma diagnosis is commonly based on the 7-point visual category checklist (7PC), which involves identifying specific characteristics of skin lesions. The 7PC contains intrinsic relationships between categories that can aid classification, such as shared features, correlations, and the contributions of categories towards diagnosis. Manual classification is subjective and prone to intra- and interobserver variability. This presents an opportunity for automated methods to aid in diagnostic decision support. Current state-of-the-art methods focus on a single image modality (either clinical or dermoscopy) and ignore information from the other, or do not fully leverage the complementary information from both modalities. Furthermore, there is not a method to exploit the ‘intercategory’ relationships in the 7PC. In this study, we address these issues by proposing a graph-based intercategory and intermodality network (GIIN) with two modules. A graph-based relational module (GRM) leverages intercategorical relations, intermodal relations, and prioritises the visual structure details from dermoscopy by encoding category representations in a graph network. The category embedding learning module (CELM) captures representations that are specialised for each category and support the GRM. We show that our modules are effective at enhancing classification performance using three public datasets (7PC, ISIC 2017, and ISIC 2018), and that our method outperforms state-of-the-art methods at classifying the 7PC categories and diagnosis. [ABSTRACT FROM AUTHOR]

Purpose: Although traditional craniotomy (TC) surgery has failed to show benefits for the functional outcome of intracerebral hemorrhage (ICH). However, a minimally invasive hematoma removal plan to avoid white matter fiber damage may be a safer and more feasible surgical approach, which may improve the prognosis of ICH. We conducted a historical cohort study on the use of multimodal image fusion-assisted neuroendoscopic surgery (MINS) for the treatment of ICH, and compared its safety and effectiveness with traditional methods., Methods: This is a historical cohort study involving 241 patients with cerebral hemorrhage. Divided into MINS group and TC group based on surgical methods. Multimodal images (CT skull, CT angiography, and white matter fiber of MRI diffusion-tensor imaging) were fused into 3 dimensional images for preoperative planning and intraoperative guidance of endoscopic hematoma removal in the MINS group. Clinical features, operative efficiency, perioperative complications, and prognoses between 2 groups were compared. Normally distributed data were analyzed using t-test of 2 independent samples, Non-normally distributed data were compared using the Kruskal-Wallis test. Meanwhile categorical data were analyzed via the Chi-square test or Fisher's exact test. All statistical tests were two-sided, and p < 0.05 was considered statistically significant., Results: A total of 42 patients with ICH were enrolled, who underwent TC surgery or MINS. Patients who underwent MINS had shorter operative time (p < 0.001), less blood loss (p < 0.001), better hematoma evacuation (p = 0.003), and a shorter stay in the intensive care unit (p = 0.002) than patients who underwent TC. Based on clinical characteristics and analysis of perioperative complications, there is no significant difference between the 2 surgical methods. Modified Rankin scale scores at 180 days were better in the MINS than in the TC group (p = 0.014)., Conclusions: Compared with TC for the treatment of ICH, MINS is safer and more efficient in cleaning ICH, which improved the prognosis of the patients. In the future, a larger sample size clinical trial will be needed to evaluate its efficacy., (Copyright © 2024 Chinese Medical Association. Published by Elsevier B.V. All rights reserved.)

Most existing change detection (CD) methods target homogeneous images. However, in real-world scenarios like disaster management, where CD is urgent and pre-changed and post-changed images are typical of different modalities, significant challenges arise for multimodal change detection (MCD). One challenge is that bi-temporal image pairs, sourced from distinct sensors, may cause an image domain gap. Another issue surfaces when multimodal bi-temporal image pairs require collaborative input from domain experts who are specialised among different image fields for pixel-level annotation, resulting in scarce annotated samples. To address these challenges, this paper proposes a novel self-supervised difference contrast learning framework (Self-DCF). This framework facilitates networks training without labelled samples by automatically exploiting the feature information inherent in bi-temporal imagery to supervise each other mutually. Additionally, a Unified Mapping Unit reduces the domain gap between different modal images. The efficiency and robustness of Self-DCF are validated on five popular datasets, outperforming state-of-the-art algorithms. • A novel Self-supervised Multimodal Change Detection Framework is proposed. • A novel Poly feature extraction network captures features from bi-temporal images. • A Unified Mapping Unit narrows the domain gap between multimodal bi-temporal data. [ABSTRACT FROM AUTHOR]

Matching remotely sensed multimodal images is a crucial process that poses significant challenges due to nonlinear radiometric differences and substantial image noise. To overcome these difficulties, this study presents a novel and practical template-matching algorithm specifically designed for this purpose. Unlike traditional approaches that rely on image intensity, the proposed algorithm focuses on matching multimodal images based on their geometric structure information. This approach enables the method to effectively adapt to variations in grayscale caused by radiometric differences. To enhance the matching performance, principal component analysis calculation based on the log-Gabor filter is proposed to estimate the structural feature of the image. The proposed method can estimate the structure feature accurately even under severe noise distortion. In addition, a learnable matching network is proposed for similarity measuring to adapt to the gradient reversal caused by the radiometric difference among remotely sensed multimodal images. Infrared, visible light, and synthetic aperture radar images are adopted for the evaluation, to verify the performance of the proposed algorithm. Based on the results, the proposed algorithm has a distinct advantage over other state-of-the-art template-matching algorithms.

• We propose a dual-branch fusion CNN based on a crossover mechanism, named MACFNet. • We propose a cross-enhancement fusion module to increase the interaction of MRI and PET low-level features. • We propose a concise multi-scale attention guidance block by setting different dilation rates. Alzheimer's disease (AD) is a dreaded degenerative disease that results in a profound decline in human cognition and memory. Due to its intricate pathogenesis and the lack of effective therapeutic interventions, early diagnosis plays a paramount role in AD. Recent research based on neuroimaging has shown that the application of deep learning methods by multimodal neural images can effectively detect AD. However, these methods only concatenate and fuse the high-level features extracted from different modalities, ignoring the fusion and interaction of low-level features across modalities. It consequently leads to unsatisfactory classification performance. In this paper, we propose a novel multi-scale attention and cross-enhanced fusion network, MACFNet, which enables the interaction of multi-stage low-level features between inputs to learn shared feature representations. We first construct a novel Cross-Enhanced Fusion Module (CEFM), which fuses low-level features from different modalities through a multi-stage cross-structure. In addition, an Efficient Spatial Channel Attention (ECSA) module is proposed, which is able to focus on important AD-related features in images more efficiently and achieve feature enhancement from different modalities through two-stage residual concatenation. Finally, we also propose a multiscale attention guiding block (MSAG) based on dilated convolution, which can obtain rich receptive fields without increasing model parameters and computation, and effectively improve the efficiency of multiscale feature extraction. Experiments on the Alzheimer's Disease Neuroimaging Initiative (ADNI) dataset demonstrate that our MACFNet has better classification performance than existing multimodal methods, with classification accuracies of 99.59 %, 98.85 %, 99.61 %, and 98.23 % for AD vs. CN, AD vs. MCI, CN vs. MCI and AD vs. CN vs. MCI, respectively, and specificity of 98.92 %, 97.07 %, 99.58 % and 99.04 %, and sensitivity of 99.91 %, 99.89 %, 99.63 % and 97.75 %, respectively. The proposed MACFNet is a high-accuracy multimodal AD diagnostic framework. Through the cross mechanism and efficient attention, MACFNet can make full use of the low-level features of different modal medical images and effectively pay attention to the local and global information of the images. This work provides a valuable reference for multi-mode AD diagnosis. [ABSTRACT FROM AUTHOR]

Optical and synthetic aperture radar (SAR) image registration is important for multimodal remote sensing image information fusion. Recently, deep matching networks have shown better performances than traditional methods of image matching. However, due to significant differences between optical and SAR images, the performances of existing deep learning methods still need to be further improved. This article proposes a self-distillation feature learning network (SDNet) for optical and SAR image registration, improving performance from network structure and network optimization. First, we explore the impact of different weight-sharing strategies on optical and SAR image matching. Then, we design a partially unshared feature learning network for multimodal image feature learning. It has fewer parameters than the fully unshared network and has more flexibility than the fully shared network. In addition, the limited binary supervised information (matching or nonmatching) is insufficient to train the deep matching networks for optical-SAR image registration. Thus, we propose a self-distillation feature learning method to exploit more similarity information for deep network optimization enhancement, such as the similarity ordering between a series of nonmatching patch pairs. The exploited rich similarity information will significantly enhance network training and improve matching accuracy. Finally, existing deep learning methods brute-force make the matching features of the optical and SAR image patches similar, which will lead to the loss of discriminative information and degeneration of the matching performances. Thus, we build an auxiliary task reconstruction learning to optimize the feature learning network to keep more discriminative information. Extensive experiments demonstrate the effectiveness of our proposed method on multimodal image registration. [ABSTRACT FROM AUTHOR]

In the process of multimodal image fusion, how to improve the visual effect after the image fused, while taking into account the protection of energy and the extraction of details, has attracted more and more attention in recent years. Based on the research of visual saliency and the final action-level measurement of the base layer, a multimodal image fusion method based on a guided filter is proposed in this paper. Firstly, multi-scale decomposition of a guided filter is used to decompose the two source images into a small-scale layer, large-scale layer and base layer. The fusion rule of the maximum absolute value is adopted in the small-scale layer, the weight fusion rule based on regular visual parameters is adopted in the large-scale layer and the fusion rule based on activity-level measurement is adopted in the base layer. Finally, the fused three scales are laminated into the final fused image. The experimental results show that the proposed method can improve the image edge processing and visual effect in multimodal image fusion. [ABSTRACT FROM AUTHOR]

Abstract Background Cerebrovascular reactivity (CVR) is the change in cerebral blood flow in response to a vaso-active stimulus, and may assist the treatment strategy of ischemic stroke. However, previous studies reported that a therapeutic strategy for stroke mainly depends on the degree of vascular stenosis with steady-state vascular parameters (e.g., cerebral blood flow and CVR). Hence, measurement of CVR by multimodal imaging techniques may improve the treatment of ischemic stroke. Methods/design This is a prospective, randomized, controlled clinical trial that aimed to examine the capability of multimodal imaging techniques for the evaluation of CVR to improve treatment of patients with ischemic stroke. A total of 66 eligible patients will be recruited from Renji Hospital, Shanghai Jiaotong University School of Medicine. The patients will be categorized based on CVR into two subgroups as follows: CVR > 10% group and CVR

Multimodal image fusion aims to retain valid information from different modalities, remove redundant information to highlight critical targets, and maintain rich texture details in the fused image. However, current image fusion networks only use simple convolutional layers to extract features, ignoring global dependencies and channel contexts. This paper proposes GRPAFusion, a multimodal image fusion framework based on gradient residual and pyramid attention. The framework uses multiscale gradient residual blocks to extract multiscale structural features and multigranularity detail features from the source image. The depth features from different modalities were adaptively corrected for inter-channel responses using a pyramid split attention module to generate high-quality fused images. Experimental results on public datasets indicated that GRPAFusion outperforms the current fusion methods in subjective and objective evaluations.

Imaging methods have the overwhelming advantage of being non-invasive in the diagnosis of hepatic lesions and, thanks to technical developments in the field of ultrasound (US), radiation exposure can also be avoided in many clinical situations. In particular, contrast-enhanced US (CEUS) outperforms other radiological methods in regard to real-time images, repeatability, and prompt reporting and demonstrates relatively few contraindications and adverse reactions. In this study, we reported in detail a rare benign tumor: hepatic sclerosed hemangioma (HSH). We described US-based multimodal imaging (B-flow imaging, US elastography, and Sonazoid CEUS) features of this HSH case. Furthermore, by summarizing the recently published literature on the imaging diagnosis of HSH, we offered readers comprehensive knowledge of conventional imaging methods (CT, MRI) and CEUS in the diagnosis of HSH and preliminarily discussed their mechanism of pathology-based diagnosis. Our multimodal imaging approach may provide a diagnostic strategy for HSH, thus avoiding unnecessary biopsy or resection.

Objective: To address the feasibility and predictive value of multimodal image-based 3D reconstruction in assessing features of neurovascular relationship particularly regarding the detection of offending vessels, degree of compression exerted on the nerve root, in patients who underwent microvascular decompression for primary trigeminal neuralgia. Methods: This study includes 58 patients who underwent microvascular decompression for primary trigeminal neuralgia. All patients underwent preoperative 3.0-T magnetic resonance imaging (MRI) scan of 3D-TOF MRA and 3D-FIESTA sequences in combination. Multimodal MRIs were then registered with open-source software 3D Slicer, followed by 3D image reconstruction to get 3D models for detection of possible neurovascular relationship in the cerebellopontine angle (CPA). Evaluations were performed by 2 doctors and compared with the intraoperative findings and surgery video. Results: For detection of offending vessels, multimodal image-based 3D reconstruction sensitivity was 93.1% (54/58). Compared with the intraoperative findings, the k coefficients for evaluating the offending vessel and the degree of compression were >0.70 (P

Objective To explore the application value of 3D Slicer in the simulation and planning of transsphenoidal surgery for pituitary lesions. Methods 12 patients( 5 males and 7 females) with pituitary lesions who underwent transsphenoidal surgery were selected. In order to obtain the 3D visualization models, display the 3D spatial relationship of the sellar floor and surrounding blood vessels, formulate the surgical plan and simulate the surgical operation, 3D Slicer with original image data was used to perform image 3D reconstruction before the operation. Results All patients in study were operated on according to the optimized simulated surgical plan, and the endoscopic lesion resection was successfully completed without serious complications occurred after the operation. Re-examination of the brain CT within 24 hours after the operation showed that the position of the sellar bone window was satisfactory, there was no new bleeding in the operation area. Conclusion The use of 3D Slicer software to perform preoperative 3D reconstruction of transsphenoidal approach surgery and assist in the formulation of surgical plans can help surgeons obtain richer visual anatomical information and increase the success rate of surgery. [ABSTRACT FROM AUTHOR]

Multimodal image registration is critical yet challenging for remote sensing image processing. Due to the large nonlinear intensity differences between the multimodal images, conventional search algorithms tend to get trapped into local optima when optimizing the transformation parameters by maximizing mutual information (MI). To address this problem, inspired by transfer learning, we propose a novel search algorithm named transfer optimization (TO), which can be applied to any optimizer. In TO, an optimizer transfers its better individuals to the other optimizer in each iteration. Thus, TO can share information between two optimizers and take advantage of their search mechanisms, which is helpful to avoid the local optima. Then, the registration of the multimodal remote sensing images using TO is presented. We compare the proposed algorithm with several state-of-the-art algorithms on real and simulated image pairs. Experimental results demonstrate the superiority of our algorithm in terms of registration accuracy. [ABSTRACT FROM AUTHOR]

Current methods for the imaging of radioactive sources ignore the surrounding scene information. Reconstruction results do not have real scale meaning, creating difficulties for further disposal and management of nuclear waste. In addition, the traditional reconstruction method requires fixed hardware to directly measure the reconstruction parameters. In recent years, the combination of computer vision and traditional radiation imaging has been greatly performed. The binocular camera is a small, light-insusceptible visual instrument, suitable for integration with other instruments. In this article, we present a method based on binocular vision technology to radioactive source image reconstruction. The binocular camera provides the assisting information about the surrounding scene and replaces the fixed hardware to indirectly achieve the measurement target of reconstruction parameters, which includes an azimuth angle and a distance. In addition, we designed a simple algorithm based on the Hough transform and the regular shape of the waste drum to improve the spatial resolution for good visualization. The experimental rendering results with different transparency degrees provide a good sense of spatial scale. [ABSTRACT FROM AUTHOR]

Multispectral imaging plays a vital role in the area of computer vision and computational photography. As spectral band images can be misaligned due to imaging device movement or alternation, image registration is necessary to avoid spectral information distortion. The current registration measures specialized for multispectral data are typically robust yet complex, requiring excessive computation. The common measures such as sum of squared differences (SSD) and sum of absolute differences (SAD) are computationally efficient whereas they perform poorly on multispectral data. To cope with this challenge, we propose a structure consistency boosting (SCB) transform that aims at boosting the structural similarity of multispectral images. With SCB, the common measures can be employed for multispectral image registration. The SCB transform exploits the fact that inherent edge structures maintain relative saliency locally despite the nonlinear variation between band images. A statistical prior of the natural image, which is based on the gradient-intensity correlation, is explored to build a parametric form of SCB. Experimental results validate that the SCB transform outperforms current similarity enhancement algorithms, and performs better than the state-of-the-art multispectral registration measures. Thanks to the generality of the statistical prior, the SCB transform is also applicable to various multimodal data such as flash/no-flash images and medical images. [ABSTRACT FROM AUTHOR]

Multimodal image registration is becoming increasingly important in remote sensing. However, due to the significant nonlinear intensity differences between multimodal images, conventional registration methods tend to get trapped into local optima. To address this issue, we present a new approach for multimodal image registration using histogram of oriented gradient distance (HOGD) and data-driven grey wolf optimizer (DDGWO). First, we propose a novel similarity measure for area-based registration methods that is HOGD. We investigate the performance of HOGD by analyzing its similarity curve. HOGD has a large range of values, which is helpful to find the global optimum. Second, we use GWO to optimize the transformation parameters. Since it is time-consuming to calculate HOGD, we propose DDGWO to minimize HOGD. In DDGWO, the iterations are divided into two parts: the training and prediction iterations. A support vector machine (SVM) regression model is trained by the historical HOGD computed in the training iterations. The trained SVM model predicts HOGD instead of calculating in the prediction iterations, which can reduce the computational time. Finally, we test the proposed approach that uses HOGD as the similarity measure and DDGWO as the search algorithm on 12 real and four simulated image pairs. Extensive experiments demonstrate that our approach saves up to 83.35-84.15% of computational time and outperforms the state-of-the-art algorithms in terms of registration accuracy. [ABSTRACT FROM AUTHOR]

Background: Cerebrovascular reactivity (CVR) is the change in cerebral blood flow in response to a vaso-active stimulus, and may assist the treatment strategy of ischemic stroke. However, previous studies reported that a therapeutic strategy for stroke mainly depends on the degree of vascular stenosis with steady-state vascular parameters (e.g., cerebral blood flow and CVR). Hence, measurement of CVR by multimodal imaging techniques may improve the treatment of ischemic stroke.Methods/design: This is a prospective, randomized, controlled clinical trial that aimed to examine the capability of multimodal imaging techniques for the evaluation of CVR to improve treatment of patients with ischemic stroke. A total of 66 eligible patients will be recruited from Renji Hospital, Shanghai Jiaotong University School of Medicine. The patients will be categorized based on CVR into two subgroups as follows: CVR > 10% group and CVR < 10% group. The patients will be randomly assigned to medical management, percutaneous transluminal angioplasty and stenting, and intracranial and extra-cranial bypass groups in a 1:1:1 ratio. The primary endpoint is all adverse events and ipsilateral stroke recurrence at 6, 12, and 24 months after management. The secondary outcomes include the CVR, the National Institute of Health stroke scale and the Modified Rankin Scale at 6, 12, and 24 months.Discussion: Measurement of cerebrovascular reserve by multimodal image is recommended by most recent studies to guide the treatment of ischemic stroke, and thus its efficacy and evaluation accuracy need to be established in randomized controlled settings. This prospective, parallel, randomized, controlled registry study, together with other ongoing studies, should present more evidence for optimal individualized accurate treatment of ischemic stroke.Trial Registration: Chinese Clinical Trial Registry, ID: ChiCTR-IOR-16009635; Registered on 16 October 2016. All items are from the World Health Organization Trial Registration Data Set and registration in the Chinese Clinical Trial Registry: ChiCTR-IOR-16009635. [ABSTRACT FROM AUTHOR]

Prostate cancer (PCa) is one of the most common cancer diseases in men in the western countries [1]. Besides the palpation, and the amount of prostate-specific-antigen’s (PSA) inside the blood, the current diagnostic imaging technologies are not appropriate. Early diagnosis defining the exact tumor location, spread and margins could make efficient targeted biopsies and image-guided surgery. A multimodal imaging technique containing a transmit-receive surface coil for anatomical MR imaging, a (S)PET detector module, consisting of silicon photomultipliers (SiPM), for functional imaging and an ultrasound (US) probe are placed as close as possible to the prostate designed as an endorectal tube to increase sensitivity and spatial resolution. All materials that are used are non-magnetic. Advantages of the SiPM are diversified, like non-sensitive to magnetic fields, higher gain (105–106) than standard avalanche photodiodes (APD), good timing properties and compactness. The PET detector should reach approximately 1mm3 spatial resolution together with 60ps FWHM Time-of-Flight resolution and a high efficiency to reduce scanning time and injected dose. A home-made transmit-receive coil surrounding the PET module improves signal-to-noise-ratio (SNR) with respect to standard coils will be present. The system will be used as a MRI-insert and be able to visualize anatomic and metabolic information together. The US-probe is guiding examination for correct overlapping of the multimodal images. This procedure will save time, costs and the need of co-registration. By combining all advantages of each system, it will necessarily update the non-invasive treatment of PCa. The system is adapted and tested to a 3 Tesla MR scanner called Trio A Tim system and Allegra system from the company Siemens healthcare with a larmor frequency of 123.2 MHz and an input of 50 Ω free from artifacts. First results on homogeneity of the transmit-receive coil will be presented. Preliminary measurements showing the proposed device is challenging but feasible.

Objective: The objective is to describe a clinical case of paravenous atrophy using a multimodal image. Methods: A 48-year-old man was visited and followed in the ophthalmology department of Hospital Clinic (Barcelona). Visual acuity, slit-lamp exam, retinography, autofluorescence, visual field, optical coherence tomography, and electrophysiology test were performed. Results: The patient had a history of Behcet’s disease. Autofluorescence revealed hypoautofluorescence at the paravenous region, th visual field was also altered and electrophysiologic test were reduced.Conclusion: Paravenous atrophy is a rare entity not well described currently. Patients are frequently misdiagnosed. Multimodal image could help to characterize this condition properly and improve the management.

• We propose a methodology for the hierarchical representation of multimodal images. • The hierarchical representation relies on the novel concept of braids of partitions. • We introduce a braid-based method to conduct multimodal image segmentation. • The segmentation method is based on an energy minimization procedure. • The proposed approach is validated on different multimodal images. Hierarchical data representations are powerful tools to analyze images and have found numerous applications in image processing. When it comes to multimodal images however, the fusion of multiple hierarchies remains an open question. Recently, the concept of braids of partitions has been proposed as a theoretical tool and possible solution to this issue. In this paper, we demonstrate the relevance of the braid structure for the hierarchical representation of multimodal images. We first propose a fully operable procedure to build a braid of partitions from two hierarchical representations. We then derive a framework for multimodal image segmentation, relying on an energetic minimization scheme conducted on the braid structure. The proposed approach is investigated on different multimodal images scenarios, and the obtained results confirm its ability to efficiently handle the multimodal information to produce more accurate segmentation outputs. [ABSTRACT FROM AUTHOR]

Multimodal medical image fusion has played an important role to diagnose the diseases for clinical treatment analysis and improving the performance and precision of the computer assisted system. In a recent development of medical filed single multimodal medical image cannot provide all the details of human body. For example, the soft tissue information can be represented by magnetic resonance imaging, computed tomography imaging represent the bones dense structure with less distortion. In this paper, proposed the combination of NSCT with DTCWT hybrid fusion algorithm. The hybrid fusion algorithm is verified through a simulation experiment on different multimodality images. Experimental results are evaluated by the number of well-known performance evaluation metrics. [ABSTRACT FROM AUTHOR]

In recent years, the incidence of thyroid diseases has increased significantly and ultrasound examination is the first choice for the diagnosis of thyroid diseases. At the same time, the level of medical image analysis based on deep learning has been rapidly improved. Ultrasonic image analysis has made a series of milestone breakthroughs, and deep learning algorithms have shown strong performance in the field of medical image segmentation and classification. This article first elaborates on the application of deep learning algorithms in thyroid ultrasound image segmentation, feature extraction, and classification differentiation. Secondly, it summarizes the algorithms for deep learning processing multimodal ultrasound images. Finally, it points out the problems in thyroid ultrasound image diagnosis at the current stage and looks forward to future development directions. This study can promote the application of deep learning in clinical ultrasound image diagnosis of thyroid, and provide reference for doctors to diagnose thyroid disease.