Search

IntroductionImaging fusion technology is promising as it is radiation and contrast sparing. Herein, we compare conventional biplane angiography to multimodality image fusion with live fluoroscopy using two-dimensional (2D)–three-dimensional (3D) registration (MMIF2D−3D) and assess MMIF2D−3D impact on radiation exposure and contrast volume during cardiac catheterization of patients with congenital heart disease (CHD).MethodsWe matched institutional MMIF2D−3D procedures and controls according to patient characteristics (body mass index, age, and gender) and the seven procedure-type subgroups. Then, we matched the number of tests and controls per subgroup using chronological ordering or propensity score matching. Subsequently, we combined the matched subgroups into larger subgroups of similar procedure type, keeping subgroups with at least 10 test and 10 control cases. Air kerma (AK) and dose area product (DAP) were normalized by body weight (BW), product of body weight and fluoroscopy time (BW × FT), or product of body weight and number of frames (BW × FR), and stratified by acquisition plane and irradiation event type (fluoroscopy or acquisition). Three senior interventionists evaluated the relevance of MMIF2D−3D (5-point Likert scale).ResultsThe Overall group consisted of 54 MMIF2D−3D cases. The combined and matched subgroups were pulmonary artery stenting (StentPUL), aorta angioplasty (PlastyAO), pulmonary artery angioplasty (PlastyPUL), or a combination of the latter two (Plasty). The FT of the lateral plane reduced significantly by 69.6% for the Overall MMIF2D−3D population. AKBW and DAPBW decreased, respectively, by 43.9% and 39.3% (Overall group), 49.3% and 54.9% (PlastyAO), and 36.7% and 44.4% for the Plasty subgroup. All the aforementioned reductions were statistically significant except for DAPBW in the Overall and Plasty (sub)groups. The decrease of AKBW and DAPBW in the StentPUL and PlastyPUL subgroups was not statistically significant. The decrease in the median values of the weight-normalized contrast volume (CMCBW) in all five subgroups was not significant. Cardiologists considered MMIF2D−3D very useful with a median score of 4.ConclusionIn our institution, MMIF2D−3D overall enabled significant AKBW reduction during the catheterization of CHD patients and was mainly driven by reduced FT in the lateral plane. We observed significant AKBW reduction in the Plasty and PlastyAO subgroups and DAPBW reduction in the PlastyAO subgroup. However, the decrease in CMCBW was not significant.

During surgery for foci-related epilepsy, neurosurgeons face significant difficulties in identifying and resecting MRI-negative or deep-seated epileptic foci. Here, we present a neuro-robotic navigation system that is specifically designed for resection of MRI negative epileptic foci. We recruited 52 epileptic patients, and randomly assigned them to treatment group with either neuro-robotic navigation or conventional neuronavigation system. For each patient, in the neuro-robotic navigation group, we integrated multimodality imaging including MRI and PET-CT into the robotic workstation and marked the boundary of foci from the fused image. During surgery, this boundary was delineated by the robotic laser device with high accuracy, guiding resection for the surgeon. For deeply seated foci, we exploited the neuro-robotic navigation system to localize the deepest point with biopsy needle insertion and methylene dye application to locate the boundary of the foci. Our results show that, compared with the conventional neuronavigation, the neuro-robotic navigation system performs equally well in MRI positive epilepsy patients (ENGEL I ratio: 71.4% vs 100%, p = 0.255) systems and show better performance in patients with MRI-negative focal cortical dysplasia (ENGEL I ratio: 88.2% vs 50%, p = 0.0439). At present, there are no documented neurosurgery robots with similar function and application in the field of epilepsy. Our research highlights the added value of using neuro-robotic navigation systems in resection surgery for epilepsy, particularly in cases that involve MRI-negative or deep-seated epileptic foci. [ABSTRACT FROM AUTHOR]

With the rapid development of remote sensing technology, the application of convolutional neural networks in remote sensing object detection has become very widespread, and some multimodal feature fusion networks have also been proposed in recent years. However, these methods generally do not consider the long-tailed problem that is widely present in remote sensing images, which limits the further improvement of model detection performance. To solve this problem, we propose a novel long-tailed object detection method for multimodal remote sensing images, which can effectively fuse the complementary information of visible light and infrared images and adapt to the imbalance between positive and negative samples of different categories. Firstly, the dynamic feature fusion module (DFF) based on image entropy can dynamically adjust the fusion coefficient according to the information content of different source images, retaining more key feature information for subsequent object detection. Secondly, the instance-balanced mosaic (IBM) data augmentation method balances instance sampling during data augmentation, providing more sample features for the model and alleviating the negative impact of data distribution imbalance. Finally, class-balanced BCE loss (CBB) can not only consider the learning difficulty of specific instances but also balances the learning difficulty between categories, thereby improving the model's detection accuracy for tail instances. Experimental results on three public benchmark datasets show that our proposed method achieves state-of-the-art performance; in particular, the optimization of the long-tailed problem enables the model to meet various application scenarios of remote sensing image detection. [ABSTRACT FROM AUTHOR]

Abstract Background Positron emission tomography (PET)-derived LV MBF quantification is usually measured in standard anatomical vascular territories potentially averaging flow from normally perfused tissue with those from areas with abnormal flow supply. Previously we reported on an image-based tool to noninvasively measure absolute myocardial blood flow at locations just below individual epicardial vessel to help guide revascularization. The aim of this work is to determine the robustness of vessel-specific flow measurements (MBFvs) extracted from the fusion of dynamic PET (dPET) with coronary computed tomography angiography (CCTA) myocardial segmentations, using flow measured from the fusion with CCTA manual segmentation as the reference standard. Methods Forty-three patients’ 13NH3 dPET, CCTA image datasets were used to measure the agreement of the MBFvs profiles after the fusion of dPET data with three CCTA anatomical models: (1) a manual model, (2) a fully automated segmented model and (3) a corrected model, where major inaccuracies in the automated segmentation were briefly edited. Pairwise accuracy of the normality/abnormality agreement of flow values along differently extracted vessels was determined by comparing, on a point-by-point basis, each vessel’s flow to corresponding vessels’ normal limits using Dice coefficients (DC) as the metric. Results Of the 43 patients CCTA fully automated mask models, 27 patients’ borders required manual correction before dPET/CCTA image fusion, but this editing process was brief (2–3 min) allowing a 100% success rate of extracting MBFvs in clinically acceptable times. In total, 124 vessels were analyzed after dPET fusion with the manual and corrected CCTA mask models yielding 2225 stress and 2122 rest flow values. Forty-seven vessels were analyzed after fusion with the fully automatic masks producing 840 stress and 825 rest flow samples. All DC coefficients computed globally or by territory were ≥ 0.93. No statistical differences were found in the normal/abnormal flow classifications between manual and corrected or manual and fully automated CCTA masks. Conclusion Fully automated and manually corrected myocardial CCTA segmentation provides anatomical masks in clinically acceptable times for vessel-specific myocardial blood flow measurements using dynamic PET/CCTA image fusion which are not significantly different in flow accuracy and within clinically acceptable processing times compared to fully manually segmented CCTA myocardial masks.

In medical applications, medical image fusion methods are capable of fusing the medical images from various morphologies to obtain a reliable medical diagnosis. A single modality image cannot provide sufficient information for an exact diagnosis. Hence, an efficient multimodal medical image fusion-based artificial intelligence model is proposed in this paper. Initially, the multimodal medical images are obtained for an effective fusion process by using a modified discrete wavelet transform (MDWT) thereby attaining an image with high visual clarity. Then, the fused images are classified as malignant or benign using the proposed convolutional neural network-based hybrid optimization dynamic algorithm (CNN-HOD). To enhance the weight function and classification accuracy of the CNN, a hybrid optimization dynamic algorithm (HOD) is proposed. The HOD is the integration of the sailfish optimizer algorithm and seagull optimization algorithm. Here, the seagull optimizer algorithm replaces the migration operation toobtain the optimal location. The experimental analysis is carried out and acquired with standard deviation (58%), average gradient (88%), and fusion factor (73%) compared with the other approaches. The experimental results demonstrate that the proposed approach performs better than other approaches and offers high-quality fused images for an accurate diagnosis. [ABSTRACT FROM AUTHOR]

Background: Positron emission tomography (PET)-derived LV MBF quantification is usually measured in standard anatomical vascular territories potentially averaging flow from normally perfused tissue with those from areas with abnormal flow supply. Previously we reported on an image-based tool to noninvasively measure absolute myocardial blood flow at locations just below individual epicardial vessel to help guide revascularization. The aim of this work is to determine the robustness of vessel-specific flow measurements (MBFvs) extracted from the fusion of dynamic PET (dPET) with coronary computed tomography angiography (CCTA) myocardial segmentations, using flow measured from the fusion with CCTA manual segmentation as the reference standard. Methods: Forty-three patients' 13NH3 dPET, CCTA image datasets were used to measure the agreement of the MBFvs profiles after the fusion of dPET data with three CCTA anatomical models: (1) a manual model, (2) a fully automated segmented model and (3) a corrected model, where major inaccuracies in the automated segmentation were briefly edited. Pairwise accuracy of the normality/abnormality agreement of flow values along differently extracted vessels was determined by comparing, on a point-by-point basis, each vessel's flow to corresponding vessels' normal limits using Dice coefficients (DC) as the metric. Results: Of the 43 patients CCTA fully automated mask models, 27 patients' borders required manual correction before dPET/CCTA image fusion, but this editing process was brief (2–3 min) allowing a 100% success rate of extracting MBFvs in clinically acceptable times. In total, 124 vessels were analyzed after dPET fusion with the manual and corrected CCTA mask models yielding 2225 stress and 2122 rest flow values. Forty-seven vessels were analyzed after fusion with the fully automatic masks producing 840 stress and 825 rest flow samples. All DC coefficients computed globally or by territory were ≥ 0.93. No statistical differences were found in the normal/abnormal flow classifications between manual and corrected or manual and fully automated CCTA masks. Conclusion: Fully automated and manually corrected myocardial CCTA segmentation provides anatomical masks in clinically acceptable times for vessel-specific myocardial blood flow measurements using dynamic PET/CCTA image fusion which are not significantly different in flow accuracy and within clinically acceptable processing times compared to fully manually segmented CCTA myocardial masks. [ABSTRACT FROM AUTHOR]

Background: The effective non-invasive identification of coronary artery disease (CAD) and its proper referral for invasive treatment are still unresolved issues. We evaluated our quantification of myocardium at risk (MAR) from our second generation 3D MPI/CTA fusion framework for the detection and localization of obstructive coronary disease.Methods: Studies from 48 patients who had rest/stress MPI, CTA, and ICA were analyzed from 3 different institutions. From the CTA, a 3D biventricular surface of the myocardium with superimposed coronaries was extracted and fused to the perfusion distribution. Significant lesions were identified from CTA readings and positioned on the fused display. Three estimates of MAR were computed on the 3D LV surface on the basis of the MPI alone (MARp), the CTA alone (MARa), and the fused information (MARf). The extents of areas at risk were used to generate ROC curves using ICA anatomical findings as reference standard.Results: Areas under the ROC curve (AUC) for CAD detection using MARf was 0.88 (CI = 0.75-0.95) and for MARp and MARa were, respectively 0.82 (CI = 0.69-0.92) and 0.75 (CI = 0.60-0.86) using the ≥70% stenosis criterion. AUCs for CAD localization (all vessels) using MARf showed significantly higher performance than either MARa or MARp or both.Conclusions: Using ICA as the reference standard, MAR as the quantitative parameter, and AUC to measure diagnostic performance, MPI-CTA fusion imaging provided incremental diagnostic information compared to MPI or CTA alone for the diagnosis and localization of CAD. [ABSTRACT FROM AUTHOR]

The combination of different modality images can provide detailed and comprehensive information for the prognostic assessment and therapeutic strategy of patients with ischemic heart disease. In this study, a 3D fusion framework is designed to integrate coronary computed tomography (CT) angiography (CTA), 2-deoxy-2-[18F]fluoro-D-glucose ([18F]DG) positron emission tomography (PET)/CT, and [68Ga]-1,4,7-triazacyclononane-1,4,7-triacetic acid-(Arg-Gly-Asp)2 ([68Ga]-NOTA-PRGD2) PET/CT images of the myocardial infarction model in minipigs. First, the structural anatomy of the heart in coronary CTA and CT is segmented using a multi-atlas-based method. Then, the hearts are registered using the B-spline-based free form deformation. Finally, the [18F]DG and [68Ga]-NOTA-PRGD2 signals are mapped into the heart in coronary CTA, which produces a single fusion image to delineate both the cardiac structural anatomy and the functional information of myocardial viability and angiogenesis. Heart segmentation demonstrates high accuracy with good agreement between manual delineation and automatic segmentation. The fusion result intuitively reflects the extent of the [18F]DG uptake defect as well as the location where the [68Ga]-NOTAPRGD2 signal appears. The fusion result verified the occurrence of angiogenesis based on the in vivo noninvasive molecular imaging approach. The presented framework is helpful in facilitating the study of the relationship between infarct territories and blocked coronary arteries as well as angiogenesis. [ABSTRACT FROM AUTHOR]

Background: Conventional laparoscopes provide a flat representation of the three-dimensional (3D) operating field and are incapable of visualizing internal structures located beneath visible organ surfaces. Computed tomography (CT) and magnetic resonance (MR) images are difficult to fuse in real time with laparoscopic views due to the deformable nature of soft-tissue organs. Utilizing emerging camera technology, we have developed a real-time stereoscopic augmented-reality (AR) system for laparoscopic surgery by merging live laparoscopic ultrasound (LUS) with stereoscopic video. The system creates two new visual cues: (1) perception of true depth with improved understanding of 3D spatial relationships among anatomical structures, and (2) visualization of critical internal structures along with a more comprehensive visualization of the operating field. Methods: The stereoscopic AR system has been designed for near-term clinical translation with seamless integration into the existing surgical workflow. It is composed of a stereoscopic vision system, a LUS system, and an optical tracker. Specialized software processes streams of imaging data from the tracked devices and registers those in real time. The resulting two ultrasound-augmented video streams (one for the left and one for the right eye) give a live stereoscopic AR view of the operating field. The team conducted a series of stereoscopic AR interrogations of the liver, gallbladder, biliary tree, and kidneys in two swine. Results: The preclinical studies demonstrated the feasibility of the stereoscopic AR system during in vivo procedures. Major internal structures could be easily identified. The system exhibited unobservable latency with acceptable image-to-video registration accuracy. Conclusions: We presented the first in vivo use of a complete system with stereoscopic AR visualization capability. This new capability introduces new visual cues and enhances visualization of the surgical anatomy. The system shows promise to improve the precision and expand the capacity of minimally invasive laparoscopic surgeries. [ABSTRACT FROM AUTHOR]

Objectives: To evaluate the effect of intraoperative guidance by means of live fluoroscopy image fusion with computed tomography angiography (CTA) on iodinated contrast material volume, procedure time, and fluoroscopy time in endovascular thoraco-abdominal aortic repair. Methods: CTA with fluoroscopy image fusion road-mapping was prospectively evaluated in patients with complex aortic aneurysms who underwent fenestrated and/or branched endovascular repair (FEVAR/BEVAR). Total iodinated contrast material volume, overall procedure time, and fluoroscopy time were compared between the fusion group (n = 31) and case controls (n = 31). Reasons for potential fusion image inaccuracy were analyzed. Results: Fusion imaging was feasible in all patients. Fusion image road-mapping was used for navigation and positioning of the devices and catheter guidance during access to target vessels. Iodinated contrast material volume and procedure time were significantly lower in the fusion group than in case controls (159 mL [95% CI 132–186 mL] vs. 199 mL [95% CI 170–229 mL], p = .037 and 5.2 hours [95% CI 4.5–5.9 hours] vs. 6.3 hours (95% CI 5.4–7.2 hours), p = .022). No significant differences in fluoroscopy time were observed (p = .38). Respiration-related vessel displacement, vessel elongation, and displacement by stiff devices as well as patient movement were identified as reasons for fusion image inaccuracy. Conclusion: Image fusion guidance provides added value in complex endovascular interventions. The technology significantly reduces iodinated contrast material dose and procedure time. [Copyright &y& Elsevier]

Abstract: In most of complex wavelet based fusion methods, only magnitude (or absolute value) of a complex coefficient is considered and phase information is neglected. However, more salient image features can be determined by the phase. In this paper, a multimodality image fusion algorithm is proposed with the shiftable complex directional pyramid transform (SCDPT), where phase and magnitudes of complex coefficients are jointly considered. Firstly, a novel similarity index (CCC-EM) is presented by combining the circular correlation coefficient (CCC) of relative phase angles and the traditional energy matching (EM) index. When bandpass directional subband coefficients are merged, the CCC-EM index is employed as the similarity measure and three types of regions between source images are determined for each bandpass directional subband. Then, based on some weights or salience measures, different fusion rules are designed for each type of regions. Especially, for regions with similarity in energy and positive or negative correlation relationship in relative phase, the weighted circular variance (WCV) of relative phase angles is employed. When lowpass subband coefficients are merged, the traditional structural similarity index is employed to distinguish different types of regions. For most of regions, the local energy of lowpass subband coefficients is employed as weights or salience measures. While for regions with similar intensity values but different intensity variation directions, an inter-scale based salience measure is defined by combining the local energy of the lowpass subband coefficients and the WCV of the coarsest bandpass directional subband coefficients. Several pairs of multimodality images are fused with the proposed methods. Fusion results demonstrate that the proposed fusion method can extract more salient features (not just in energy) from source images than some other complex wavelet based fusion methods. Especially, more phase information of source images can be preserved into the fused image, which makes the proposed fusion method with higher performance in spatial consistency. [Copyright &y& Elsevier]

Abstract: For multimodality images, a novel fusion algorithm based on the shiftable complex directional pyramid transform (SCDPT) is proposed in this paper. As well, with the aid of the structural similarity (SSIM) index, a ‘similarity-based’ idea is employed to distinguish regions with ‘redundant’ or ‘complementary’ information between source imagers before the SCDPT coefficients are merged. A ‘weighted averaging’ scheme for regions with ‘redundant’ information and a ‘selecting’ scheme for regions with ‘complementary’ information are then employed, respectively. When merging the low-pass subband coefficients, the SSIM index in spatial domain (SP-SSIM) is employed as similarity measure, and three types of regions are thus determined. Especially, for regions with similar intensity values but different intensity changing directions between source images, a ‘selecting’ scheme based on gradient and energy is proposed. When merging the directional band-pass subband coefficients, the SSIM index in complex wavelet domain (CW-SSIM) is employed as similarity measure. With the CW-SSIM index, not only the magnitude information but also the phase information of SCDPT coefficients can be employed. Compared to the traditional energy matching (EM) index based fusion methods, the proposed method can better deal with ‘redundant’ and ‘complementary’ information of source images. In addition, because of the shift-invariance of the SCDPT and the CW-SSIM index, the proposed fusion algorithm performs well even if the input images are not well registered. Several sets of experimental results demonstrate the validity and feasibility of the proposed method in terms of both visual quality and objective evaluation. [Copyright &y& Elsevier]

We present a novel multimodality image registration system for spinal surgery. The system comprises a surface-based algorithm that performs computed tomography/magnetic resonance (CT/MR) rigid registration and MR image segmentation in an iterative manner. The segmentation/registration process progressively refines the result of MR image segmentation and CT/MR registration. For MR image segmentation, we propose a method based on the double-front level set that avoids boundary leakages, prevents interference from other objects in the image, and reduces computational time by constraining the search space. In order to reduce the registration error from the misclassification of the soft tissue surrounding the bone in MR images, we propose a weighted surface-based CT/MR registration scheme. The resultant weighted surface is registered to the segmented surface of the CT image. Contours are generated from the reconstructed CT surfaces for subsequent MR image segmentation. This process iterates till convergence. The registration method achieves accuracy comparable to conventional techniques while being significantly faster. Experimental results demonstrate the advantages of the proposed approach and its application to different anatomies. [ABSTRACT FROM AUTHOR]

Objectives: We developed and tested a novel transcatheter circumferential annuloplasty technique to reduce mitral regurgitation in porcine ischemic cardiomyopathy. Background: Catheter-based annuloplasty for secondary mitral regurgitation exploits the proximity of the coronary sinus to the mitral annulus, but is limited by anatomic variants and coronary artery entrapment. Methods: The procedure, “cerclage annuloplasty,” is guided by magnetic resonance imaging (MRI) roadmaps fused with live X-ray. A coronary sinus guidewire traverses a short segment of the basal septal myocardium to re-enter the right heart where it is exchanged for a suture. Tension is applied interactively during imaging and secured with a locking device. Results: We found 2 feasible suture pathways from the great cardiac vein across the interventricular septum to create cerclage. Right ventricular septal re-entry required shorter fluoroscopy times than right atrial re-entry, which entailed a longer intramyocardial traversal but did not cross the tricuspid valve. Graded tension progressively reduced septal-lateral annular diameter, but not end-systolic elastance or regional myocardial function. A simple arch-like device protected entrapped coronary arteries from compression even during supratherapeutic tension. Cerclage reduced mitral regurgitation fraction (from 22.8 ± 12.7% to 7.2 ± 4.4%, p = 0.04) by slice tracking velocity-encoded MRI. Flexible cerclage reduced annular size but preserved annular motion. Cerclage also displaced the posterior annulus toward the papillary muscles. Cerclage introduced reciprocal constraint to the left ventricular outflow tract and mitral annulus that enhanced leaflet coaptation. A sample of human coronary venograms and computed tomography angiograms suggested that most have suitable venous anatomy for cerclage. Conclusions: Transcatheter mitral cerclage annuloplasty acutely reduces mitral regurgitation in porcine ischemic cardiomyopathy. Entrapped coronary arteries can be protected. MRI provided insight into the mechanism of cerclage action. [Copyright &y& Elsevier]

Objectives: We hypothesized that X-ray fused with magnetic resonance imaging (XFM) roadmaps might permit direct antegrade crossing and delivery of a ventricular septal defect (VSD) closure device and thereby reduce procedure time and radiation exposure. Background: Percutaneous device closure of membranous VSD is cumbersome and time-consuming. The procedure requires crossing the defect retrograde, snaring and exteriorizing a guidewire to form an arteriovenous loop, then delivering antegrade a sheath and closure device. Methods: Magnetic resonance imaging roadmaps of cardiac structures were obtained from miniature swine with spontaneous VSD and registered with live X-ray using external fiducial markers. We compared antegrade XFM-guided VSD crossing with conventional retrograde X-ray–guided crossing for repair. Results: Antegrade XFM crossing was successful in all animals. Compared with retrograde X-ray, antegrade XFM was associated with shorter time to crossing (167 ± 103 s vs. 284 ± 61 s; p = 0.025), shorter time to sheath delivery (71 ± 32 s vs. 366 ± 145 s; p = 0.001), shorter fluoroscopy time (158 ± 95 s vs. 390 ± 137 s; p = 0.003), and reduced radiation dose–area product (2,394 ± 1,522 mG·m2 vs. 4,865 ± 1,759 mG·m2; p = 0.016). Conclusions: XFM facilitates antegrade access to membranous VSD from the right ventricle in swine. The simplified procedure is faster and reduces radiation exposure compared with the conventional retrograde approach. [Copyright &y& Elsevier]

Objective: To investigate the clinical application of multimodality navigation for liver resection in the treatment of complicated alveolar echinococcosis (AE). Methods: From October 2019 to February 2020, the clinical data and perioperative results of patients with AE treated by surgery in our department were retrospectively studied. Hepatic parenchyma disconnection plane and liver resection were navigated and performed with three-dimensional reconstruction and HITACHI real-time multi-image fusion interventional navigation system (RVS). Results: All of six patients were successful performed radical liver resection without mortality. The operation time was (301±106)min and the median blood loss was 200 ml. Two patients needed blood transfusion intraoperative (33.33%). The postoperative hospital stay was (10.8±2.8) day, and the cost of hospitalization was (82 584±995.61) yuan. Clavien-Dindo grade Ⅲ complication occurred in one patient. Conclusions: Multimodality navigation might provide precise intraoperative navigation of the surgical plane and effectively assist liver resection for the treatment of complicated AE.