Search

• A novel anatomy-preserved GAN (ApGAN) is developed for generating the simulated intraoperative ultrasound of liver without any preprocessing steps. • The simulated intraoperative ultrasound has high-level precision of anatomical structures and focal areas. • Empirical results of clinical data showed that the proposed method obtained higher accuracy compared to existing methods. In ultrasound-guided liver surgery, the lack of large-scale intraoperative ultrasound images with important anatomical structures remains an obstacle hindering the successful application of AI to ultrasound guidance. In this case, intraoperative ultrasound (iUS) simulation should be conducted from preoperative magnetic resonance (pMR), which not only helps doctors understand the characteristics of iUS in advance, but also expands the iUS dataset from various imaging positions, thereby promoting the automatic iUS analysis in ultrasound guidance. Herein, a novel anatomy preserving generative adversarial network (ApGAN) framework was proposed to generate simulated intraoperative ultrasound (Sim-iUS) of liver with precise structure information from pMR. Specifically, the low-rank factors based bimodal fusion was first established focusing on the effective information of hepatic parenchyma. Then, a deformation field based correction module was introduced to learn and correct the slight structural distortion from surgical operations. Meanwhile, the multiple loss functions were designed to constrain the simulation of the content, structures, and style. Empirical results of clinical data showed that the proposed ApGAN obtained higher Structural Similarity (SSIM) of 0.74 and Fr´echet Inception Distance (FID) of 35.54 compared to existing methods. Furthermore, the average Hausdorff Distance (HD) error of the liver capsule structure was less than 0.25 mm, and the average relative (Euclidean Distance) ED error for polyps was 0.12 mm, indicating the high-level precision of this ApGAN in simulating the anatomical structures and focal areas. [ABSTRACT FROM AUTHOR]

Complete removal of malignant gliomas is important for the prognosis in neurosurgery treatment. Currently, the challenge is how to detect any remaining tumors and resect them during the operation. We have developed a laser ablation system with accurate tumor analysis and fluorescence guidance for high-precision brain tumor resection during neurosurgery. A 5-aminolevulinic acid-induced fluorescent protoporphyrins IX (PpIX)-based intra-operative fluorescence measurement and corresponding spectra analysis technique is used to identify the position of tumors. A galvano mirror scanning mechanism is integrated into the fluorescence measurement and the laser ablation devices for automatic tumor area scanning and corresponding laser ablation. A set of phantom experiments was performed to evaluate the proposed system. Results showed that the galvano scanning mechanism enabled both PpIX fluorescence detection and laser ablation in the same optical axis. In vitro experiments using porcine brain were performed to evaluate the effectiveness of the automatic laser scanning, fluorescence detection, and laser ablation system. The proposed fluorescence-guided laser ablation system can provide accurate analysis and high-precision treatment for tumor resection in neurosurgery. With further improvement, the system can be used in neurosurgical implementation to provide accurate, safe, and simple surgical diagnosis and therapy. [ABSTRACT FROM AUTHOR]

Abstract: Geometric flux maximizing flow (FLUX) is an active contour based method which evolves an initial surface to maximize the flux of a vector field on the surface. For blood vessel segmentation, the vector field is defined as the vectors specified by vascular edge strengths and orientations. Hence, the segmentation performance depends on the quality of the detected edge vector field. In this paper, we propose a new method for level set based segmentation of blood vessels by generalizing the FLUX on a Riemannian manifold (R-FLUX). We consider a 3D scalar image I(x) as a manifold embedded in the 4D space (x, I(x)) and compute the image metric by pullback from the 4D space, whose metric tensor depends on the vessel enhancing diffusion (VED) tensor. This allows us to devise a non-linear filter which both projects and normalizes the original image gradient vectors under the inverse of local VED tensors. The filtered gradient vectors pertaining to the vessels are less sensitive to the local image contrast and more coherent with the local vessel orientation. The method has been applied to both synthetic and real TOF MRA data sets. Comparisons are made with the FLUX and vesselsness response based segmentations, indicating that the R-FLUX outperforms both methods in terms of leakage minimization and thiner vessel delineation. [Copyright &y& Elsevier]

Abstract: We have developed an integrated diagnosis and therapeutic system for precision malignant gliomas resection during neurosurgery. A combination of three-dimensional (3-D) magnetic resonance imaging (MRI) navigation and 5-aminolevulinic acid (5-ALA)-induced fluorescence based intra-operative tumor diagnosis technique has been incorporated into a robotic laser ablation neurosurgery system with an automatic focusing and robotic scanning mechanism. 5-ALA is a non-fluorescent prodrug that leads to intracellular accumulation of fluorescent protoporphyrins IX (PpIX) in malignant glioma. The PpIX tends to accumulate in pathological lesions, and emits red fluorescence when excited by blue light. This fluorescence is illuminated with laser excitation, enables intra-operative identification of the position of a tumor and provides guidance for resection with laser photocoagulation. The information provided by the MRI is enhanced by the intra-operative 5-ALA fluorescence data, and this enhanced information is integrated into a robotic laser ablation system. The accuracy of the fluorescent measurement of the tumor is improved using high-precision spectral analysis. The fluorescence assists in the detection of malignant brain tumors intra-operatively and improves their removal rate. [Copyright &y& Elsevier]

Abstract: This paper describes a precision-guided surgical navigation system for minimally invasive surgery. The system combines a laser guidance technique with a three-dimensional (3D) autostereoscopic image overlay technique. Images of surgical anatomic structures superimposed onto the patient are created by employing an animated imaging method called integral videography (IV), which can display geometrically accurate 3D autostereoscopic images and reproduce motion parallax without the need for special viewing or tracking devices. To improve the placement accuracy of surgical instruments, we integrated an image overlay system with a laser guidance system for alignment of the surgical instrument and better visualization of patient''s internal structure. We fabricated a laser guidance device and mounted it on an IV image overlay device. Experimental evaluations showed that the system could guide a linear surgical instrument toward a target with an average error of 2.48mm and standard deviation of 1.76mm. Further improvement to the design of the laser guidance device and the patient-image registration procedure of the IV image overlay will make this system practical; its use would increase surgical accuracy and reduce invasiveness. [Copyright &y& Elsevier]

This paper describes a fast image mapping system that integrates endoscopic image mosaics with three-dimensional (3-D) ultrasound images for assisting intrauterine laser photocoagulation treatment. Endoscopic laser photocoagulation treatment has a good survival rate and a low complication rate for twins. However, the small field of view and lack of surrounding information makes the identification of vessel anastomosis difficult. We have developed an extended visualization system with the fusion of endoscopic image mosaics with a 3-D ultrasound-image model. Fully automatic and fast calibration is used for endoscope calibration in fluid. The 3-D spatial position of the endoscopic images and the ultrasound image are tracked by a 3-D position tracking device. The mosaiced endoscope images are registered to the surface of the 3-D ultrasound placenta model by using a fast GPU-based image rendering method and a seamless multi-images processing. Experimental results show that the system may provide an improved and efficient way of planning and guidance in laser photocoagulation treatment. [ABSTRACT FROM AUTHOR]

Diffusion magnetic resonance imaging is an important tool for mapping tissue microstructure and structural connectivity non‐invasively in the in vivo human brain. Numerous diffusion signal models are proposed to quantify microstructural properties. Nonetheless, accurate estimation of model parameters is computationally expensive and impeded by image noise. Supervised deep learning‐based estimation approaches exhibit efficiency and superior performance but require additional training data and may be not generalizable. A new DIffusion Model OptimizatioN framework using physics‐informed and self‐supervised Deep learning entitled "DIMOND" is proposed to address this problem. DIMOND employs a neural network to map input image data to model parameters and optimizes the network by minimizing the difference between the input acquired data and synthetic data generated via the diffusion model parametrized by network outputs. DIMOND produces accurate diffusion tensor imaging results and is generalizable across subjects and datasets. Moreover, DIMOND outperforms conventional methods for fitting sophisticated microstructural models including the kurtosis and NODDI model. Importantly, DIMOND reduces NODDI model fitting time from hours to minutes, or seconds by leveraging transfer learning. In summary, the self‐supervised manner, high efficacy, and efficiency of DIMOND increase the practical feasibility and adoption of microstructure and connectivity mapping in clinical and neuroscientific applications. [ABSTRACT FROM AUTHOR]

Objectives: To investigate the value of diffusion MRI (dMRI) in H3K27M genotyping of brainstem glioma (BSG). Methods: A primary cohort of BSG patients with dMRI data (b = 0, 1000 and 2000 s/mm2) and H3K27M mutation information were included. A total of 13 diffusion tensor and kurtosis imaging (DTI; DKI) metrics were calculated, then 17 whole-tumor histogram features and 29 along-tract white matter (WM) microstructural measurements were extracted from each metric and assessed within genotypes. After feature selection through univariate analysis and the least absolute shrinkage and selection operator method, multivariate logistic regression was used to build dMRI-derived genotyping models based on retained tumor and WM features separately and jointly. Model performances were tested using ROC curves and compared by the DeLong approach. A nomogram incorporating the best-performing dMRI model and clinical variables was generated by multivariate logistic regression and validated in an independent cohort of 27 BSG patients. Results: At total of 117 patients (80 H3K27M-mutant) were included in the primary cohort. In total, 29 tumor histogram features and 41 WM tract measurements were selected for subsequent genotyping model construction. Incorporating WM tract measurements significantly improved diagnostic performances (p < 0.05). The model incorporating tumor and WM features from both DKI and DTI metrics showed the best performance (AUC = 0.9311). The nomogram combining this dMRI model and clinical variables achieved AUCs of 0.9321 and 0.8951 in the primary and validation cohort respectively. Conclusions: dMRI is valuable in BSG genotyping. Tumor diffusion histogram features are useful in genotyping, and WM tract measurements are more valuable in improving genotyping performance. Clinical relevance statement: This study found that diffusion MRI is valuable in predicting H3K27M mutation in brainstem gliomas, which is helpful to realize the noninvasive detection of brainstem glioma genotypes and improve the diagnosis of brainstem glioma. Key Points: • Diffusion MRI has significant value in brainstem glioma H3K27M genotyping, and models with satisfactory performances were built. • Whole-tumor diffusion histogram features are useful in H3K27M genotyping, and quantitative measurements of white matter tracts are valuable as they have the potential to improve model performance. • The model combining the most discriminative diffusion MRI model and clinical variables can help make clinical decision. [ABSTRACT FROM AUTHOR]

Due to the limitations of the lens array of the integral imaging display, the viewing angle and resolution are mutually restricted in space, and are difficult to improve at the same time. We propose an integral imaging system based on dual time division multiplexing using a liquid crystal aperture array to improve viewing angle and resolution simultaneously. A liquid crystal panel is introduced into the traditional integral imaging display system as an adjustable aperture array, which is used to change the imaging effect of the micro‐lens array and to realize the dual time division multiplexing of elemental image pixels. The liquid crystal aperture array is controlled to gate different lenses and different positions on the lens at different times, then the corresponding elemental image array is rendered. Thus, the spatial constraints of viewing angle and resolution are transformed into the time domains by multiplexing. Based on the proposed scheme, this paper also proposes a synchronous rendering algorithm for elemental image array and aperture array patterns in the different multiplexing periods. To verify the feasibility of the scheme, we built a prototype display and the experimental results verified that our method achieved certain improvements in viewing angle and resolution. [ABSTRACT FROM AUTHOR]

Objective: In this paper, we introduce an autonomous robotic ultrasound (US) imaging system based on reinforcement learning (RL). The proposed system and framework are committed to controlling the US probe to perform fully autonomous imaging of a soft, moving and marker-less target based only on single RGB images of the scene. Methods: We propose several different approaches and methods to achieve the following objectives: real-time US probe controlling, soft surface constant force tracking and automatic imaging. First, to express the state of the robotic US imaging task, we proposed a state representation model to reduce the dimensionality of the imaging state and encode the force and US information into the scene image space. Then, an RL agent is trained by a policy gradient theorem based RL model with the single RGB image as the only observation. To achieve adaptable constant force tracking between the US probe and the soft moving target, we propose a force-to-displacement control method based on an admittance controller. Results: In the simulation experiment, we verified the feasibility of the integrated method. Furthermore, we evaluated the proposed force-to-displacement method to demonstrate the safety and effectiveness of adaptable constant force tracking. Finally, we conducted phantom and volunteer experiments to verify the feasibility of the method on a real system. Conclusion: The experiments indicated that our approaches were stable and feasible in the autonomic and accurate control of the US probe. Significance: The proposed system has potential application value in the image-guided surgery and robotic surgery. [ABSTRACT FROM AUTHOR]

Objectives: The histologic subtype of intracranial germ cell tumours (IGCTs) is an important factor in deciding the treatment strategy, especially for teratomas. In this study, we aimed to non-invasively diagnose teratomas based on fractal and radiomic features. Materials and methods: This retrospective study included 330 IGCT patients, including a discovery set (n = 296) and an independent validation set (n = 34). Fractal and radiomic features were extracted from T1-weighted, T2-weighted, and post-contrast T1-weighted images. Five classifiers, including logistic regression, random forests, support vector machines, K-nearest neighbours, and XGBoost, were compared for our task. Based on the optimal classifier, we compared the performance of clinical, fractal, and radiomic models and the model combining these features in predicting teratomas. Results: Among the diagnostic models, the fractal and radiomic models performed better than the clinical model. The final model that combined all the features showed the best performance, with an area under the curve, precision, sensitivity, and specificity of 0.946 [95% confidence interval (CI): 0.882–0.994], 95.65% (95% CI: 88.64–100%), 88.00% (95% CI: 77.78–96.36%), and 91.67% (95% CI: 78.26–100%), respectively, in the test set of the discovery set, and 0.944 (95% CI: 0.855–1.000), 85.71% (95% CI: 68.18–100%), 94.74% (95% CI: 83.33–100%), and 80.00% (95% CI: 58.33–100%), respectively, in the independent validation set. SHapley Additive exPlanations indicated that two fractal features, two radiomic features, and age were the top five features highly associated with the presence of teratomas. Conclusion: The predictive model including image and clinical features could help guide treatment strategies for IGCTs. Clinical relevance statement: Our machine learning model including image and clinical features can non-invasively predict teratoma components, which could help guide treatment strategies for intracranial germ cell tumours (IGCT). Key Points: • Fractals and radiomics can quantitatively evaluate imaging characteristics of intracranial germ cell tumours. • Model combing imaging and clinical features had the best predictive performance. • The diagnostic model could guide treatment strategies for intracranial germ cell tumours. [ABSTRACT FROM AUTHOR]

Floating three‐dimensional (3D) display implements direct interaction between human hands and virtual 3D images, which offers natural and effective augmented reality interaction. In this study, we propose a novel floating autostereoscopic display, combining head tracking lenticular display with an image projection system, to offer the observers with an accurate 3D image floating in midair without any optical elements between observers and the virtual 3D image. Combined with a gesture recognition device, the proposed system can achieve in situ augmented reality interaction with the floating 3D image. A distortion correction method is developed to achieve 3D display with accurate spatial information. Moreover, a coordinate calibration method is designed to improve the accuracy in the in situ interaction. Experiments were performed to prove the feasibility of the proposed system, and the good results show the potential of human‐computer interaction in medicine and life sciences. [ABSTRACT FROM AUTHOR]

Objective: Precise minimally invasive surgery (MIS) has significant advantages over traditional open surgery in clinic. Although pre-/intraoperative diagnosis images can provide necessary guidance for therapy, hand–eye discoordination occurs when guidance information is displayed away from the surgical area. In this study, we introduce a real three-dimensional (3-D) see-through guidance system for precision surgery. Methods: To address the resolution and viewing angle limitation as well as the accuracy degradation problems of autostereoscopic 3-D display, we design a high quality and high accuracy 3-D integral videography (IV) medical image display method. Furthermore, a novel see-through microscopic device is proposed to assist surgeons with the superimposition of real 3-D guidance onto the surgical target is magnified by an optical visual magnifier module. Results: Spatial resolutions of 3-D IV image in different depths have been increased 50%∼70%, viewing angles of different image sizes have been increased 9%∼19% compared with conventional IV display methods. Average accuracy of real 3-D guidance superimposed on surgical target was 0.93 mm ± 0.41 mm. Preclinical studies demonstrated that our system could provide real 3-D perception of anatomic structures inside the patient's body. Conclusion: The system showed potential clinical feasibility to provide intuitive and accurate in situ see-through guidance for microsurgery without restriction on observers’ viewing position. Significance: Our system can effectively improve the precision and reliability of surgical guidance. It will have wider applicability in surgical planning, microscopy, and other fields. [ABSTRACT FROM PUBLISHER]

In endovascular catheter interventions, the determination of the three-dimensional (3D) catheter shape can increase navigation information and help reduce trauma. This study describes a shape determination method for a flexible interventional catheter using ultrasound scanning and a two-step particle filter without X-ray fluoroscopy. First, we propose a multi-feature, multi-template particle filter algorithm for accurate catheter tracking from ultrasound images. Second, we model the mechanical behavior of the catheter and apply a particle filter shape optimization algorithm to refine the results from the first step. Finally, the acquired catheter’s 3D shapes are displayed together with the preoperative 3D images of the cardiac structures to provide intuitive endovascular navigation. We validated our method using ultrasound scanning of the straight and curved catheters in a water tank, and the shape determination errors were 1.44 ± 0.38 mm and 1.95 ± 0.46 mm, respectively. Further, endovascular catheter shape determination was validated in a catheter intervention experiment with a heart phantom. The error of the acquired endovascular catheter shape was 2.23 ± 0.87 mm. These results demonstrate that our two-step method is both accurate and effective. Using ultrasound scanning for shape determination of a flexible catheter will be helpful in endovascular interventions, reducing exposure to radiation and providing rich navigation information. [ABSTRACT FROM AUTHOR]

3D segmentation of lower-limb bones such as tibia and fibula from volumetric computed tomography (CT) is very important for surgical planning and navigation. However, the input data of the large-scale 3D volumetric CT will increase the computation cost, and meantime, the spatial connections of the long-range voxels will be easily ignored. To solve these problems, this paper proposes an accurate and efficient 3D bone segmentation approach based on the distribution characteristics of the lower-limb bones in volumetric CT. On one hand, the efficient segmentation framework was built for the large-scale volumetric CT using 2D projection view-based slice filtering and parameter-reduced separable convolution. On the other hand, we developed a new voxel group attention mechanism to emphasize the connection of the long-range voxel groups and improve the representational capability of the segmentation network. The experimental results showed that the proposed 3D bone segmentation approach achieved high segmentation accuracy under the conditions of limited computations. It was additionally shown that the proposed approach outperformed the state-of-the-art 3D models for the segmentation of lower-limb bones. [ABSTRACT FROM AUTHOR]

Autostereoscopic 3D image overlay for augmented reality (AR) based surgical navigation has been studied and reported many times. For the purpose of surgical overlay, the 3D image is expected to have the same geometric shape as the original organ, and can be transformed to a specified location for image overlay. However, how to generate a 3D image with high geometric fidelity and quantitative evaluation of 3D image's geometric accuracy have not been addressed. This paper proposes a graphics processing unit (GPU) based computer-generated integral imaging pipeline for real-time autostereoscopic 3D display, and an automatic closed-loop 3D image calibration paradigm for displaying undistorted 3D images. Based on the proposed methods, a novel AR device for 3D image surgical overlay is presented, which mainly consists of a 3D display, an AR window, a stereo camera for 3D measurement, and a workstation for information processing. The evaluation on the 3D image rendering performance with 2560 × 1600 elemental image resolution shows the rendering speeds of 50–60 frames per second (fps) for surface models, and 5–8 fps for large medical volumes. The evaluation of the undistorted 3D image after the calibration yields sub-millimeter geometric accuracy. A phantom experiment simulating oral and maxillofacial surgery was also performed to evaluate the proposed AR overlay device in terms of the image registration accuracy, 3D image overlay accuracy, and the visual effects of the overlay. The experimental results show satisfactory image registration and image overlay accuracy, and confirm the system usability. [ABSTRACT FROM AUTHOR]

Establishing correct correspondences between two images should consider both local and global spatial context. Given putative correspondences of feature points in two views, in this paper, we propose Order-Aware Network, which infers the probabilities of correspondences being inliers and regresses the relative pose encoded by the essential or fundamental matrix. Specifically, this proposed network is built hierarchically and comprises three operations. First, to capture the local context of sparse correspondences, the network clusters unordered input correspondences by learning a soft assignment matrix. These clusters are in canonical order and invariant to input permutations. Next, the clusters are spatially correlated to encode the global context of correspondences. After that, the context-encoded clusters are interpolated back to the original size and position to build a hierarchical architecture. We intensively experiment on both outdoor and indoor datasets. The accuracy of the two-view geometry and correspondences are significantly improved over the state-of-the-arts. Besides, based on the proposed method and advanced local feature, we won the first place in CVPR 2019 image matching workshop challenge and also achieve state-of-the-art results in the Visual Localization benchmark. Code is available at https://github.com/zjhthu/OANet. [ABSTRACT FROM AUTHOR]

Recently, deep learning-based brain segmentation methods have achieved great success. However, most approaches focus on supervised segmentation, which requires many high-quality labeled images. In this paper, we pay attention to one-shot segmentation, aiming to learn from one labeled image and a few unlabeled images. We propose an end-to-end unified network that joints deformation modeling and segmentation tasks. Our network consists of a shared encoder, a deformation modeling head, and a segmentation head. In the training phase, the atlas and unlabeled images are input to the encoder to get multi-scale features. The features are then fed to the multi-scale deformation modeling module to estimate the atlas-to-image deformation field. The deformation modeling module implements the estimation at the feature level in a coarse-to-fine manner. Then, we employ the field to generate the augmented image pair through online data augmentation. We do not apply any appearance transformations cause the shared encoder could capture appearance variations. Finally, we adopt supervised segmentation loss for the augmented image. Considering that the unlabeled images still contain rich information, we introduce confidence aware pseudo label for them to further boost the segmentation performance. We validate our network on three benchmark datasets. Experimental results demonstrate that our network significantly outperforms other deep single-atlas-based and traditional multi-atlas-based segmentation methods. Notably, the second dataset is collected from multi-center, and our network still achieves promising segmentation performance on both the seen and unseen test sets, revealing its robustness. The source code will be available at https://github.com/zhangliutong/brainseg. [Display omitted] • Propose an end-to-end network for one-shot neuroanatomy segmentation. • Develop online data augmentation to generate diverse labeled image pairs. • Introduce confidence aware pseudo label to leverage the unlabeled images. • Experimental results show our network consistently outperforms other methods. [ABSTRACT FROM AUTHOR]

Lateral motion estimation has been a challenge in ultrasound elastography mainly due to the low resolution, low sampling frequency, and lack of phase information in the lateral direction. Synthetic transmit aperture (STA) can achieve high resolution due to two-way focusing and can beamform high-density image lines for improved lateral motion estimation with the disadvantages of low signal-to-noise ratio (SNR) and limited penetration depth. In this study, Hadamard-encoded STA (Hadamard-STA) is proposed for the improvement of lateral motion estimation in elastography, and it is compared with STA and conventional focused wave (CFW) imaging. Simulations, phantom, and in vivo experiments were conducted to make the comparison. The normalized root mean square error (NRMSE) and the contrast-to-noise ratio (CNR) were calculated as the evaluation criteria in the simulations. The results show that, at a noise level of −10 dB and an applied strain of −1% (compression), Hadamard-STA decreases the NRMSEs of lateral displacements by 46.92% and 35.35%, decreases the NRMSEs of lateral strains by 52.34% and 39.75%, and increases the CNRs by 9.70 and 9.75 dB compared with STA and CFW, respectively. In the phantom experiments performed on a heterogeneous tissue-mimicking phantom, the sum of squared differences (SSD) between the reference and the motion-compensated RF data, and the CNR were calculated as the evaluation criteria. At an applied strain of −1.80%, Hadamard-STA is found to decrease the SSDs by 20.91% and 30.99% and increase the CNRs by 14.15 and 24.66 dB compared with STA and CFW, respectively. In the experiments performed on a breast phantom, Hadamard-STA achieves better visualization of the breast inclusion with a clearer boundary between the inclusion and the background than STA and CFW. The in vivo experiments were performed on a patient with a breast tumor, and the tumor could also be better visualized with a more homogeneous background in the strain image obtained by Hadamard-STA than by STA and CFW. These results demonstrate that Hadamard-STA achieves a substantial improvement in lateral motion estimation and maybe a competitive method for quasi-static elastography. [ABSTRACT FROM AUTHOR]

E Reference data and automated completion of defect (green).F Twophase DCIA flap reconstruction plan. B Correction to: Clinical Oral Investigations b https://doi.org/10.1007/s00784-021-04181-3 In the article entitled "Automatic virtual reconstruction of maxillofacial bone defects assisted by ICP (iterative closest point) algorithm and normal people database, by Jie et al., there was error in the Figures and Funding. [Extracted from the article]

Objectives: The aim of this study was to propose and validate an automatic approach based on iterative closest point algorithm for virtual complement and reconstruction for maxillofacial bone defects. Materials and methods: A 3D craniomaxillofacial database of normal Chinese people including 500 skull models was established. Modified iterative closest point (ICP) algorithm was developed to complete bone defects automatically. The performances were evaluated by two approaches: (1) model experiment, virtual bony defects were created on 30 intact normal skull models not included in the database. For each defect model, the algorithm was applied to select the reference skull model from the database. 3-Dimensional and 2-dimensional comparison were conducted to evaluate the error between reference skull model with original intact model. Root mean square error (RMSE) and processing time were calculated. (2) Clinical application, the algorithm was utilized to assist reconstruction of 5 patients with maxillofacial bone defects. The symmetry of post-operative skull model was evaluated by comparing with its mirrored model. Results: The algorithm was tested on an CPU with 1.80 GHz and average processing time was 493.5 s. (1) Model experiment, the average root-mean-square deviation of defect area was less than 2 mm. (2) Clinical application, the RMSE of post-operative skull and its mirrored model was 1.72 mm. Conclusion: It is feasible using iterative closest point algorithm based on normal people database to automatically predict the reference data of missing maxillofacial bone. Clinical relevance: An automated approach based on ICP algorithm and normal people database for maxillofacial bone defect reconstruction has been proposed and validated. [ABSTRACT FROM AUTHOR]

Abstract: We present a method to localize intraoperative target vessel bifurcations under bones for ultrasound (US) image-guided catheter interventions. A catheter path is recorded to acquire skeletons for the target vessel bifurcations that cannot be imaged by intraoperative US. The catheter path is combined with the centerlines of the three-dimensional (3D) US image to construct a preliminary skeleton. Based on the preliminary skeleton, the orientations of target vessels are determined by registration with the preoperative image and the bifurcations were localized by computing the vessel length. An accurate intraoperative vessel skeleton is obtained for correcting the preoperative image to compensate for vessel deformation. A reality check of the proposed method was performed in a phantom experiment. Reasonable results were obtained. The in vivo experiment verified the clinical workflow of the proposed method in an in vivo environment. The accuracy of the centerline length of the vessel for localizing the target artery bifurcation was 2.4mm. These results suggest that the proposed method can allow the catheter tip to stop at the target artery bifurcations and enter into the target arteries. This method can be applied for virtual reality-enhanced image-guided catheter intervention of oral cancers. [Copyright &y& Elsevier]

Specific patterns of lung ultrasound (LUS) images are used to assess the severity of coronavirus disease 2019 (COVID-19) pneumonia, while such assessment is mainly based on clinicians’ qualitative and subjective observations. In this study, we quantitatively analyze the LUS images to assess the severity of COVID-19 pneumonia by characterizing the patterns related to the pleural line (PL) and B-lines (BLs). Twenty-seven patients with COVID-19 pneumonia, including 13 moderate cases, seven severe cases, and seven critical cases, are enrolled. Features related to the PL, including the thickness (TPL) and roughness of the PL (RPL), and the mean (MPLI) and standard deviation (SDPLI) of the PL intensities are extracted from the LUS images. Features related to the BLs, including the number (NBL), accumulated width (AWBL), attenuation coefficient (ACBL), and accumulated intensity (AIBL) of BLs, are also extracted. The correlations of these features with the disease severity are evaluated. The performances of the binary severe/non-severe classification are assessed for each feature and support vector machine (SVM) classifiers with various combinations of features as input. Several features, including the RPL, NBL, AWBL, and AIBL, show significant correlations with disease severity (all ${p} < 0.05$). The classification performance is optimal using the SVM classifier using all the features as input (area under the receiver operating characteristic (ROC) curve = 0.96, sensitivity = 0.93, and specificity = 1). These findings demonstrate that the proposed method may be a promising tool for automatic grading diagnosis and follow-up of patients with COVID-19 pneumonia. [ABSTRACT FROM AUTHOR]

The three-dimensional medical information obtained from Magnetic Resonance Image (MRI), X-ray Computed Tomography (CT), etc. is used for an operation-supporting image, and operations under image guidance have also been conducted. One of the stereoscopic methods is Integral Videography (IV), which is an animated extension of Integral Photography, reproducing a computer-generated graphical object. Though the advantage of IV has been proven in both feasibility studies and clinical applications, one of the issues still unsolved is the notable quantity of calculation that causes significant delay in rendering. Then using parallel processing method integrating Message Passing Interface (MPI) on High Performance Computer (HPC), we shortened the calculating time of IV picture by the shortest at about 0.2 s. Furthermore, by using socket communication, it transmitted to another portable note PC, which is for a display. Then, we achieved the fast presentation of autostereoscopic images seen from an arbitrary direction that was specified with the mouse from the note PC side at anywhere we can access to the network. [Copyright &y& Elsevier]

Objective: In minimally invasive surgery (MIS), in situ augmented reality (AR) navigation systems are usually implemented using a glasses-free 3D display to represent the preoperative tissue structure, and can provide intuitive see-through guidance information. However, due to changes in intraoperative tissue, the preoperative tissue structure is not able to exactly correspond to reality, which influences the precision of in situ AR navigation. To solve this problem, we propose a method to update the tissue structure for in situ AR navigation in such way to reflect changes in intraoperative tissue. Methods: The proposed method to update the tissue structure is based on the calibrated ultrasound and two-level surface warping technologies. Firstly, the particle filter-based calibration is implemented to perform ultrasound calibration and obtain intraoperative position of anatomical points. Secondly, intraoperative positions of anatomical points are inputted in the two-level surface warping method to update the preoperative tissue structure. Finally, the glasses-free real 3-D display of the updated tissue structure is finished, and is superimposed onto a patient by a translucent mirror for in situ AR navigation. Results: we validated the proposed method by simulating liver tissue intervention, and achieved the tissue updating accuracy of 92.86%. Furthermore, the targeting error of AR navigation based on the proposed method was also evaluated through minimally invasive liver surgery, and the acquired mean targeting error was 1.92 mm. Conclusion: The results demonstrate that the proposed AR navigation method is effective. Significance: The proposed method can facilitate MIS, as it provides accurate 3D navigation. [ABSTRACT FROM AUTHOR]

Breast cancer is the leading killer of Chinese women. Immunohistochemistry index has great significance in the treatment strategy selection and prognosis analysis for breast cancer patients. Currently, histopathological examination of the tumor tissue through surgical biopsy is the gold standard to determine immunohistochemistry index. However, this examination is invasive and commonly causes discomfort in patients. There has been a lack of noninvasive method capable of predicting immunohistochemistry index for breast cancer patients. This paper proposes a machine learning method to predict the immunohistochemical index of breast cancer patients by using noninvasive contrast-enhanced ultrasound. A total of 119 breast cancer patients were included in this retrospective study. Each patient implemented the pathological examination of immunohistochemical expression and underwent contrast-enhanced ultrasound imaging of breast tumor. The multi-dimension features including 266 three-dimension features and 837 two-dimension dynamic features were extracted from the contrast-enhanced ultrasound sequences. Using the machine learning prediction method, 21 selected multi-dimension features were integrated to generate a model for predicting the immunohistochemistry index noninvasively. The immunohistochemical index of human epidermal growth factor receptor-2 (HER2) was predicted based on multi-dimension features in contrast-enhanced ultrasound sequence with the sensitivity of 71%, and the specificity of 79% in the testing cohort. Therefore, the noninvasive contrast-enhanced ultrasound can be used to predict the immunohistochemical index. To our best knowledge, no studies have been reported about predicting immunohistochemical index by using contrast-enhanced ultrasound sequences for breast cancer patients. Our proposed method is noninvasive and can predict immunohistochemical index by using contrast-enhanced ultrasound in several minutes, instead of relying totally on the invasive and biopsy-based histopathological examination. Graphical abstract Immunohistochemical index prediction of breast tumor based on multi-dimension features in contrast-enhanced ultrasound. [ABSTRACT FROM AUTHOR]

Hepatic echinococcosis (HE) is a life-threatening liver disease caused by parasites that requires a precise diagnosis and proper treatments. To assess HE lesions accurately, we propose a novel automatic HE lesion segmentation and classification network that contains lesion region positioning (LRP) and lesion region segmenting (LRS) modules. First, we used the LRP module to obtain the probability map of the lesion distribution and the position of the lesion. Then, based on the result of the LRP module, we used the LRS module to precisely segment the HE lesions within the high-probability region. Finally, we classified the HE lesions and identified the lesion types by a convolutional neural network (CNN). The entire dataset was delineated by the hospital's senior radiologist. We collected CT slices of 160 patients from Qinghai Provincial People's Hospital. The Dice score of the final segmentation result reached 89.89%. The Dice scores, indicating the classification accuracy, for cystic vs. alveolar echinococcosis and calcified vs. noncalcified lesions were 80.32% and 82.45%, the sensitivities were 72.41% and 75.17%, the specificities were 83.72% and 86.04%, the NPVs were 80.01% and 86.96%, the PPVs were 80.45% and 81.74%, and the areas under the ROC curves were 0.8128 and 0.8205, respectively. Graphical abstract. [ABSTRACT FROM AUTHOR]

It is challenging to achieve high implant accuracy in dental implant placement, because high risk tissues need to be avoided. In this study, we present an augmented reality (AR) surgical navigation with an accurate cone beam computed tomography (CBCT)-patient registration method to provide clinically desired dental implant accuracy. A registration device is used for registration between preoperative data and patient outside the patient's mouth. After registration, the registration device is worn on the patient's teeth for tracking the patient. Naked-eye 3D images of the planning path and the mandibular nerve are superimposed onto the patient in situ to form an AR scene. Simultaneously, a 3D image of the drill is overlaid accurately on the real one to guide the implant procedure. Finally, implant accuracy is evaluated postoperatively. A model experiment was performed by an experienced dentist. Totally, ten parallel pins were inserted into five 3D-printed mandible models guided by our AR navigation method and through the dentist's experience, respectively. AR-guided dental implant placement showed better results than the dentist's experience (mean target error = 1.25 mm vs. 1.63 mm; mean angle error = 4.03° vs. 6.10°). Experimental results indicate that the proposed method is expected to be applied in the clinic. Graphical abstract ᅟ. [ABSTRACT FROM AUTHOR]

Abstract Background H3K27M is the most frequent mutation in brainstem gliomas (BSGs), and it has great significance in the differential diagnosis, prognostic prediction and treatment strategy selection of BSGs. There has been a lack of reliable noninvasive methods capable of accurately predicting H3K27M mutations in BSGs. Methods A total of 151 patients with newly diagnosed BSGs were included in this retrospective study. The H3K27M mutation status was obtained by whole-exome, whole-genome or Sanger's sequencing. A total of 1697 features, including 6 clinical parameters and 1691 imaging features, were extracted from pre- and post-contrast T1-weighted and T2-weighted images. Using a random forest algorithm, 36 selected MR image features were integrated with 3 selected clinical features to generate a model that was predictive of H3K27M mutations. Additionally, a simplified prediction model comprising the Karnofsky Performance Status (KPS) at diagnosis, symptom duration at diagnosis and edge sharpness on T2 was established for practical clinical utility using the least squares estimation method. Results H3K27M mutation was an independent prognostic factor that conferred a worse prognosis (p = 0.01, hazard ratio = 3.0, 95% confidence interval [CI], 1.57–5.74). The machine learning-based model achieved an accuracy of 84.44% (area under the curve [AUC] = 0.8298) in the test cohort. The simplified model achieved an AUC of 0.7839 in the test cohort. Conclusions Using conventional MRI and clinical features, we established a machine learning-based model with high accuracy and a simplified model with improved clinical utility to predict H3K27M mutations in BSGs. [ABSTRACT FROM AUTHOR]

• A heatmap guided two-stage network is proposed to detect the position and estimate the angle of the capsule robot in ultrasound images. • The proposed model develops the probability distribution and angle prediction modules to reduce the interference information in GI ultrasound images. • Our method obtained small position center error of 0.48 mm and high angle estimation accuracy of 96.32%. A capsule robot can be controlled inside gastrointestinal (GI) tract by an external permanent magnet outside of human body for finishing non-invasive diagnosis and treatment. Locomotion control of capsule robot relies on the precise angle feedback that can be achieved by ultrasound imaging. However, ultrasound-based angle estimation of capsule robot is interfered by gastric wall tissue and the mixture of air, water, and digestive matter existing in the stomach. To tackle these issues, we introduce a heatmap guided two-stage network to detect the position and estimate the angle of the capsule robot in ultrasound images. Specifically, this network proposes the probability distribution module and skeleton extraction-based angle calculation to obtain accurate capsule robot position and angle estimation. Extensive experiments were finished on the ultrasound image dataset of capsule robot within porcine stomach. Empirical results showed that our method obtained small position center error of 0.48 mm and high angle estimation accuracy of 96.32%. Our method can provide precise angle feedback for locomotion control of capsule robot. [ABSTRACT FROM AUTHOR]

• Conventional MRI and dMRI provide different information in BSGs genotyping. • Machine learning-based models were built for noninvasive H3K27M mutation prediction. • dMRI improves predictive accuracy of H3K27M mutation with brain connectivity analysis. • Connectomics features differ between H3K27M-mutant and H3K27-wildtype subgroups. • The derived signatures may help define BSGs at high risk of H3K27M mutation. To establish an individualized predictive model to identify patients with brainstem gliomas (BSGs) at high risk of H3K27M mutation, with the inclusion of brain structural connectivity analysis based on diffusion MRI (dMRI). A primary cohort of 133 patients with BSGs (80 H3K27M-mutant) were retrospectively included. All patients underwent preoperative conventional MRI and dMRI. Tumor radiomics features were extracted from conventional MRI, while two kinds of global connectomics features were extracted from dMRI. A machine learning-based individualized H3K27M mutation prediction model combining radiomics and connectomics features was generated with a nested cross validation strategy. Relief algorithm and SVM method were used in each outer LOOCV loop to select the most robust and discriminative features. Additionally, two predictive signatures were established using the LASSO method, and simplified logistic models were built using multivariable logistic regression analysis. An independent cohort of 27 patients was used to validate the best model. 35 tumor-related radiomics features, 51 topological properties of brain structural connectivity networks, and 11 microstructural measures along white matter tracts were selected to construct a machine learning-based H3K27M mutation prediction model, which achieved an AUC of 0.9136 in the independent validation set. Radiomics- and connectomics-based signatures were generated and simplified combined logistic model was built, upon which derived nomograph achieved an AUC of 0.8827 in the validation cohort. dMRI is valuable in predicting H3K27M mutation in BSGs, and connectomics analysis is a promising approach. Combining multiple MRI sequences and clinical features, the established models have good performance. [ABSTRACT FROM AUTHOR]

The main objective of anatomically plausible results for deformable image registration is to improve model's registration accuracy by minimizing the difference between a pair of fixed and moving images. Since many anatomical features are closely related to each other, leveraging supervision from auxiliary tasks (such as supervised anatomical segmentation) has the potential to enhance the realism of the warped images after registration. In this work, we employ a Multi-Task Learning framework to formulate registration and segmentation as a joint issue, in which we utilize anatomical constraint from auxiliary supervised segmentation to enhance the realism of the predicted images. First, we propose a Cross-Task Attention Block to fuse the high-level feature from both the registration and segmentation network. With the help of initial anatomical segmentation, the registration network can benefit from learning the task-shared feature correlation and rapidly focusing on the parts that need deformation. On the other hand, the anatomical segmentation discrepancy from ground-truth fixed annotations and predicted segmentation maps of initial warped images are integrated into the loss function to guide the convergence of the registration network. Ideally, a good deformation field should be able to minimize the loss function of registration and segmentation. The voxel-wise anatomical constraint inferred from segmentation helps the registration network to reach a global optimum for both deformable and segmentation learning. Both networks can be employed independently during the testing phase, enabling only the registration output to be predicted when the segmentation labels are unavailable. Qualitative and quantitative results indicate that our proposed methodology significantly outperforms the previous state-of-the-art approaches on inter-patient brain MRI registration and pre- and intra-operative uterus MRI registration tasks within our specific experimental setup, which leads to state-of-the-art registration quality scores of 0.755 and 0.731 (i.e., by 0.8% and 0.5% increases) DSC for both tasks, respectively. [Display omitted] • Utilizing segmentation does foster prior knowledge in the registration process. • Registration performance improves with the proposed multi-task learning strategy. • Predictions are refined with the proposed anatomical constraint strategy. • Feature transferring by Cross-Task Attention maximizes the flow of information. • Attention mechanism can reweight attention to deformed regions. [ABSTRACT FROM AUTHOR]

Three-dimensional (3-D) autostereoscopic display with dedicated multiple spatial information under corresponding illumination is critical, especially for anti-counterfeiting, entertainment, etc. In this paper, we propose a 3-D spatial floating display using multi-wavelength integral photography (IP). Using dedicated inkjet printer and refraction-based IP algorithm, a complex two-dimensional (2-D) elemental image array (EIA) can be printed for both fluorescent and normal 3-D autostereoscopic display. With a micro-convex lens array (MLA) and a medium attached on the EIA, normal 3-D images are reconstructed under visible light, while fluorescent 3-D images can be reconstructed under ultraviolet (UV) light. Moreover, to provide comfortable 3-D images with multiple information in space, a feasible 3-D spatial floating display system is also proposed considering the spatial position of the observer with less UV radiation. The proposed method takes the wavelength of 3-D display into consideration to provide spatial multi-information, and can be applied for media, entertainment, etc. Experimental results verified the availability of the proposed method. [ABSTRACT FROM AUTHOR]

Goal: Automatic segmentation of brainstem gliomas and prediction of genotype (H3 K27M) mutation status based on magnetic resonance (MR) images are crucial but challenging tasks for computer-aided diagnosis in neurosurgery. In this paper, we present a novel cascaded deep convolutional neural network (CNN) to address these two challenging tasks simultaneously. Methods: Our novel segmentation task contains two feature-fusion modules: the Gaussian-pyramid multiscale input features-fusion technique and the brainstem-region feature enhancement. The aim is to resolve very difficult problems in brainstem glioma segmentation. Our prediction model combines CNN features and support-vector-machine classifier to automatically predict genotypes without region-of-interest labeled-MR images and is learned jointly with the segmentation task. First, Gaussian-pyramid multiscale input feature fusion is added to our glioma-segmentation task to solve the problems of size variety and weak brainstem-gliomas boundaries. Second, the two feature-fusion modules provide both local and global contexts to retain higher frequency details for sharper tumor boundaries, handling the problem of the large variation of tumor shape, and volume resolution. Results and Conclusion: Experiments demonstrate that our cascaded CNN method achieves not only a good tumor segmentation result with a high Dice similarity coefficient of 77.03%, but also a competitive genotype prediction result with an average accuracy of 94.85% upon fivefold cross-validation. [ABSTRACT FROM AUTHOR]

Lumen and media–adventitia (MA) borders in intravascular ultrasound (IVUS) images are critical for assessing the dimensions of vascular structures and providing plaque information in the diagnosis and navigation of vascular interventions. However, manual delineation of the lumen and MA borders is an intricate and time-consuming process. In this paper, a texture-enhanced deformable model (TEDM) is proposed to accurately detect these borders by incorporating texture information with the morphological factors of deformable model. An ensemble support vector machine classifier is used to classify IVUS pixels presented by texture features into different tissue types. The image regionalization maps of different tissue types are further used for texture enhancement modules in the TEDM. The proposed TEDM method has been tested on 1500 images from 15 clinical IVUS datasets by comparing with the manual delineations. Evaluation results demonstrate that our method can accurately detect lumen and MA surfaces with small surface distance errors of 0.17 and 0.19 mm, respectively. Accurate segmentation results provide 2D measurements of MA/lumen areas and 3D vessel visualizations for vascular interventions. [ABSTRACT FROM AUTHOR]

We develop a spatial position measurement system using three-dimensional (3-D) image marker-based tracking tools targeted at surgical navigation in minimally invasive surgery. We generate 3-D image markers with spatial information encoded to 2-D images, design tracking tools with the 3-D image markers, and analyze the tracking tools’ theoretical spatial errors, which are primarily limited by the spatial distribution of reconstructed fiducial 3-D markers. A pattern analysis-based positional measurement algorithm is developed to calculate the tool's spatial information using its spatial configuration. Evaluation experiments were conducted to demonstrate the accuracy and effectiveness of the proposed system. Furthermore, surgical navigation feasibility studies were performed. With a patient-image registration algorithm, a navigation interface that shows preoperative medical data and intraoperative information about the tool can intuitively and accurately assist surgeons. The results demonstrate that the proposed tracking tools, which have compact volume and spatial positional information, are of potential use in minimally invasive surgery in a limited space. [ABSTRACT FROM AUTHOR]

Objectives. To evaluate the abdominal visceral fat area (VFA), we developed novel ultrasonographic (US) methods for estimating. Methods. 100 male volunteers were recruited, and their VFA was calculated by two novel US methods, the triangle method and the ellipse method. The VFA calculated by these methods was compared with the VFA calculated by CT. Results. Both the VFA calculated by the triangle method (r=0.766, p<0.001) and the ellipse method (r=0.781, p<0.001) showed a high correlation coefficient with the VFA calculated by CT. Also, the VFA calculated by our novel methods were significantly increased in subjects with one or more metabolic risk factors than in those without any risk factors. Furthermore, the correlation coefficients obtained using the two methods were enhanced by the addition of multiple regression analysis (with the triangle method, r=0.8586, p<0.001; with the ellipse method, r=0.8642, p<0.001). Conclusions. The VFA calculated by the triangle or ellipse method showed a high correlation coefficient with the VFA calculated by CT. These US methods are easy to use, they involve no radiation exposure, and the measurements can be conducted frequently. We hope that our simple methods would be widely adopted for the evaluation of VFA. [ABSTRACT FROM AUTHOR]

Over the past few years, deep learning-based image registration methods have achieved remarkable performance in medical image analysis. However, many existing methods struggle to ensure accurate registration while preserving the desired diffeomorphic properties and inverse consistency of the final deformation field. To address the problem, this paper presents a novel symmetric pyramid network for medical image inverse consistent diffeomorphic registration. Specifically, we first encode the multi-scale images to the feature pyramids via a shared-weights encoder network and then progressively conduct the feature-level diffeomorphic registration. The feature-level registration is implemented symmetrically to ensure inverse consistency. We independently carry out the forward and backward feature-level registration and average the estimated bidirectional velocity fields for more robust estimation. Finally, we employ symmetric multi-scale similarity loss to train the network. Experimental results on three public datasets, including Mindboggle101, CANDI, and OAI, show that our method significantly outperforms others, demonstrating that the proposed network can achieve accurate alignment and generate the deformation fields with expected properties. Our code will be available at https://github.com/zhangliutong/SPnet. • Propose symmetric pyramid network for inverse consistent diffeomorphic registration. • Encode images to feature pyramids and implement feature registration progressively. • Formulate symmetric feature registration by bidirectional velocity fields average. • Experimental results show our network consistently outperforms other methods. [ABSTRACT FROM AUTHOR]

A branched DNA amplification strategy was employed to design a colorimetric aptameric biosensor using unmodified gold nanoparticles (AuNPs). First, a programmed DNA dendritic nanostructure was formed using two double-stranded substrate DNAs and two single-stranded auxiliary DNAs as assembly components via a target-assisted cascade amplification reaction, and it was then captured by DNA sensing probe-stabilized AuNPs. The release of sensing probes from AuNPs led to the formation of unstable AuNPs, promoting salt-induced aggregation. By integrating the signal amplification capacity of the branched DNA cascade reaction and unmodified AuNPs as a sensing indicator, this amplified colorimetric sensing strategy allows protein detection with high sensitivity (at the femtomole level) and selectivity. The limit of detection of this approach for VEGF was lower than those of other aptamer-based detection methods. Moreover, this assay provides modification-free and enzyme-free protein detection without sophisticated instrumentation and might be generally applicable to the detection of other protein targets in the future. [ABSTRACT FROM AUTHOR]

Computer-assisted oral and maxillofacial surgery (OMS) has been rapidly evolving since the last decade. State-of-the-art surgical navigation in OMS still suffers from bulky tracking sensors, troublesome image registration procedures, patient movement, loss of depth perception in visual guidance, and low navigation accuracy. We present an augmented reality navigation system with automatic marker-free image registration using 3-D image overlay and stereo tracking for dental surgery. A customized stereo camera is designed to track both the patient and instrument. Image registration is performed by patient tracking and real-time 3-D contour matching, without requiring any fiducial and reference markers. Real-time autostereoscopic 3-D imaging is implemented with the help of a consumer-level graphics processing unit. The resulting 3-D image of the patient’s anatomy is overlaid on the surgical site by a half-silvered mirror using image registration and IP-camera registration to guide the surgeon by exposing hidden critical structures. The 3-D image of the surgical instrument is also overlaid over the real one for an augmented display. The 3-D images present both stereo and motion parallax from which depth perception can be obtained. Experiments were performed to evaluate various aspects of the system; the overall image overlay error of the proposed system was 0.71 mm. [ABSTRACT FROM PUBLISHER]

Introduction: Cardiac electrophysiology aims to describe and treat the electrical activity of the heart. Although an epicardial approach is valuable in many surgical treatments such as coronary artery bypass grafting, maze ablation, and cell transplantation, very few techniques suited for minimally invasive surgery are available for measurement of epicardial electrophysiology. Material and methods: We developed a novel endoscopically-deployable expanding electrode array that can be applied for minimally invasive surgery. Our device consists of a flexible electrode array attached to arms which open and close the electrode sheet. Furthermore, we also developed a computer program to overlay an epicardial electrophysiological map on an endoscopic image. We performed both laboratory and in vivo experiments to examine the feasibility in clinical situations. Results: Evaluation experiments demonstrated that our novel mapping process that assumes spherical deformation of the electrode array enables us to overlay each electrode position with an accuracy of < 1 mm. Results of animal experiments using large animals (one dog and two pigs) demonstrated that our system enables construction of epicardial electrophysiological maps. Conclusion: A novel endoscopically deployable expanding electrode array was developed. Evaluation experiments demonstrated that our device can be manipulated in simulated minimally invasive surgery, and enables construction of epicardial electrophysiological maps. [ABSTRACT FROM AUTHOR]

• Α novel type-aware encoder-decoder network (TypeSeg) is proposed for multi-type ultrasound images co-segmentation. • The TypeSeg model develops the type-aware metric learning and decision modules to fully utilize the tissue type information during semantic segmentation. • The proposed method outperforms all the compared state-of-the-art algorithms for the multi-type ultrasound images co-segmentation task. As a portable and radiation-free imaging modality, ultrasound can be easily used to image various types of tissue structures. It is important to develop a method which supports the multi-type ultrasound images co-segmentation. However, state-of-the-art ultrasound segmentation methods commonly only focus on the single type images or ignore the type-aware information. To solve the above problem, this work proposes a novel type-aware encoder-decoder network (TypeSeg) for the multi-type ultrasound images co-segmentation. First, we develop a type-aware metric learning module to find an optimum latent feature space where the ultrasound images of the same types are close and that of the different types are separated by a certain margin. Second, depending on the extracted features, a decision module decides whether the input ultrasound images have the common tissue type or not, and the encoder-decoder network produces a segmentation mask accordingly. We evaluate the performance of the proposed TypeSeg model on the ultrasound dataset that contains four types of tissues. The proposed TypeSeg model achieves the overall best results with the mean IOU score of 87.51% ± 3.93% for the multi-type ultrasound images. The experimental results indicate that the proposed method outperforms all the compared state-of-the-art algorithms for the multi-type ultrasound images co-segmentation task. [ABSTRACT FROM AUTHOR]

Background: To improve the accuracy of catheter navigation, it is important to develop a method to predict shifts of carotid artery (CA) bifurcations caused by intraoperative deformation. An important factor affecting the accuracy of electromagnetic maxillofacial catheter navigation systems is CA deformations. We aimed to assess CA deformation in different head and neck positions. Methods: Using two sets of computed tomography angiography (CTA) images of six patients, displacements of the skull (maxillofacial segments), C1–C4 cervical vertebrae, mandible (mandibular segment), and CA along with its branches were analyzed. Segmented rigid bones around CA were considered the main causes of CA deformation. After superimposition of maxillofacial segments, C1–C4 and mandible segments were superimposed separately for displacement measurements. Five bifurcation points (vA–vE) were assessed after extracting the CA centerline. A new standardized coordinate system, regardless of patient-specific scanning positions, was employed. It was created using the principal axes of inertia of the maxillofacial bone segments of patients. Position and orientation parameters were transferred to this coordinate system. CA deformation in different head and neck positions was assessed. Results: Absolute shifts in the center of gravity in the bone models for different segments were C1, 1.02 ± 0.9; C2, 2.18 ± 1.81; C3, 4.25 ± 3.85; C4, 5.90 ± 5.14; and mandible, 1.75 ± 2.76 mm. Shifts of CA bifurcations were vA, 5.52 ± 4.12; vB, 4.02 ± 3.27; vC, 4.39 ± 2.42; vD, 4.48 ± 1.88; and vE, 2.47 ± 1.32. Displacements, position changes, and orientation changes of C1–C4 segments as well as the displacements of all CA bifurcation points were similar in individual patients. Conclusions: CA deformation was objectively proven as an important factor contributing to errors in maxillofacial navigation. Our study results suggest that small movements of the bones around CA can result in small CA deformations. Although patients’ faces were not fixed properly during CT scanning, C1–C4 and vA–vE displacements were similar in individual patients. We proposed a novel method for accumulation of the displacement data, and this study indicated the importance ofsurrounding bone displacements in predicting CA bifurcation. [ABSTRACT FROM AUTHOR]

• In-phase/quadrature (IQ) data were used to accelerate CS-STA. • IQ-domain CS-STA maintains high image quality of conventional RF-domain CS-STA. • CS-STA obtains similar resolution, higher contrast and higher frame rate, compared with STA. • More compatible with clinical ultrasound system than RF-domain CS-STA. Compressed sensing-based synthetic transmit aperture (CS-STA) was previously proposed to recover the full radio-frequency (RF) channel dataset of synthetic transmit aperture (STA) from that of a smaller number of randomly apodized plane wave (PW) transmissions. In this way, the imaging frame rate (FR) and contrast are improved with maintained spatial resolution, compared with those of STA. Because CS-STA reconstruction is repeated for all receive elements and RF samples (with a high sampling frequency), the recovery of STA dataset in RF domain is time-consuming. In the meantime, a large amount of RF data needs to be transferred and stored, resulting in an increase of system complexity and required memory space. In this study, CS-STA is extended to in-phase/quadrature (IQ) domain (with lower sampling frequency) for the recovery of baseband STA IQ dataset to accelerate the CS-STA reconstruction by reducing the amount of data to be processed. More importantly, CS-STA reconstruction using IQ data is of practical importance, as clinical ultrasound systems typically record baseband IQ signal instead of RF signal. Simulations, phantom and in vivo experiments verify the feasibility of CS-STA in IQ domain for the recovery of STA dataset. More specifically, CS-STA using IQ data achieves similar image quality and appreciably improves reconstruction speed (by ∼3 times) compared with that using RF data. These findings demonstrate that IQ-domain CS-STA is capable of relieving the computational and storage burdens, which may facilitate the implementation of CS-STA in practical ultrasound systems. [ABSTRACT FROM AUTHOR]

• A comparative analysis framework of 3T and 7T Time-Of-Flight Magnetic Resonance Angiography (MRA) was proposed. • The framework can segment the cerebrovascular structure automatically and compare TOF-MRA with different field strengths. • It is helpful for clinical cerebrovascular disease, especially cerebral small vessel diseases. High field strength 3T and 7T Time-Of-Flight Magnetic Resonance Angiography (TOF- MRA) achieves better visualization of intracranial vessels, so it attracts much attention. However, quantitative comparison between 3T and 7T MRA is lacking in the aspects of image quality and the practical application of cerebrovascular diseases. In this paper, a quantitative framework of 3T and 7T TOF-MRA comparison is proposed, which contains two steps including the automated cerebrovascular segmentation and statistical analysis. Firstly, the whole vascular structures on both 3T and 7T TOF-MRA images are segmented automatically, especially those small blood vessels in 7T MRA. The skeleton extraction-based automatic seed point detection is implemented to ensure the segmented vascular structure complete and precise. Secondly, the statistical analysis of the differences between 3T and 7T MRA is carried out in the aspects of image quality and the characteristics of some important vessels. The objects of statistical analysis are achieved and analyzed automatically without needing the time- consuming human beings' participation, therefore, it is efficient and objective. The comparison experiments on seven pairs of 3T and 7T TOF MRA images validated that about image quality, the contrast-to-noise ratio of 7T MRA was about 4.53 ± 0.95 times as much as that of 3T MRA. About the cerebrovascular information, small vessels were more abundant in 7T MRA compared with 3T MRA (branches number: 462.0 ± 58.5 vs 393.1 ± 63.3). The proposed framework can segment the whole cerebrovascular structure automatically and compare TOF-MRA with different field strengths objectively and quantitatively. It is helpful for clinical cerebrovascular disease, especially cerebral small vessel diseases. [ABSTRACT FROM AUTHOR]

• Α novel in-situ AR navigation based on the enhanced arthroscopic information is developed to solve the existed problems of in-situ AR navigation. • This system uses tissue properties-based model deformation method to update the 3D preoperative knee model. • Preclinical experiments on knee phantom and in-vitro swine knee demonstrate the effectiveness the proposed in-situ AR navigation. During the minimally invasive knee surgery, surgeons insert surgical instruments and arthroscopy through small incisions, and implement treatment assisted by 2D arthroscopic images. However, this 2D arthroscopic navigation faces several problems. Firstly, the guidance information is displayed on a screen away from the surgical area, which makes hand/eye coordination difficult. Secondly, the small incision limits the surgeons to view the internal knee structures only from an arthroscopic camera. In addition, arthroscopic images commonly appear obscure visions. To solve these problems, we proposed a novel in-situ augmented reality navigation system with the enhanced arthroscopic information. Firstly, intraoperative anatomical locations were obtained by using arthroscopic images and arthroscopy calibration. Secondly, tissue properties-based model deformation method was proposed to update the 3D preoperative knee model with anatomical location information. Then, the updated model was further rendered with glasses-free real 3D display for achieving the global in-situ augmented reality view. In addition, virtual arthroscopic images were generated from the updated preoperative model to provide the anatomical information of the operation area. Experimental results demonstrated that virtual arthroscopic images could reflect the correct structure information with a mean error of 0.32 mm. Compared with 2D arthroscopic navigation, the proposed augmented reality navigation reduced the targeting errors by 2.10 mm and 2.70 mm for the experiments of knee phantom and in-vitro swine knee, respectively. Our navigation method is helpful for minimally invasive knee surgery since it can provide the global in-situ information and detail anatomical information. [ABSTRACT FROM AUTHOR]

• The gait acquisition platform is built for osteoarthritis by using an RGB-D camera and the developed software of gait data recording • Experimental results demonstrate that the gait analysis system can quantitatively predict gait anomalies with a high accuracy of 98.77% • The gait acquisition and analysis system achieves quantitative recording and objective assessment of gait anomalies Osteoarthritis (OA) is the most common type of joint-related diseases, which affects millions of people worldwide. Expensive and time-consuming medical imaging can provide precise structural description of knee joints, but lacks the functional descriptions. Gait analysis can provide functional descriptions of knee joints. However, orthopedic surgeons always observe the patient's gait qualitatively and perform subjective assessments through rating scales at present due to the lack of a quantitative gait analysis system. To solve these problems, a gait acquisition and analysis system is developed to provide a cheap, easy-to-use solution for quantitative recording and functional description of OA patients. Firstly, an automatic gait acquisition platform is designed for the clinical setting based on the RGB-D camera and the developed software of gait data recording. In addition, the effective working space of gait acquisition platform is evaluated for clinical applications by comparing with the ground-truth from infrared optical trackers. Secondly, the acquired gait data is analyzed with a novel hybrid prediction model to assess the gait anomalies quantitatively and objectively. In the hybrid model, the extracted features of gait data contain the manually-extracted features and the automatically-extracted features from Long Short-Term Memory network. Experimental results on real patients demonstrate that the proposed gait analysis system can quantitatively predict gait anomalies with a high accuracy of 98.77 %. Therefore, this gait acquisition and analysis system achieves quantitative recording and objective assessment of gait anomalies for clinical OA treatments. [ABSTRACT FROM AUTHOR]