Search

publicly available ultrasound data and labelling

Locating the start, apex and end keyframes of moving contrast agents for keyframe counting during X-ray coronary angiography (XCA) is very important in the diagnosis and treatment of cardiovascular diseases. To locate these keyframes from the class-imbalanced and boundary-agnostic foreground vessel actions that overlap complex backgrounds, we propose long short-term spatiotemporal attention by integrating a convolutional long short-term memory (CLSTM) network into a multiscale Transformer to learn the segment- and sequence-level dependences in the consecutive-frame-based deep features. Image-to-patch contrastive learning is further embedded between the CLSTM-based long-term spatiotemporal attention and Transformer-based short-term attention modules. The imagewise contrastive module reuses the long-term attention to contrast image-level foreground/background of XCA sequence, while patchwise contrastive projection selects the random patches of backgrounds as convolution kernels to project foreground/background frames into different latent spaces. A new XCA video dataset is collected to evaluate the proposed neural network. The experimental results show that the proposed method achieves a mAP of 70.51\% and an F1-score of 0.8188, considerably outperforming the state-of-the-art methods. The source code and dataset are available at https://github.com/Binjie-Qin/STA-IPCon.

The heart rate (HR) signal is so weak in remote photoplethysmography (rPPG) and ballistocardiogram (BCG) that HR estimation is very sensitive to face and body motion disturbance caused by spontaneous head and body movements as well as facial expressions of subjects in conversation. This article proposed a novel multimodal quasi-contactless HR sensor to ensure the robustness and accuracy of HR estimation under extreme facial poses, large-motion disturbances, and multiple faces in a video for computer-aided police interrogation. Specifically, we propose a novel landmark-based approach for a deep facial region of interest (ROI) tracker and face pose constrained Kalman filter to continuously and robustly track target facial ROIs for estimating HR from face and head motion disturbances in rPPG. This motion-disturbed rPPG signal is further fused with a minimally disturbed BCG signal by the face and head movements via a bank of notch filters with a recursive weighting scheme to obtain the dominant HR frequency for final accurate HR estimation. To facilitate reproducible HR estimation research, we synchronously acquire and publicly share a multimodal data set that contains 20 sets of ECG and BCG signals as well as uncompressed, rPPG-dedicated videos from ten subjects in a stable state and large-motion state (MS) without and with large face and body movements in a sitting position. We demonstrate through experimental comparisons that the proposed multimodal HR sensor is more robust and accurate than the state-of-the-art single-modal HR sensor solely with rPPG- or BCG-based methods. The mean absolute error (MAE) of HR estimation is 7.13 BPM lower than the BCG algorithm and 3.12 BPM lower than the model-based plane-orthogonal-to-skin (POS) algorithm in the MS.

In X-ray computed tomography, radiation doses are harmful but can be significantly reduced by intuitively decreasing the number of projections. However, less projection views usually lead to low-resolution images. To address this issue, we propose a robust and enhanced mechanism on the basis of denoising autoencoding prior, or robust EDAEP (REDAEP) for sparse-view computed tomography reconstruction. REDAEP can substantially improve the reconstruction quality with two novel contributions. First, by employing the variable augmentation technique, REDAEP learns higher-dimensional network with three-channel image and proceeds to the single-channel image reconstruction. Second, REDAEP replaces the $L^{2}$ regression loss function with a more robust $L^{p} $ ( $0 ) regression to preserve more texture details. The empirical results demonstrate that REDAEP can achieve better performance than state-of-the-arts, in terms of quantitative measures and subjective visual quality.

The aim of this study was to identify independent anatomic, morphologic and hemodynamic features of the ACoA (anterior communicating artery) complex that serve as risk factors for the occurrence of ACoA aneurysms.Fifteen consecutive patients with 15 ACoA aneurysms were included. Computational fluid dynamics (CFD) simulations based on patient-specific models were carried out using 3D time-of-flight magnetic resonance angiography (3D-TOF-MRA) images. A reverse reconstruction technique was used to generate a pre-aneurysm vessel anatomy. Geometric parameters and hemodynamic changes were compared and evaluated.The overall prevalence of symmetric, dysplastic, and absent A1 segments were 53.3%, 26.7%, and 20%. The mean wall shear stress (WSS) of the absent group (AG) was significantly higher than that of the symmetric group (SG) and dysplastic group (DG). The absolute mean A1 artery flow rate (410.2 ± 88 versus 439.4 ± 101 mL/min;ACoA aneurysms are a high-WSS pathology. Severe flow impingement and the anatomic vasculature structures play a role in triggering the occurrence of ACoA aneurysms.

Based on low-rank plus sparse decomposition and low-rank approximation techniques, this chapter introduces three newly developed algorithms within machine learning framework for X-ray coronary angiography (XCA) vessel extraction. First, a motion coherency regularized robust principal component analysis (RPCA) was proposed to accurately extract contrast-filled vessel candidates via a statistically structured RPCA, which then subsequently refine vessel extraction via total variation regularization on the trajectory decomposition of the candidate foreground vessels. Second, an encoder–decoder-based sequential vessel segmentation deep network called SVS-net was developed by exploiting several contextual frames of sequential images to segment the vessel masks for the current frame, which was built upon U-net to extract spatiotemporal features in encoder stage and fuse the features in skip connection layers as well as design channel attention mechanism in decoder stage. Lastly, by exploiting the low rankness and interframe spatiotemporal connectivity in the complex and noisy backgrounds, we recovered the vessel-masked background regions using tensor completion with the twist tensor nuclear norm being minimized to complete the background layers. This sequential XCA background completion method accurately extracted foreground vessels' heterogeneous intensities from the noisy XCA data by subtracting the completed background layers from the XCA images. Experimental results showed the superiority of the proposed machine learning-based frameworks over the state-of-the-art methods.

This paper presents a supervised data-driven algorithm for impulse noise removal via iterative scheme-inspired network (IIN). IIN is defined over a data flow graph, which is derived from the iterative procedures in Alternating Direction Method of Multipliers (ADMM) algorithm for optimizing the L1-guided variational model. In the training phase, the L1-minimization is reformulated into an augmented Lagrangian scheme through adding a new auxiliary variable. In the testing phase, it has computational overhead similar to ADMM but uses optimized parameters learned from the training data for restoration task. Experimental results demonstrate that the newly proposed method can obtain very significantly superior performance than current state-of-the-art variational and dictionary learning-based approaches for salt-and-pepper noise removal.

Contactless measurement of heart rate(HR) based on videos can be essential for tele-monitoring medical system and public security. This paper proposed a modified POS-based remote photoplethysmography(rPPG) algorithm by replacing alpha tuning with singular value decomposition(SVD) to eliminate the specular reflection's spatiotemporal characteristic. What's more, we propose a novel landmark-based approach for a deep facial ROI tracker and face pose constrained Kalman filter to continuously and robustly track target facial ROIs for estimating HR from large head motion disturbances in rPPG. We demonstrate through experimental comparisons that the proposed method is more robust and accurate than the state-of-the-art rPPG-based methods in stable state. The mean absolute error (MAE) of HR estimation is 2.15 BPM lower than the POS and the AUC of HR estimation accuracy is 0.04 higher than POS in stable state. In motion state, the performance of our modified is a little bit better than POS.

This paper develops a novel encoder-decoder deep network architecture which exploits the several contextual frames of 2D+t sequential images in a sliding window centered at current frame to segment 2D vessel masks from the current frame. The architecture is equipped with temporal-spatial feature extraction in encoder stage, feature fusion in skip connection layers and channel attention mechanism in decoder stage. In the encoder stage, a series of 3D convolutional layers are employed to hierarchically extract temporal-spatial features. Skip connection layers subsequently fuse the temporal-spatial feature maps and deliver them to the corresponding decoder stages. To efficiently discriminate vessel features from the complex and noisy backgrounds in the XCA images, the decoder stage effectively utilizes channel attention blocks to refine the intermediate feature maps from skip connection layers for subsequently decoding the refined features in 2D ways to produce the segmented vessel masks. Furthermore, Dice loss function is implemented to train the proposed deep network in order to tackle the class imbalance problem in the XCA data due to the wide distribution of complex background artifacts. Extensive experiments by comparing our method with other state-of-the-art algorithms demonstrate the proposed method's superior performance over other methods in terms of the quantitative metrics and visual validation. The source codes are at https://github.com/Binjie-Qin/SVS-net, Comment: 14

Ultrasound(US) imaging is widely employed in diagnosing diseases because of its low cost, high efficiency and less harmfulness to patients. However, experienced doctors are required to operate the system and interpret ultrasound image. In this paper, a robot assisted ultrasound imaging system is developed for remote ultrasonic diagnosis, in which a master-slave mapping relationship is established based on velocity control of incremental position and orientation. Butterworth lowpass filtering is applied to suppress the jitter signal at the master side and improve the smoothness at the slave side. Finally, an ultrasound imaging experiment is carried out and the effectiveness and feasibility of remote robot-assisted ultrasound imaging system are verified.

Image outliers such as missing correspondences and large local deformations break the one-to-one pixelwise mapping between target image and moving image to be registered. Both traditional registration methods and deep-learning based deformable image registration methods fail to tackle this problem. This paper proposed an unsupervised globalto-local deformable registration network reinforced by joint saliency map to accurately, robustly and fast address the problem. The global-to-local network divided the overall learning of a complex mapping of image registration into a simpler global mapping learning and local residual mapping. The joint saliency map of the two images to be registered bidirectionally reinforced the whole network's forward estimation and back-propagation with uncertainty modeling and context-aware intelligence. The experimental results confirm the proposed method's performance advantages over the state-of-the-arts registration methods in the challenges image registration with missing correspondences and large local deformations.

This paper proposes a detail-preserving image denoising method via cluster-wise progressive principal component analysis (PCA) thresholding based on the Marchenko–Pastur (MP) law in random matrix theory. According to random matrix theory, an efficient and stable noise-level estimation method is also presented. Specifically, a global Gaussian noise level is estimated by interpreting the relationship between noise and eigenvalues of PCA for noisy patch matrices via the MP law in conjunction with the observation that vectors extracted from a noise-free image often lie in a low-dimensional subspace. Before noise removal, an adaptive clustering method is developed to automatically determine a suitable number of clusters segregating patches with different features (edges and textures). To denoise each cluster matrix, progressive PCA thresholding is performed. First, a hard thresholding of singular values in the singular value decomposition domain based on the MP law is applied to find a low-rank approximation to the cluster matrix. Second, the remaining noises of the low-rank matrix are further removed in the PCA transform domain using a special soft thresholding, i.e., the linear minimum mean-square-error technique with locally estimated parameters. The experiments show that the proposed method not only achieves state-of-the-art denoising performance in terms of quantitative indices, but also preserves visually important image details best.

This paper proposes a locally adaptive kernel regression with adaptive-scale kernels for deformable image registration with outliers (i.e., missing correspondences and large local deformations). The adaptive kernel regression locally constructs dense deformation fields from the weighted contributions of each pixel's surrounding discrete displacement fields in a moving anisotropic kernel by exploiting the contextual deformations of the corresponding saliency structures in the two images. Specifically, we first propose an effective superpixel-based structure scale estimator to estimate the boundary-aware structure scale of each reference structure. We further propose an edge-aware mismatch scale measuring the mismatch degree of the edge structures to be matched in the images. By combining the boundary-aware structure scale with the edge-aware mismatch scale of the underlying saliency structures to be matched, we define edge-aware adaptive-scale kernels for the locally adaptive kernel regression to efficiently construct deformations for deformable registration with outliers. The experiments show that the proposed method achieves not only state-of-the-art matching accuracy for normal corresponding structures but also the best matching efficiency for outlier structures in deformable image registration.

This paper presents a novel semi-reference inspired color-to-gray conversion model for faithfully preserving the contrast details of the color image, essentially differs from most of the no-reference and reference approaches. In the proposed model, on the basic assumption that a good gray conversion should make the conveyed gradient values (i.e., contrast) to be maximal, we present a projection maximum function to model the decolorization procedure. We further incorporate weights of the original gradients into the maximum function. The Gaussian weighted factor consisting of the gradients of each channel of the input color image is employed to better reflect the degree of preserving feature discriminability and color ordering in color-to-gray conversion. The projected gradient descent and discrete searching techniques are developed to solve the proposed model with and without nonnegative constraint, respectively. Extensive experimental evaluations on two existing datasets, containing abundant colors and patterns, show that the proposed method outperforms the state-of-the-art methods quantitatively and qualitatively.

Purpose Single image super-resolution (SR) is highly desired in many fields but obtaining it is often technically limited in practice. The purpose of this study was to propose a simple, rapid and robust single image SR method in magnetic resonance (MR) imaging (MRI). Methods The idea is based on the mathematical formulation of the intrinsic link in k-space between a given (modulus) low-resolution (LR) image and the desired SR image. The method consists of two steps: 1) estimating the low-frequency k-space data of the desired SR image from a single LR image; 2) reconstructing the SR image using the estimated low-frequency and zero-filled high-frequency k-space data. The method was evaluated on digital phantom images, physical phantom MR images and real brain MR images, and compared with existing SR methods. Results The proposed SR method exhibited a good robustness by reaching a clearly higher PSNR (25.77dB) and SSIM (0.991) averaged over different noise levels in comparison with existing edge-guided nonlinear interpolation (EGNI) (PSNR=23.78dB, SSIM=0.983), zero-filling (ZF) (PSNR=24.09dB, SSIM=0.985) and total variation (TV) (PSNR=24.54dB, SSIM=0.987) methods while presenting the same order of computation time as the ZF method but being much faster than the EGNI or TV method. The average PSNR or SSIM over different slice images of the proposed method (PSNR=26.33 dB or SSIM=0.955) was also higher than the EGNI (PSNR=25.07dB or SSIM=0.952), ZF (PSNR=24.97dB or SSIM=0.950) and TV (PSNR=25.70dB or SSIM=0.953) methods, demonstrating its good robustness to variation in anatomical structure of the images. Meanwhile, the proposed method always produced less ringing artifacts than the ZF method, gave a clearer image than the EGNI method, and did not exhibit any blocking effect presented in the TV method. In addition, the proposed method yielded the highest spatial consistency in the inter-slice dimension among the four methods. Conclusions This study proposed a fast, robust and efficient single image SR method with high spatial consistency in the inter-slice dimension for clinical MR images by estimating the low-frequency k-space data of the desired SR image from a single spatial modulus LR image.

X-ray coronary angiography can provide rich dynamic information of cardiac and vascular function. Extracting contrast-filled vessel from the complex dynamic background (caused by the movement of diaphragm, lung, bones, etc.) in X-ray coronary angiograms has great clinical significance in assisting myocardial perfusion evaluation, reconstructing vessel structures for diagnosis and treatment of heart disease. Considering the angiography image sequence is a sum of a low-rank background matrix and a sparse flowing contrast agent matrix, we propose a novel graduated robust principal component analysis (RPCA) with spatio-temporal motion coherency constraint to accurately extract contrast-filled vessel from the X-ray coronary angiograms: (1) We first use a statistically structured RPCA with complex noise model exploiting the complex structural connectivity of vessel regions to identify all candidate foreground contrast-filled vessels; (2) To eliminate the background remained in the candidates, we further introduce trajectory decomposition on the candidate foregrounds to accurately extract contrast-filled vessels using motion coherency regularized RPCA, which imposes total variation minimization on the foreground trajectories to model the spatio-temporal contiguity and smoothness of the foreground trajectories. The graduated RPCA with motion coherency constraint shows to consistently outperform other state-of-the-art methods, in particular on real-world X-ray coronary angiograms that contain a significant amount of complex dynamic background motion. Experimental results on twelve sequences of real X-ray coronary angiograms are evaluated using both qualitative and quantitative methods to demonstrate the obvious advantages of our method over the state-of-the-art alternatives.

Graphical abstractDisplay Omitted Matching outlier structures with missing correspondences and/or local large deformations is very difficult in image registration. In this paper, we define structure matching as an iterative local adaptive kernel regression which locally reconstructs moving image's dense deformation fields from the discrete displacement fields computed by multi-resolution block matching. First, a new joint saliency map (JSM) is proposed to match a structure-tensor-based local saliency distribution for each overlapping pixel pair and highlight the corresponding saliency structures (called joint saliency structures, JSSs) between the images. To explore the consistency of normal JSSs and their deformations around the outliers, we use JSM to guide the dense deformation reconstruction by emphasizing the JSSs' discrete displacement vectors in kernel regression. The JSS adaptive kernel regression adapts anisotropic kernel's shape and orientation to reference image's structure and weights more contribution from JSS's displacement vectors for the iterative regression, whereby moving image's local deformations can be compliant with reference image's corresponding normal structures. The experimental results demonstrate that the proposed method achieves almost the best performance in structure matching of all challenging image pairs with outlier structures compared with other state-of-the-art intensity-based nonrigid registration algorithms.