Search

The heart rate (HR) signal is so weak in remote photoplethysmography (rPPG) and ballistocardiogram (BCG) that HR estimation is very sensitive to face and body motion disturbance caused by spontaneous head and body movements as well as facial expressions of subjects in conversation. This article proposed a novel multimodal quasi-contactless HR sensor to ensure the robustness and accuracy of HR estimation under extreme facial poses, large-motion disturbances, and multiple faces in a video for computer-aided police interrogation. Specifically, we propose a novel landmark-based approach for a deep facial region of interest (ROI) tracker and face pose constrained Kalman filter to continuously and robustly track target facial ROIs for estimating HR from face and head motion disturbances in rPPG. This motion-disturbed rPPG signal is further fused with a minimally disturbed BCG signal by the face and head movements via a bank of notch filters with a recursive weighting scheme to obtain the dominant HR frequency for final accurate HR estimation. To facilitate reproducible HR estimation research, we synchronously acquire and publicly share a multimodal data set that contains 20 sets of ECG and BCG signals as well as uncompressed, rPPG-dedicated videos from ten subjects in a stable state and large-motion state (MS) without and with large face and body movements in a sitting position. We demonstrate through experimental comparisons that the proposed multimodal HR sensor is more robust and accurate than the state-of-the-art single-modal HR sensor solely with rPPG- or BCG-based methods. The mean absolute error (MAE) of HR estimation is 7.13 BPM lower than the BCG algorithm and 3.12 BPM lower than the model-based plane-orthogonal-to-skin (POS) algorithm in the MS.

In X-ray computed tomography, radiation doses are harmful but can be significantly reduced by intuitively decreasing the number of projections. However, less projection views usually lead to low-resolution images. To address this issue, we propose a robust and enhanced mechanism on the basis of denoising autoencoding prior, or robust EDAEP (REDAEP) for sparse-view computed tomography reconstruction. REDAEP can substantially improve the reconstruction quality with two novel contributions. First, by employing the variable augmentation technique, REDAEP learns higher-dimensional network with three-channel image and proceeds to the single-channel image reconstruction. Second, REDAEP replaces the $L^{2}$ regression loss function with a more robust $L^{p} $ ( $0 ) regression to preserve more texture details. The empirical results demonstrate that REDAEP can achieve better performance than state-of-the-arts, in terms of quantitative measures and subjective visual quality.

Based on low-rank plus sparse decomposition and low-rank approximation techniques, this chapter introduces three newly developed algorithms within machine learning framework for X-ray coronary angiography (XCA) vessel extraction. First, a motion coherency regularized robust principal component analysis (RPCA) was proposed to accurately extract contrast-filled vessel candidates via a statistically structured RPCA, which then subsequently refine vessel extraction via total variation regularization on the trajectory decomposition of the candidate foreground vessels. Second, an encoder–decoder-based sequential vessel segmentation deep network called SVS-net was developed by exploiting several contextual frames of sequential images to segment the vessel masks for the current frame, which was built upon U-net to extract spatiotemporal features in encoder stage and fuse the features in skip connection layers as well as design channel attention mechanism in decoder stage. Lastly, by exploiting the low rankness and interframe spatiotemporal connectivity in the complex and noisy backgrounds, we recovered the vessel-masked background regions using tensor completion with the twist tensor nuclear norm being minimized to complete the background layers. This sequential XCA background completion method accurately extracted foreground vessels' heterogeneous intensities from the noisy XCA data by subtracting the completed background layers from the XCA images. Experimental results showed the superiority of the proposed machine learning-based frameworks over the state-of-the-art methods.

This paper presents a supervised data-driven algorithm for impulse noise removal via iterative scheme-inspired network (IIN). IIN is defined over a data flow graph, which is derived from the iterative procedures in Alternating Direction Method of Multipliers (ADMM) algorithm for optimizing the L1-guided variational model. In the training phase, the L1-minimization is reformulated into an augmented Lagrangian scheme through adding a new auxiliary variable. In the testing phase, it has computational overhead similar to ADMM but uses optimized parameters learned from the training data for restoration task. Experimental results demonstrate that the newly proposed method can obtain very significantly superior performance than current state-of-the-art variational and dictionary learning-based approaches for salt-and-pepper noise removal.

Contactless measurement of heart rate(HR) based on videos can be essential for tele-monitoring medical system and public security. This paper proposed a modified POS-based remote photoplethysmography(rPPG) algorithm by replacing alpha tuning with singular value decomposition(SVD) to eliminate the specular reflection's spatiotemporal characteristic. What's more, we propose a novel landmark-based approach for a deep facial ROI tracker and face pose constrained Kalman filter to continuously and robustly track target facial ROIs for estimating HR from large head motion disturbances in rPPG. We demonstrate through experimental comparisons that the proposed method is more robust and accurate than the state-of-the-art rPPG-based methods in stable state. The mean absolute error (MAE) of HR estimation is 2.15 BPM lower than the POS and the AUC of HR estimation accuracy is 0.04 higher than POS in stable state. In motion state, the performance of our modified is a little bit better than POS.

This paper develops a novel encoder-decoder deep network architecture which exploits the several contextual frames of 2D+t sequential images in a sliding window centered at current frame to segment 2D vessel masks from the current frame. The architecture is equipped with temporal-spatial feature extraction in encoder stage, feature fusion in skip connection layers and channel attention mechanism in decoder stage. In the encoder stage, a series of 3D convolutional layers are employed to hierarchically extract temporal-spatial features. Skip connection layers subsequently fuse the temporal-spatial feature maps and deliver them to the corresponding decoder stages. To efficiently discriminate vessel features from the complex and noisy backgrounds in the XCA images, the decoder stage effectively utilizes channel attention blocks to refine the intermediate feature maps from skip connection layers for subsequently decoding the refined features in 2D ways to produce the segmented vessel masks. Furthermore, Dice loss function is implemented to train the proposed deep network in order to tackle the class imbalance problem in the XCA data due to the wide distribution of complex background artifacts. Extensive experiments by comparing our method with other state-of-the-art algorithms demonstrate the proposed method's superior performance over other methods in terms of the quantitative metrics and visual validation. The source codes are at https://github.com/Binjie-Qin/SVS-net, Comment: 14

Ultrasound(US) imaging is widely employed in diagnosing diseases because of its low cost, high efficiency and less harmfulness to patients. However, experienced doctors are required to operate the system and interpret ultrasound image. In this paper, a robot assisted ultrasound imaging system is developed for remote ultrasonic diagnosis, in which a master-slave mapping relationship is established based on velocity control of incremental position and orientation. Butterworth lowpass filtering is applied to suppress the jitter signal at the master side and improve the smoothness at the slave side. Finally, an ultrasound imaging experiment is carried out and the effectiveness and feasibility of remote robot-assisted ultrasound imaging system are verified.

This paper proposes a detail-preserving image denoising method via cluster-wise progressive principal component analysis (PCA) thresholding based on the Marchenko–Pastur (MP) law in random matrix theory. According to random matrix theory, an efficient and stable noise-level estimation method is also presented. Specifically, a global Gaussian noise level is estimated by interpreting the relationship between noise and eigenvalues of PCA for noisy patch matrices via the MP law in conjunction with the observation that vectors extracted from a noise-free image often lie in a low-dimensional subspace. Before noise removal, an adaptive clustering method is developed to automatically determine a suitable number of clusters segregating patches with different features (edges and textures). To denoise each cluster matrix, progressive PCA thresholding is performed. First, a hard thresholding of singular values in the singular value decomposition domain based on the MP law is applied to find a low-rank approximation to the cluster matrix. Second, the remaining noises of the low-rank matrix are further removed in the PCA transform domain using a special soft thresholding, i.e., the linear minimum mean-square-error technique with locally estimated parameters. The experiments show that the proposed method not only achieves state-of-the-art denoising performance in terms of quantitative indices, but also preserves visually important image details best.

X-ray coronary angiography can provide rich dynamic information of cardiac and vascular function. Extracting contrast-filled vessel from the complex dynamic background (caused by the movement of diaphragm, lung, bones, etc.) in X-ray coronary angiograms has great clinical significance in assisting myocardial perfusion evaluation, reconstructing vessel structures for diagnosis and treatment of heart disease. Considering the angiography image sequence is a sum of a low-rank background matrix and a sparse flowing contrast agent matrix, we propose a novel graduated robust principal component analysis (RPCA) with spatio-temporal motion coherency constraint to accurately extract contrast-filled vessel from the X-ray coronary angiograms: (1) We first use a statistically structured RPCA with complex noise model exploiting the complex structural connectivity of vessel regions to identify all candidate foreground contrast-filled vessels; (2) To eliminate the background remained in the candidates, we further introduce trajectory decomposition on the candidate foregrounds to accurately extract contrast-filled vessels using motion coherency regularized RPCA, which imposes total variation minimization on the foreground trajectories to model the spatio-temporal contiguity and smoothness of the foreground trajectories. The graduated RPCA with motion coherency constraint shows to consistently outperform other state-of-the-art methods, in particular on real-world X-ray coronary angiograms that contain a significant amount of complex dynamic background motion. Experimental results on twelve sequences of real X-ray coronary angiograms are evaluated using both qualitative and quantitative methods to demonstrate the obvious advantages of our method over the state-of-the-art alternatives.

Image restoration (IR) is a long-standing challenging problem in low-level image processing. It is of utmost importance to learn good image priors for pursuing visually pleasing results. In this paper, we develop a multi-channel and multi-model-based denoising autoencoder network as image prior for solving IR problem. Specifically, the network that trained on RGB-channel images is used to construct a prior at first, and then the learned prior is incorporated into single-channel grayscale IR tasks. To achieve the goal, we employ the auxiliary variable technique to integrate the higher-dimensional network-driven prior information into the iterative restoration procedure. In addition, according to the weighted aggregation idea, a multi-model strategy is put forward to enhance the network stability that favors to avoid getting trapped in local optima. Extensive experiments on image deblurring and deblocking tasks show that the proposed algorithm is efficient, robust, and yields state-of-the-art restoration quality on grayscale images.

We propose an ultrasound speckle filtering method for not only preserving various edge features but also filtering tissue-dependent complex speckle noises in ultrasound images. The key idea is to detect these various edges using a phase congruence-based edge significance measure called phase asymmetry (PAS), which is invariant to the intensity amplitude of edges and takes 0 in non-edge smooth regions and 1 at the idea step edge, while also taking intermediate values at slowly varying ramp edges. By leveraging the PAS metric in designing weighting coefficients to maintain a balance between fractional-order anisotropic diffusion and total variation (TV) filters in TV cost function, we propose a new fractional TV framework to not only achieve the best despeckling performance with ramp edge preservation but also reduce the staircase effect produced by integral-order filters. Then, we exploit the PAS metric in designing a new fractional-order diffusion coefficient to properly preserve low-contrast edges in diffusion filtering. Finally, different from fixed fractional-order diffusion filters, an adaptive fractional order is introduced based on the PAS metric to enhance various weak edges in the spatially transitional areas between objects. The proposed fractional TV model is minimized using the gradient descent method to obtain the final denoised image. The experimental results and real application of ultrasound breast image segmentation show that the proposed method outperforms other state-of-the-art ultrasound despeckling filters for both speckle reduction and feature preservation in terms of visual evaluation and quantitative indices., 12

Image textures, as a kind of local variations, provide important information for the human visual system. Many image textures, especially the small-scale or stochastic textures, are rich in high-frequency variations, and are difficult to be preserved. Current state-of-the-art denoising algorithms typically adopt a nonlocal approach consisting of image patch grouping and group-wise denoising filtering. To achieve a better image denoising while preserving the variations in texture, we first adaptively group high correlated image patches with the same kinds of texture elements (texels) via an adaptive clustering method. This adaptive clustering method is applied in an over-clustering-and-iterative-merging approach, where its noise robustness is improved with a custom merging threshold relating to the noise level and cluster size. For texture-preserving denoising of each cluster, considering that the variations in texture are captured and wrapped in not only the between-dimension energy variations but also the within-dimension variations of PCA transform coefficients, we further propose a PCA-transform-domain variation adaptive filtering method to preserve the local variations in textures. Experiments on natural images show the superiority of the proposed transform-domain variation adaptive filtering to traditional PCA-based hard or soft threshold filtering. As a whole, the proposed denoising method achieves a favorable texture-preserving performance both quantitatively and visually, especially for irregular textures, which is further verified in camera raw image denoising.

This letter proposes to extract contrast-filled vessels from overlapped noisy complex backgrounds in an x-ray coronary angiogram image sequence using low-rank and sparse decomposition. A refined vessel segmentation is finally achieved by implementing a radon-like feature filtering plus local-to-global adaptive thresholding to tackle the spatially varying noisy residuals in the extracted vessels. Based on real and synthetic XCA data, the experiment results demonstrate the superiority of the proposed method over the state-of-the-art methods.

Reconstructing magnetic resonance images from undersampled k-space data is a challenging problem. This paper introduces a novel method of image reconstruction from undersampled k-space data based on the concept of singularizing operators and a novel singular k-space model. Exploring the sparsity of an image in the k-space, the singular k-space model (SKM) is proposed in terms of the k-space functions of a singularizing operator. The singularizing operator is constructed by combining basic difference operators. An algorithm is developed to reliably estimate the model parameters from undersampled k-space data. The estimated parameters are then used to recover the missing k-space data through the model, subsequently achieving high-quality reconstruction of the image using inverse Fourier transform. Experiments on physical phantom and real brain MR images have shown that the proposed SKM method constantly outperforms the popular total variation (TV) and the classical zero-filling (ZF) methods regardless of the undersampling rates, the noise levels, and the image structures. For the same objective quality of the reconstructed images, the proposed method requires much less k-space data than the TV method. The SKM method is an effective method for fast MRI reconstruction from the undersampled k-space data. Graphical abstract Two Real Images and their sparsified images by singularizing operator.

Mutual information (MI) is a popular similarity measure for image registration, whereby good registration can be achieved by maximizing the compactness of the clusters in the joint histogram. However, MI is sensitive to the "outlier" objects that appear in one image but not the other, and also suffers from local and biased maxima. We propose a novel joint saliency map (JSM) to highlight the corresponding salient structures in the two images, and emphatically group those salient structures into the smoothed compact clusters in the weighted joint histogram. This strategy could solve both the outlier and the local maxima problems. Experimental results show that the JSM-MI based algorithm is not only accurate but also robust for registration of challenging image pairs with outliers., Preprint version for publication in IEEE Signal Processing Letters, 17(1):91-94, 2010

This paper proposes a novel global-to-local nonrigid brain MR image registration to compensate for the brain shift and the unmatchable outliers caused by the tumor resection. The mutual information between the corresponding salient structures, which are enhanced by the joint saliency map (JSM), is maximized to achieve a global rigid registration of the two images. Being detected and clustered at the paired contiguous matching areas in the globally registered images, the paired pools of DoG keypoints in combination with the JSM provide a useful cluster-to-cluster correspondence to guide the local control-point correspondence detection and the outlier keypoint rejection. Lastly, a quasi-inverse consistent deformation is smoothly approximated to locally register brain images through the mapping the clustered control points by compact support radial basis functions. The 2D implementation of the method can model the brain shift in brain tumor resection MR images, though the theory holds for the 3D case.

The problem addressed in this paper is matching corresponding regions in two images, even when the image has correspondence deficiency and local deformations. We present a novel algorithm for establishing region correspondences across images by graph matching and a novel local histogram based feature descriptor. Firstly, we segment the images into structural meaningful regions using simple linear iterative clustering (SLIC). Secondly, we extract the features of the regions and combine the most salient features into an attribute vector for the region. Thirdly, we use the spectral graph matching to compute the correspondence of the regions. The experimental results show that the algorithm has a good performance.

Modern computer science allows powerful and versatile computer-based knowledge representations of acupuncture, one part of Traditional Chinese Medicine. For further research and development of acupuncture therapy, it is critical to define where to accurately localize acupoints onto such a computer based pictorial representation of the human body. Using the segmentation and 3D visualization of the VOXEL-MAN software system, original work for localizing the acupoints on a head based of a virtual body is reported in this paper. The proposed 2D acupoint description links the description taken from literature for locating acupoints in Traditional Chinese Medicine to the data in the absolute reference frame of a 3D virtual body. It offers a simple and useful way for the localization of acupoints on a 3D model, especially one derived from the data from Visible Human Project.

Bioluminescence Imaging (BLI) is a powerful methodology for molecular imaging of small laboratory animals. This paper proposes our latest developed bioluminescence technology that enables the study of ongoing biological processes in vivo. This prototype technology includes the hardware/ software integrated system with some image restoration algorithm based on structure enhanced median filtering and morphological structure detection. Successful experiment results using in vivo small animal are presented in this paper.

In non-rigid image registration there are some distinct outliers introduced by the insertion or removal of some image contents, abrupt intensity changes, and local non-smooth deformations in the two images to register. Recently, non-rigid registration of images with outliers has received a great deal of attention in medical imaging research. This work presents a framework for robust non-rigid registration based on the theory of optical flow, within which we use outlier detection to implement locally-refined control point setup in multilevel free- form deformation (FFD) model. We also introduce robust M-estimator to weaken the effect of outliers in global optimization to get improved registration result. We have successfully applied our proposed method to a number of registration tasks to demonstrate its applicability and robustness to outliers.