Search

Partly due to the use of exhaustive-annotated data, deep networks have achieved impressive performance on medical image segmentation. Medical imaging data paired with noisy annotation are, however, ubiquitous, but little is known about the effect of noisy annotation on deep learning based medical image segmentation. We studied the effect of noisy annotation in the context of mandible segmentation from CT images. First, 202 images of head and neck cancer patients were collected from our clinical database, where the organs-at-risk were annotated by one of twelve planning dosimetrists. The mandibles were roughly annotated as the planning avoiding structure. Then, mandible labels were checked and corrected by a head and neck specialist to get the reference standard. At last, by varying the ratios of noisy labels in the training set, deep networks were trained and tested for mandible segmentation. The trained models were further tested on other two public datasets. Experimental results indicated that the network trained with noisy labels had worse segmentation than that trained with reference standard, and in general, fewer noisy labels led to better performance. When using 20% or less noisy cases for training, no significant difference was found on the segmentation results between the models trained by noisy or reference annotation. Cross-dataset validation results verified that the models trained with noisy data achieved competitive performance to that trained with reference standard. This study suggests that the involved network is robust to noisy annotation to some extent in mandible segmentation from CT images. It also highlights the importance of labeling quality in deep learning. In the future work, extra attention should be paid to how to utilize a small number of reference standard samples to improve the performance of deep learning with noisy annotation.

We propose a novel BIRADS-SSDL network that integrates clinically-approved breast lesion characteristics (BIRADS features) into task-oriented semi-supervised deep learning (SSDL) for accurate diagnosis of ultrasound (US) images with a small training dataset. Breast US images are converted to BIRADS-oriented feature maps (BFMs) using a distance-transformation coupled with a Gaussian filter. Then, the converted BFMs are used as the input of an SSDL network, which performs unsupervised stacked convolutional auto-encoder (SCAE) image reconstruction guided by lesion classification. This integrated multi-task learning allows SCAE to extract image features with the constraints from the lesion classification task, while the lesion classification is achieved by utilizing the SCAE encoder features with a convolutional network. We trained the BIRADS-SSDL network with an alternative learning strategy by balancing the reconstruction error and classification label prediction error. To demonstrate the effectiveness of our approach, we evaluated it using two breast US image datasets. We compared the performance of the BIRADS-SSDL network with conventional SCAE and SSDL methods that use the original images as inputs, as well as with an SCAE that use BFMs as inputs. The experimental results on two breast US datasets show that BIRADS-SSDL ranked the best among the four networks, with a classification accuracy of around 94.23 ± 3.33% and 84.38 ± 3.11% on two datasets. In the case of experiments across two datasets collected from two different institutions/and US devices, the developed BIRADS-SSDL is generalizable across the different US devices and institutions without overfitting to a single dataset and achieved satisfactory results. Furthermore, we investigate the performance of the proposed method by varying the model training strategies, lesion boundary accuracy, and Gaussian filter parameters. The experimental results showed that a pre-training strategy can help to speed up model convergence during training but with no improvement of the classification accuracy on the testing dataset. The classification accuracy decreases as the segmentation accuracy decreases. The proposed BIRADS-SSDL achieves the best results among the compared methods in each case and has the capacity to deal with multiple different datasets under one model. Compared with state-of-the-art methods, BIRADS-SSDL could be promising for effective breast US computer-aided diagnosis using small datasets.

Throughout the course of delivering a radiation therapy treatment, which may take several weeks, a patient's anatomy may change drastically, and adaptive radiation therapy (ART) may be needed. Cone-beam computed tomography (CBCT), which is often available during the treatment process, can be used for both patient positioning and ART re-planning. However, due to the prominent amount of noise, artifacts, and inaccurate Hounsfield unit (HU) values, the dose calculation based on CBCT images could be inaccurate for treatment planning. One way to solve this problem is to convert CBCT images to more accurate synthesized CT (sCT) images. In this work, we have developed a cycle-consistent generative adversarial network framework (CycleGAN) to synthesize CT images from CBCT images. This model is capable of image-to-image translation using unpaired CT and CBCT images in an unsupervised learning setting. The sCT images generated from CBCT through this CycleGAN model are visually and quantitatively similar to real CT images with decreased mean absolute error (MAE) from 69.29 HU to 29.85 HU for head-and-neck (H&N) cancer patients. The dose distributions calculated on the sCT by CycleGAN demonstrated a higher accuracy than those on CBCT in a 3D gamma index analysis with increased gamma index pass rate from 86.92% to 96.26% under 1 mm/1% criteria, when using the deformed planning CT image (dpCT) as the reference. We also compared the CycleGAN model with other unsupervised learning methods, including deep convolutional generative adversarial networks (DCGAN) and progressive growing of GANs (PGGAN), and demonstrated that CycleGAN outperformed the other two models. A phantom study has been conducted to compare sCT with dpCT, and the increase of structural similarity index from 0.91 to 0.93 shows that CycleGAN performed better than DIR in terms of preserving anatomical accuracy.

The aim of this work is to develop a novel recursive ensemble OARs segmentation (REOS) framework for accurate organs-at-risk (OARs) automatic segmentation. The REOS recursively segment individual OARs by ensembling images features extracted from an organ localization module and a contour detection module. Both modules are based on a 3D U-Net architecture. The organ localization module is trained for rough segmentation to localize a region of interest (ROI) that encompasses the to-be-delineated OAR, while the contour detection module is trained to segment the OAR within the identified ROI. In this study, the developed REOS framework is applied for brain radiotherapy on segmenting six OARs including the eyes, the brainstem (BS), the optical nerves and the chiasm. Eighty T1-weighted magnetic resonance images (MRI) from 80 brain cancer patients' cases with OARs' gold standard contours were collected for training and testing REOS. On 20 testing cases, the REOS achieve a high segmentation accuracy with Dice similarity coefficient (DSC) mean and standard deviation of 93.9%  ±  1.4%, 94.5%  ±  2.0%, 90.6%  ±  2.7%, on the left and right eyes and the BS, respectively. On small and segmentation-challenging organs, the left and right optical nerves and the chiasm, the REOS achieves DSC of 78.0%  ±  10.5%, 82.2%  ±  5.9% and 71.1%  ±  9.1%. The satisfactory performances demonstrated the effectiveness of the REOS in OARs segmentation.

The purpose of this work is to develop an automated landmark-guided deformable image registration (LDIR) algorithm between the planning CT and daily cone-beam CT (CBCT) with low image quality. This method uses an automated landmark generation algorithm in conjunction with a local small volume gradient matching search engine to map corresponding landmarks between the CBCT and the planning CT. The landmarks act as stabilizing control points in the following Demons deformable image registration. LDIR is implemented on graphics processing units (GPUs) for parallel computation to achieve ultra fast calculation. The accuracy of the LDIR algorithm has been evaluated on a synthetic case in the presence of different noise levels and data of six head and neck cancer patients. The results indicate that LDIR performed better than rigid registration, Demons, and intensity corrected Demons for all similarity metrics used. In conclusion, LDIR achieves high accuracy in the presence of multimodality intensity mismatch and CBCT noise contamination, while simultaneously preserving high computational efficiency.

With the aim of maximally reducing imaging dose while meeting requirements for adaptive radiation therapy (ART), we propose in this paper a new cone beam CT (CBCT) acquisition and reconstruction method that delivers images with a low noise level inside a region of interest (ROI) and a relatively high noise level outside the ROI. The acquired projection images include two groups: densely sampled projections at a low exposure with a large field of view (FOV) and sparsely sampled projections at a high exposure with a small FOV corresponding to the ROI. A new algorithm combining the conventional filtered back-projection algorithm and the tight-frame iterative reconstruction algorithm is also designed to reconstruct the CBCT based on these projection data. We have validated our method on a simulated head-and-neck (HN) patient case, a semi-real experiment conducted on a HN cancer patient under a full-fan scan mode, as well as a Catphan phantom under a half-fan scan mode. Relative root-mean-square errors (RRMSEs) of less than 3% for the entire image and ~1% within the ROI compared to the ground truth have been observed. These numbers demonstrate the ability of our proposed method to reconstruct high-quality images inside the ROI. As for the part outside ROI, although the images are relatively noisy, it can still provide sufficient information for radiation dose calculations in ART. Dose distributions calculated on our CBCT image and on a standard CBCT image are in agreement, with a mean relative difference of 0.082% inside the ROI and 0.038% outside the ROI. Compared with the standard clinical CBCT scheme, an imaging dose reduction of approximately 3-6 times inside the ROI was achieved, as well as an 8 times outside the ROI. Regarding computational efficiency, it takes 1-3 min to reconstruct a CBCT image depending on the number of projections used. These results indicate that the proposed method has the potential for application in ART.

Purpose: Image reconstruction and motion model estimation in four-dimensional cone-beam CT (4D-CBCT) are conventionally handled as two sequential steps. Due to the limited number of projections at each phase, the image quality of 4D-CBCT is degraded by view aliasing artifacts, and the accuracy of subsequent motion modeling is decreased by the inferior 4D-CBCT. The objective of this work is to enhance both the image quality of 4D-CBCT and the accuracy of motion model estimation with a novel strategy enabling simultaneous motion estimation and image reconstruction (SMEIR)., Methods: The proposed SMEIR algorithm consists of two alternating steps: (1) model-based iterative image reconstruction to obtain a motion-compensated primary CBCT (m-pCBCT) and (2) motion model estimation to obtain an optimal set of deformation vector fields (DVFs) between the m-pCBCT and other 4D-CBCT phases. The motion-compensated image reconstruction is based on the simultaneous algebraic reconstruction technique (SART) coupled with total variation minimization. During the forward- and backprojection of SART, measured projections from an entire set of 4D-CBCT are used for reconstruction of the m-pCBCT by utilizing the updated DVF. The DVF is estimated by matching the forward projection of the deformed m-pCBCT and measured projections of other phases of 4D-CBCT. The performance of the SMEIR algorithm is quantitatively evaluated on a 4D NCAT phantom. The quality of reconstructed 4D images and the accuracy of tumor motion trajectory are assessed by comparing with those resulting from conventional sequential 4D-CBCT reconstructions (FDK and total variation minimization) and motion estimation (demons algorithm). The performance of the SMEIR algorithm is further evaluated by reconstructing a lung cancer patient 4D-CBCT., Results: Image quality of 4D-CBCT is greatly improved by the SMEIR algorithm in both phantom and patient studies. When all projections are used to reconstruct a 3D-CBCT by FDK, motion-blurring artifacts are present, leading to a 24.4% relative reconstruction error in the NACT phantom. View aliasing artifacts are present in 4D-CBCT reconstructed by FDK from 20 projections, with a relative error of 32.1%. When total variation minimization is used to reconstruct 4D-CBCT, the relative error is 18.9%. Image quality of 4D-CBCT is substantially improved by using the SMEIR algorithm and relative error is reduced to 7.6%. The maximum error (MaxE) of tumor motion determined from the DVF obtained by demons registration on a FDK-reconstructed 4D-CBCT is 3.0, 2.3, and 7.1 mm along left-right (L-R), anterior-posterior (A-P), and superior-inferior (S-I) directions, respectively. From the DVF obtained by demons registration on 4D-CBCT reconstructed by total variation minimization, the MaxE of tumor motion is reduced to 1.5, 0.5, and 5.5 mm along L-R, A-P, and S-I directions. From the DVF estimated by SMEIR algorithm, the MaxE of tumor motion is further reduced to 0.8, 0.4, and 1.5 mm along L-R, A-P, and S-I directions, respectively., Conclusions: The proposed SMEIR algorithm is able to estimate a motion model and reconstruct motion-compensated 4D-CBCT. The SMEIR algorithm improves image reconstruction accuracy of 4D-CBCT and tumor motion trajectory estimation accuracy as compared to conventional sequential 4D-CBCT reconstruction and motion estimation.

In adaptive radiotherapy, deformable image registration is often conducted between the planning CT and treatment CT (or cone beam CT) to generate a deformation vector field (DVF) for dose accumulation and contour propagation. The auto-propagated contours on the treatment CT may contain relatively large errors, especially in low-contrast regions. A clinician's inspection and editing of the propagated contours are frequently needed. The edited contours are able to meet the clinical requirement for adaptive therapy; however, the DVF is still inaccurate and inconsistent with the edited contours. The purpose of this work is to develop a contour-guided deformable image registration (CG-DIR) algorithm to improve the accuracy and consistency of the DVF for adaptive radiotherapy. Incorporation of the edited contours into the registration algorithm is realized by regularizing the objective function of the original demons algorithm with a term of intensity matching between the delineated structures set pairs. The CG-DIR algorithm is implemented on computer graphics processing units (GPUs) by following the original GPU-based demons algorithm computation framework (Gu et al 2010 Phys Med Biol. 55 207-219). The performance of CG-DIR is evaluated on five clinical head-and-neck and one pelvic cancer patient data. It is found that compared with the original demons, CG-DIR improves the accuracy and consistency of the DVF, while retaining similar high computational efficiency.

Computed tomography (CT) to cone-beam CT (CBCT) deformable image registration (DIR) is a crucial step in adaptive radiation therapy. Current intensity-based registration algorithms, such as demons, may fail in the context of CT-CBCT DIR because of inconsistent intensities between the two modalities. In this paper, we propose a variant of demons, called deformation with intensity simultaneously corrected (DISC), to deal with CT-CBCT DIR. DISC distinguishes itself from the original demons algorithm by performing an adaptive intensity correction step on the CBCT image at every iteration step of the demons registration. Specifically, the intensity correction of a voxel in CBCT is achieved by matching the first and the second moments of the voxel intensities inside a patch around the voxel with those on the CT image. It is expected that such a strategy can remove artifacts in the CBCT image, as well as ensuring the intensity consistency between the two modalities. DISC is implemented on computer graphics processing units in compute unified device architecture (CUDA) programming environment. The performance of DISC is evaluated on a simulated patient case and six clinical head-and-neck cancer patient data. It is found that DISC is robust against the CBCT artifacts and intensity inconsistency and significantly improves the registration accuracy when compared with the original demons.

Online adaptive radiation therapy (ART) promises the ability to deliver an optimal treatment in response to daily patient anatomic variation. A major technical barrier for the clinical implementation of online ART is the requirement of rapid image segmentation. Deformable image registration (DIR) has been used as an automated segmentation method to transfer tumor/organ contours from the planning image to daily images. However, the current computational time of DIR is insufficient for online ART. In this work, this issue is addressed by using computer graphics processing units (GPUs). A gray-scale-based DIR algorithm called demons and five of its variants were implemented on GPUs using the compute unified device architecture (CUDA) programming environment. The spatial accuracy of these algorithms was evaluated over five sets of pulmonary 4D CT images with an average size of 256 x 256 x 100 and more than 1100 expert-determined landmark point pairs each. For all the testing scenarios presented in this paper, the GPU-based DIR computation required around 7 to 11 s to yield an average 3D error ranging from 1.5 to 1.8 mm. It is interesting to find out that the original passive force demons algorithms outperform subsequently proposed variants based on the combination of accuracy, efficiency and ease of implementation.

It is well known that the inverse problem in optical tomography is highly ill-posed. The image reconstruction process is often unstable and nonunique, because the number of the boundary measurements data is far fewer than the number of the unknown parameters to be reconstructed. To overcome this problem, one can either increase the number of measurement data (e.g., multispectral or multifrequency methods), or reduce the number of unknowns (e.g., using prior structural information from other imaging modalities). We introduce a novel approach for reducing the unknown parameters in the reconstruction process. The discrete cosine transform (DCT), which has long been used in image compression, is here employed to parameterize the reconstructed image. In general, only a few DCT coefficients are needed to describe the main features in an optical tomographic image. Thus, the number of unknowns in the image reconstruction process can be drastically reduced. We show numerical and experimental examples that illustrate the performance of the new algorithm as compared to a standard model-based iterative image reconstructions scheme. We especially focus on the influence of initial guesses and noise levels on the reconstruction results.

We present a carefully designed phantom experimental study aimed to provide solid evidence that both absorption and scattering images of heterogeneous scattering media can be reconstructed independently from dc data. We also study the important absorption-scattering cross-talk issue. In this regard, we develop a simple normalizing scheme that is incorporated into our nonlinear finite-element-based reconstruction algorithm. Our results from the controlled phantom experiments show that the cross talk of an absorption object appearing in scattering images can be eliminated and that the cross talk of a scattering object appearing in absorption images can be reduced considerably. In addition, these carefully designed phantom experiments clearly suggest that both absorption and scattering images can be simultaneously recovered and quantitatively separated in highly scattering media by use of dc measurements. Finally, we discuss our results in light of recent theoretical findings on nonuniqueness for dc image reconstruction.