Search

Purpose: Dual-energy CT (DECT) has been used to derive relative stopping power (RSP) map by obtaining the energy dependence of photon interactions. The DECT-derived RSP maps could potentially be compromised by image noise levels and the severity of artifacts when using physics-based mapping techniques, which would affect subsequent clinical applications. This work presents a noise-robust learning-based method to predict RSP maps from DECT for proton radiation therapy. Methods: The proposed method uses a residual attention cycle-consistent generative adversarial (CycleGAN) network. CycleGAN were used to let the DECT-to-RSP mapping be close to a one-to-one mapping by introducing an inverse RSP-to-DECT mapping. We retrospectively investigated 20 head-and-neck cancer patients with DECT scans in proton radiation therapy simulation. Ground truth RSP values were assigned by calculation based on chemical compositions, and acted as learning targets in the training process for DECT datasets, and were evaluated against results from the proposed method using a leave-one-out cross-validation strategy. Results: The predicted RSP maps showed an average normalized mean square error (NMSE) of 2.83% across the whole body volume, and average mean error (ME) less than 3% in all volumes of interest (VOIs). With additional simulated noise added in DECT datasets, the proposed method still maintained a comparable performance, while the physics-based stoichiometric method suffered degraded inaccuracy from increased noise level. The average differences in DVH metrics for clinical target volumes (CTVs) were less than 0.2 Gy for D95% and Dmax with no statistical significance. Conclusion: These results strongly indicate the high accuracy of RSP maps predicted by our machine-learning-based method and show its potential feasibility for proton treatment planning and dose calculation.

Understanding the structure of complex activities in untrimmed videos is a challenging task in the area of action recognition. One problem here is that this task usually requires a large amount of hand-annotated minute- or even hour-long video data, but annotating such data is very time consuming and can not easily be automated or scaled. To address this problem, this paper proposes an approach for the unsupervised learning of actions in untrimmed video sequences based on a joint visual-temporal embedding space. To this end, we combine a visual embedding based on a predictive U-Net architecture with a temporal continuous function. The resulting representation space allows detecting relevant action clusters based on their visual as well as their temporal appearance. The proposed method is evaluated on three standard benchmark datasets, Breakfast Actions, INRIA YouTube Instructional Videos, and 50 Salads. We show that the proposed approach is able to provide a meaningful visual and temporal embedding out of the visual cues present in contiguous video frames and is suitable for the task of unsupervised temporal segmentation of actions.

This paper presents a review of deep learning (DL) in multi-organ segmentation. We summarized the latest DL-based methods for medical image segmentation and applications. These methods were classified into six categories according to their network design. For each category, we listed the surveyed works, highlighted important contributions and identified specific challenges. Following the detailed review of each category, we briefly discussed its achievements, shortcomings and future potentials. We provided a comprehensive comparison among DL-based methods for thoracic and head & neck multiorgan segmentation using benchmark datasets, including the 2017 AAPM Thoracic Auto-segmentation Challenge datasets and 2015 MICCAI Head Neck Auto-Segmentation Challenge datasets., Comment: 37 pages, 2 figures, 8 tables

This paper reviewed the machine learning-based studies for quantitative positron emission tomography (PET). Specifically, we summarized the recent developments of machine learning-based methods in PET attenuation correction and low-count PET reconstruction by listing and comparing the proposed methods, study designs and reported performances of the current published studies with brief discussion on representative studies. The contributions and challenges among the reviewed studies were summarized and highlighted in the discussion part followed by., Comment: 25 pages, 2 tables

This paper presents a review of deep learning (DL) based medical image registration methods. We summarized the latest developments and applications of DL-based registration methods in the medical field. These methods were classified into seven categories according to their methods, functions and popularity. A detailed review of each category was presented, highlighting important contributions and identifying specific challenges. A short assessment was presented following the detailed review of each category to summarize its achievements and future potentials. We provided a comprehensive comparison among DL-based methods for lung and brain deformable registration using benchmark datasets. Lastly, we analyzed the statistics of all the cited works from various aspects, revealing the popularity and future trend of development in medical image registration using deep learning., Comment: 32 pages, 4 figures, 9 tables

As bone and air produce weak signals with conventional MR sequences, segmentation of these tissues particularly difficult in MRI. We propose to integrate patch-based anatomical signatures and an auto-context model into a machine learning framework to iteratively segment MRI into air, bone and soft tissue. The proposed semantic classification random forest (SCRF) method consists of a training stage and a segmentation stage. During training stage, patch-based anatomical features were extracted from registered MRI-CT training images, and the most informative features were identified to train a series of classification forests with auto-context model. During segmentation stage, we extracted selected features from MRI and fed them into the well-trained forests for MRI segmentation. The DSC for air, bone and soft tissue obtained with proposed SCRF were 0.976, 0.819 and 0.932, compared to 0.916, 0.673 and 0.830 with RF, 0.942, 0.791 and 0.917 with U-Net. SCRF also demonstrated superior segmentation performances for sensitivity and specificity over RF and U-Net for all three structure types. The proposed segmentation technique could be a useful tool to segment bone, air and soft tissue, and have the potential to be applied to attenuation correction of PET/MRI system, MRI-only radiation treatment planning and MR-guided focused ultrasound surgery., Comment: 18 pages, 8 figures

Radiomics is a nascent field in quantitative imaging that uses advanced algorithms and considerable computing power to describe tumor phenotypes, monitor treatment response, and assess normal tissue toxicity quantifiably. Remarkable interest has been drawn to the field due to its noninvasive nature and potential for diagnosing and predicting patient prognosis. This review will attempt to comprehensively and critically discuss the various aspects of radiomics including its workflow, applications to different modalities, potential applications in cancer radiotherapy, and limitations., Comment: 30 pages, 2 tables, 2 figures

How can we effectively engineer a computer vision system that is able to interpret videos from unconstrained mobility platforms like UAVs? One promising option is to make use of image restoration and enhancement algorithms from the area of computational photography to improve the quality of the underlying frames in a way that also improves automatic visual recognition. Along these lines, exploratory work is needed to find out which image pre-processing algorithms, in combination with the strongest features and supervised machine learning approaches, are good candidates for difficult scenarios like motion blur, weather, and mis-focus -- all common artifacts in UAV acquired images. This paper summarizes the protocols and results of Track 1 of the UG^2+ Challenge held in conjunction with IEEE/CVF CVPR 2019. The challenge looked at two separate problems: (1) object detection improvement in video, and (2) object classification improvement in video. The challenge made use of the UG^2 (UAV, Glider, Ground) dataset, which is an established benchmark for assessing the interplay between image restoration and enhancement and visual recognition. 16 algorithms were submitted by academic and corporate teams, and a detailed analysis of how they performed on each challenge problem is reported here., Comment: Supplemental material: http://bit.ly/UG2Supp

The UG$^{2+}$ challenge in IEEE CVPR 2019 aims to evoke a comprehensive discussion and exploration about how low-level vision techniques can benefit the high-level automatic visual recognition in various scenarios. In its second track, we focus on object or face detection in poor visibility enhancements caused by bad weathers (haze, rain) and low light conditions. While existing enhancement methods are empirically expected to help the high-level end task, that is observed to not always be the case in practice. To provide a more thorough examination and fair comparison, we introduce three benchmark sets collected in real-world hazy, rainy, and low-light conditions, respectively, with annotate objects/faces annotated. To our best knowledge, this is the first and currently largest effort of its kind. Baseline results by cascading existing enhancement and detection models are reported, indicating the highly challenging nature of our new data as well as the large room for further technical innovations. We expect a large participation from the broad research community to address these challenges together., Comment: A summary paper on datasets, fact sheets, baseline results, challenge results, and winning methods in UG$^{2+}$ Challenge (Track 2). More materials are provided in http://www.ug2challenge.org/index.html

The subtleties of human perception, as measured by vision scientists through the use of psychophysics, are important clues to the internal workings of visual recognition. For instance, measured reaction time can indicate whether a visual stimulus is easy for a subject to recognize, or whether it is hard. In this paper, we consider how to incorporate psychophysical measurements of visual perception into the loss function of a deep neural network being trained for a recognition task, under the assumption that such information can enforce consistency with human behavior. As a case study to assess the viability of this approach, we look at the problem of handwritten document transcription. While good progress has been made towards automatically transcribing modern handwriting, significant challenges remain in transcribing historical documents. Here we describe a general enhancement strategy, underpinned by the new loss formulation, which can be applied to the training regime of any deep learning-based document transcription system. Through experimentation, reliable performance improvement is demonstrated for the standard IAM and RIMES datasets for three different network architectures. Further, we go on to show feasibility for our approach on a new dataset of digitized Latin manuscripts, originally produced by scribes in the Cloister of St. Gall in the the 9th century.