Back to Search
Start Over
JSSR: A Joint Synthesis, Segmentation, and Registration System for 3D Multi-modal Image Alignment of Large-Scale Pathological CT Scans
- Source :
- Computer Vision – ECCV 2020 ISBN: 9783030586003, ECCV (13)
- Publication Year :
- 2020
- Publisher :
- Springer International Publishing, 2020.
-
Abstract
- Multi-modal image registration is a challenging problem that is also an important clinical task for many real applications and scenarios. As a first step in analysis, deformable registration among different image modalities is often required in order to provide complementary visual information. During registration, semantic information is key to match homologous points and pixels. Nevertheless, many conventional registration methods are incapable in capturing high-level semantic anatomical dense correspondences. In this work, we propose a novel multi-task learning system, JSSR, based on an end-to-end 3D convolutional neural network that is composed of a generator, a registration and a segmentation component. The system is optimized to satisfy the implicit constraints between different tasks in an unsupervised manner. It first synthesizes the source domain images into the target domain, then an intra-modal registration is applied on the synthesized images and target images. The segmentation module are then applied on the synthesized and target images, providing additional cues based on semantic correspondences. The supervision from another fully-annotated dataset is used to regularize the segmentation. We extensively evaluate JSSR on a large-scale medical image dataset containing 1,485 patient CT imaging studies of four different contrast phases (i.e., 5,940 3D CT scans with pathological livers) on the registration, segmentation and synthesis tasks. The performance is improved after joint training on the registration and segmentation tasks by \(0.9\%\) and \(1.9\%\) respectively compared to a highly competitive and accurate deep learning baseline. The registration also consistently outperforms conventional state-of-the-art multi-modal registration methods.
- Subjects :
- Pixel
Computer science
business.industry
Deep learning
ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION
Image registration
Convolutional neural network
030218 nuclear medicine & medical imaging
Domain (software engineering)
Image (mathematics)
03 medical and health sciences
Task (computing)
0302 clinical medicine
Segmentation
Computer vision
Artificial intelligence
business
030217 neurology & neurosurgery
Subjects
Details
- ISBN :
- 978-3-030-58600-3
- ISBNs :
- 9783030586003
- Database :
- OpenAIRE
- Journal :
- Computer Vision – ECCV 2020 ISBN: 9783030586003, ECCV (13)
- Accession number :
- edsair.doi...........7c4858a0b3b4f665943acae1a227ca01
- Full Text :
- https://doi.org/10.1007/978-3-030-58601-0_16