1. Learning 3D medical image keypoint descriptors with the triplet loss
- Author
-
Sébastien Valette, Nicolas Loiseau–witon, Razmig Kéchichian, Adrien Bartoli, Centre de Recherche en Acquisition et Traitement de l'Image pour la Santé (CREATIS), Université Jean Monnet [Saint-Étienne] (UJM)-Hospices Civils de Lyon (HCL)-Institut National des Sciences Appliquées de Lyon (INSA Lyon), Université de Lyon-Institut National des Sciences Appliquées (INSA)-Université de Lyon-Institut National des Sciences Appliquées (INSA)-Université Claude Bernard Lyon 1 (UCBL), Université de Lyon-Centre National de la Recherche Scientifique (CNRS)-Institut National de la Santé et de la Recherche Médicale (INSERM), Imagerie Tomographique et Radiothérapie, Université de Lyon-Centre National de la Recherche Scientifique (CNRS)-Institut National de la Santé et de la Recherche Médicale (INSERM)-Université Jean Monnet [Saint-Étienne] (UJM)-Hospices Civils de Lyon (HCL)-Institut National des Sciences Appliquées de Lyon (INSA Lyon), Modeling & analysis for medical imaging and Diagnosis (MYRIAD), Institut Pascal (IP), Centre National de la Recherche Scientifique (CNRS)-Université Clermont Auvergne (UCA)-Institut national polytechnique Clermont Auvergne (INP Clermont Auvergne), Université Clermont Auvergne (UCA)-Université Clermont Auvergne (UCA), Valette, Sébastien, Université Claude Bernard Lyon 1 (UCBL), Université de Lyon-Université de Lyon-Institut National des Sciences Appliquées de Lyon (INSA Lyon), Université de Lyon-Institut National des Sciences Appliquées (INSA)-Institut National des Sciences Appliquées (INSA)-Hospices Civils de Lyon (HCL)-Université Jean Monnet - Saint-Étienne (UJM)-Institut National de la Santé et de la Recherche Médicale (INSERM)-Centre National de la Recherche Scientifique (CNRS), Université de Lyon-Institut National des Sciences Appliquées (INSA)-Institut National des Sciences Appliquées (INSA)-Hospices Civils de Lyon (HCL)-Université Jean Monnet - Saint-Étienne (UJM)-Institut National de la Santé et de la Recherche Médicale (INSERM)-Centre National de la Recherche Scientifique (CNRS)-Université Claude Bernard Lyon 1 (UCBL), Université de Lyon-Institut National des Sciences Appliquées (INSA)-Institut National des Sciences Appliquées (INSA)-Université Jean Monnet - Saint-Étienne (UJM)-Institut National de la Santé et de la Recherche Médicale (INSERM)-Centre National de la Recherche Scientifique (CNRS), Université de Lyon-Institut National des Sciences Appliquées (INSA)-Institut National des Sciences Appliquées (INSA)-Université Jean Monnet - Saint-Étienne (UJM)-Institut National de la Santé et de la Recherche Médicale (INSERM)-Centre National de la Recherche Scientifique (CNRS)-Université Claude Bernard Lyon 1 (UCBL), and ANR-19-CE45-0015,TOPACS,Traitement Ouvert de données PACS(2019)
- Subjects
Databases, Factual ,Matching (graph theory) ,Computer science ,[INFO.INFO-IM] Computer Science [cs]/Medical Imaging ,ComputingMethodologies_IMAGEPROCESSINGANDCOMPUTERVISION ,Biomedical Engineering ,Health Informatics ,02 engineering and technology ,Convolutional neural network ,Descriptors ,Synthetic data ,Image (mathematics) ,Imaging, Three-Dimensional ,Triplet loss ,[INFO.INFO-IM]Computer Science [cs]/Medical Imaging ,0202 electrical engineering, electronic engineering, information engineering ,Medical imaging ,Humans ,Radiology, Nuclear Medicine and imaging ,Convolution Neural Network ,business.industry ,Reproducibility of Results ,020207 software engineering ,Pattern recognition ,General Medicine ,Keypoints ,Computer Graphics and Computer-Aided Design ,Computer Science Applications ,High memory ,ComputingMethodologies_PATTERNRECOGNITION ,Benchmark (computing) ,020201 artificial intelligence & image processing ,Surgery ,Computer Vision and Pattern Recognition ,Affine transformation ,Artificial intelligence ,Tomography, X-Ray Computed ,business ,Algorithms - Abstract
International audience; Purpose: We propose to learn a 3D keypoint descriptor which we use to match keypoints extracted from full-body CT scans. Our methods are inspired by 2D keypoint descriptor learning, which was shown to outperform hand-crafted descriptors. Adapting these to 3D images is challenging because of the lack of labelled training data and high memory requirements.Method: We generate semi-synthetic training data. For that, we first estimate the distribution of local affine inter-subject transformations using labelled anatomical landmarks on a small subset of the database. We then sample a large number of transformations and warp unlabelled CT scans, for which we can subsequently establish reliable keypoint correspondences using guided matching. These correspondences serve as training data for our descriptor, which we represent by a CNN and train using the triplet loss with online triplet mining.Results: We carry out experiments on a synthetic data reliability benchmark and a registration task involving 20 CT volumes with anatomical landmarks used for evaluation purposes. Our learned descriptor outperforms the 3D-SURF descriptor on both benchmarks while having a similar runtime.Conclusion: We propose a new method to generate semi-synthetic data and a new learned 3D keypoint descriptor. Experiments show improvement compared to a hand-crafted descriptor. This is promising as literature has shown that jointly learning a detector and a descriptor gives further performance boost.
- Published
- 2021