1. Generating multi-pathological and multi-modal images and labels for brain MRI.
- Author
-
Fernandez V, Pinaya WHL, Borges P, Graham MS, Tudosiu PD, Vercauteren T, and Cardoso MJ
- Subjects
- Humans, Brain diagnostic imaging, Deep Learning, Multimodal Imaging methods, Algorithms, Imaging, Three-Dimensional methods, Image Interpretation, Computer-Assisted methods, Image Processing, Computer-Assisted methods, Magnetic Resonance Imaging methods
- Abstract
The last few years have seen a boom in using generative models to augment real datasets, as synthetic data can effectively model real data distributions and provide privacy-preserving, shareable datasets that can be used to train deep learning models. However, most of these methods are 2D and provide synthetic datasets that come, at most, with categorical annotations. The generation of paired images and segmentation samples that can be used in downstream, supervised segmentation tasks remains fairly uncharted territory. This work proposes a two-stage generative model capable of producing 2D and 3D semantic label maps and corresponding multi-modal images. We use a latent diffusion model for label synthesis and a VAE-GAN for semantic image synthesis. Synthetic datasets provided by this model are shown to work in a wide variety of segmentation tasks, supporting small, real datasets or fully replacing them while maintaining good performance. We also demonstrate its ability to improve downstream performance on out-of-distribution data., Competing Interests: Declaration of competing interest The authors declare the following financial interests/personal relationships which may be considered as potential competing interests: Virginia Fernandez reports financial support, administrative support, and travel were provided by EPSRC Centre for Doctoral Training in Smart Medical Imaging. Tom Vercauteren reports a relationship with Hypervision Surgical that includes: board membership and equity or stocks. If there are other authors, they declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper., (Copyright © 2024 The Authors. Published by Elsevier B.V. All rights reserved.)
- Published
- 2024
- Full Text
- View/download PDF