Shape registration with learned deformations for 3D shape reconstruction from sparse and incomplete point clouds.

• Deep learning-based cardiac shape reconstruction from stacked 2D contours. • Learning-based mesh-to-point cloud deformable shape registration framework. • Accurate shape reconstruction in the presence of incomplete/noisy contours. • The proposed method significantly outperforms baseline methods on various metrics. • Potential use in the reconstruction of other anatomical structures and real-time applications. [Display omitted] Shape reconstruction from sparse point clouds/images is a challenging and relevant task required for a variety of applications in computer vision and medical image analysis (e.g. surgical navigation, cardiac motion analysis, augmented/virtual reality systems). A subset of such methods, viz. 3D shape reconstruction from 2D contours, is especially relevant for computer-aided diagnosis and intervention applications involving meshes derived from multiple 2D image slices, views or projections. We propose a deep learning architecture, coined Mesh Reconstruction Network (MR-Net), which tackles this problem. MR-Net enables accurate 3D mesh reconstruction in real-time despite missing data and with sparse annotations. Using 3D cardiac shape reconstruction from 2D contours defined on short-axis cardiac magnetic resonance image slices as an exemplar, we demonstrate that our approach consistently outperforms state-of-the-art techniques for shape reconstruction from unstructured point clouds. Our approach can reconstruct 3D cardiac meshes to within 2.5-mm point-to-point error, concerning the ground-truth data (the original image spatial resolution is ∼ 1.8 × 1.8 × 10 mm 3). We further evaluate the robustness of the proposed approach to incomplete data, and contours estimated using an automatic segmentation algorithm. MR-Net is generic and could reconstruct shapes of other organs, making it compelling as a tool for various applications in medical image analysis. [ABSTRACT FROM AUTHOR]