Search

Abstract In this study, we propose a novel point cloud based 3D registration and segmentation framework using reinforcement learning. An artificial agent, implemented as a distinct actor based on value networks, is trained to predict the optimal piece-wise linear transformation of a point cloud for the joint tasks of registration and segmentation. The actor network estimates a set of plausible actions and the value network aims to select the optimal action for the current observation. Point-wise features that comprise spatial positions (and surface normal vectors in the case of structured meshes), and their corresponding image features, are used to encode the observation and represent the underlying 3D volume. The actor and value networks are applied iteratively to estimate a sequence of transformations that enable accurate delineation of object boundaries. The proposed approach was extensively evaluated in both segmentation and registration tasks using a variety of challenging clinical datasets. Our method has fewer trainable parameters and lower computational complexity compared to the 3D U-Net, and it is independent of the volume resolution. We show that the proposed method is applicable to mono- and multi-modal segmentation tasks, achieving significant improvements over the state-of-the-art for the latter. The flexibility of the proposed framework is further demonstrated for a multi-modal registration application. As we learn to predict actions rather than a target, the proposed method is more robust compared to the 3D U-Net when dealing with previously unseen datasets, acquired using different protocols or modalities. As a result, the proposed method provides a promising multi-purpose segmentation and registration framework, particular in the context of image-guided interventions.

Background and purpose: Based on artificial intelligence (AI), 3D angiography (3DA) is a novel postprocessing algorithm for “DSA-like” 3D imaging of cerebral vasculature. Because 3DA requires neither mask runs nor digital subtraction as the current standard 3D-DSA does, it has the potential to cut the patient dose by 50%. The object was to evaluate 3DA’s diagnostic value for visualization of intracranial artery stenoses (IAS) compared to 3D-DSA. Materials and methods: 3D-DSA datasets of IAS (nIAS = 10) were postprocessed using conventional and prototype software (Siemens Healthineers AG, Erlangen, Germany). Matching reconstructions were assessed by two experienced neuroradiologists in consensus reading, considering image quality (IQ), vessel diameters (VD1/2), vessel-geometry index (VGI = VD1/VD2), and specific qualitative/quantitative parameters of IAS (e.g., location, visual IAS grading [low-/medium-/high-grade] and intra-/poststenotic diameters [dintra-/poststenotic in mm]). Using the NASCET criteria, the percentual degree of luminal restriction was calculated. Results: In total, 20 angiographic 3D volumes (n3DA = 10; n3D-DSA = 10) were successfully reconstructed with equivalent IQ. Assessment of the vessel geometry in 3DA datasets did not differ significantly from 3D-DSA (VD1: r = 0.994, p = 0.0001; VD2:r = 0.994, p = 0.0001; VGI: r = 0.899, p = 0.0001). Qualitative analysis of IAS location (3DA/3D-DSA:nICA/C4 = 1, nICA/C7 = 1, nMCA/M1 = 4, nVA/V4 = 2, nBA = 2) and the visual IAS grading (3DA/3D-DSA:nlow-grade = 3, nmedium-grade = 5, nhigh-grade = 2) revealed identical results for 3DA and 3D-DSA, respectively. Quantitative IAS assessment showed a strong correlation regarding intra-/poststenotic diameters (rdintrastenotic = 0.995, pdintrastenotic = 0.0001; rdpoststenotic = 0.995, pdpoststenotic = 0.0001) and the percentual degree of luminal restriction (rNASCET 3DA = 0.981; pNASCET 3DA = 0.0001). Conclusions: The AI-based 3DA is a resilient algorithm for the visualization of IAS and shows comparable results to 3D-DSA. Hence, 3DA is a promising new method that allows a considerable patient-dose reduction, and its clinical implementation would be highly desirable.

For complex segmentation tasks, the achievable accuracy of fully automated systems is inherently limited. Specifically, when a precise segmentation result is desired for a small amount of given data sets, semi-automatic methods exhibit a clear benefit for the user. The optimization of human computer interaction (HCI) is an essential part of interactive image segmentation. Nevertheless, publications introducing novel interactive segmentation systems (ISS) often lack an objective comparison of HCI aspects. It is demonstrated that even when the underlying segmentation algorithm is the same throughout interactive prototypes, their user experience may vary substantially. As a result, users prefer simple interfaces as well as a considerable degree of freedom to control each iterative step of the segmentation. In this article, an objective method for the comparison of ISS is proposed, based on extensive user studies. A summative qualitative content analysis is conducted via abstraction of visual and verbal feedback given by the participants. A direct assessment of the segmentation system is executed by the users via the system usability scale (SUS) and AttrakDiff-2 questionnaires. Furthermore, an approximation of the findings regarding usability aspects in those studies is introduced, conducted solely from the system-measurable user actions during their usage of interactive segmentation prototypes. The prediction of all questionnaire results has an average relative error of 8.9%, which is close to the expected precision of the questionnaire results themselves. This automated evaluation scheme may significantly reduce the resources necessary to investigate each variation of a prototype’s user interface (UI) features and segmentation methodologies.