Search

Real-time scene reconstruction from depth data inevitably suffers from occlusion, thus leading to incomplete 3D models. Partial reconstructions, in turn, limit the performance of algorithms that leverage them for applications in the context of, e.g., augmented reality, robotic navigation, and 3D mapping. Most methods address this issue by predicting the missing geometry as an offline optimization, thus being incompatible with real-time applications. We propose a framework that ameliorates this issue by performing scene reconstruction and semantic scene completion jointly in an incremental and real-time manner, based on an input sequence of depth maps. Our framework relies on a novel neural architecture designed to process occupancy maps and leverages voxel states to accurately and efficiently fuse semantic completion with the 3D global model. We evaluate the proposed approach quantitatively and qualitatively, demonstrating that our method can obtain accurate 3D semantic scene completion in real-time.

We propose an online RGB-D based scene understanding method for indoor scenes running in real-time on mobile devices. First, we incrementally reconstruct the scene via SLAM and compute a 3D geometric segmentation by fusing segments obtained from each input depth image in a global 3D model. We combine this geometric segmentation with semantic annotations to obtain a semantic segmentation in form of a semantic map. To accomplish efficient semantic segmentation, we encode the segments in the global model with a fast incremental 3D descriptor and use a random forest to determine its semantic label. The predictions from successive frames are then fused to obtain a confident semantic class across time. As a result, the overall method achieves an accuracy that gets close to most state-of-the-art 3D scene understanding methods while being much more efficient, enabling real-time execution on low-power embedded systems.

Left ventricular function was assessed by two-dimensional echocardiography before and one year after coronary artery bypass grafting(CABG) in a series of patients with severe coronary artery disease with diabetes mellitus(DM) and without DM(non-DM).Twenty-three patients with DM and 50 patients without DM, all with no previous myocardial infarction, underwent two-dimensional echocardiography before CABG and one year after CABG, in a non-matched study. For a matched study, 31 patients without DM who had almost the same left ventricular function as DM patients at the baseline were selected to and compare the rate of improvement in left ventricular function between the DM group and the "matched" non-DM group.In the non-matched study, patient characteristics were not significantly different between the 2 groups except for the incidence of congestive heart failure within one year before CABG, which was significantly higher in the DM group. Fractional shortening was significantly lower in the DM group at the baseline(p0.05) and also one year after CABG(p0.0001). Significant improvement in fractional shortening was seen in the non-DM group(p0.001), but not in the DM group. The left ventricular end-diastolic diameter(LVDd) was significantly larger in the DM group at the baseline(p0.01), and was still significantly larger in the DM group at one year after CABG(p0.01). No improvement in LVDd was seen in the DM group. In the matched study, fractional shortening of the non-DM group also showed significant improvement after CABG(p0.001). Moreover, the rate of improvement in fractional shortening was higher in the non-DM group than in the DM group(p0.05). LVDd tended to be larger in the DM group(p = NS).Left ventricular dysfunction and left ventricular impaired improvement were seen in the patients with DM, and CABG improved left ventricular function in the patients without DM with poor left ventricular function. These findings indicate that CABG therapy may be inadequate for improving left ventricular function in patients with DM and severe left ventricular dysfunction at the baseline.