Forward and Backward Propagation of Stereo Matching Cost for Incremental Refinement of Multiview Disparity Maps

This study proposes a multiview stereo (MVS) method that is based on the forward and backward propagation of matching costs for the incremental refinement of multiview disparity maps. The volume-based MVS method requires numerous three-dimensional (3D) memory volumes to compute, store, and process the stereo matching costs. However, owing to memory limitations, conventional MVS methods allocate the memory of the 3D cost volumes only to the reference and its neighbor views. Thus, each reference view can only use the matching costs from a limited number of neighbor views. This study addresses this inherent MVS problem and proposes a new method by employing a forward and backward cost propagation (FBCP). First, a subpart of the input views is used to obtain disparity maps with a dense MVS method. Once all matching costs of the subpart views are sufficiently refined, the FBCP is performed for a new neighbor view. Immediately after the cost volume of the new view is computed, all matching costs of the subpart are forward propagated and fused with the initial cost of the new view. Furthermore, the new fused cost is backward propagated into the subpart to refine the previous costs again using the new fused cost. All cost volumes can be incrementally computed and refined without any limitation on the number of views using the proposed FBCP scheme. In the final step, all disparity maps are obtained from the refined cost volumes and fused into single point clouds. Moreover, we propose the use of surface consensus to obtain accurate fused point clouds for the fusion of the disparity maps. The performance of the proposed method is evaluated using the fused point clouds. The proposed method achieves less than 0.5 mm in mean distance error and about 82 percentage F-score within 2 mm distance threshold value.