Search

Video object segmentation (VOS) has been widely used in the fields of computer vision. However, existing VOS algorithms have drawbacks, such as difficulty with object deformation, occlusion, and fast motion. We therefore propose an effective VOS algorithm based on semantic visual words matching. Specifically, given the support frame and its corresponding mask, the frame is firstly input to the encoder with an embedding layer, and then a clustering algorithm is followed to generate a group of semantic visual words according to its mask. For a query frame to be segmented, a matching operation is performed against words generated from the support frame. In this manner, each pixel on query frame can be classified into different object categories by the obtained similarity. What's more, a self-attention mechanism is applied to enhance the embedding features in order to capture the global dependencies before the words matching. For further handling the object changing and global mismatch problems, an online update and correction mechanism are also employed in our method. Experiments show that our proposed method achieved competitive results on the DAVIS 2016 and DAVIS 2017 datasets. J&F-mean, the mean value between regional similarity and contour accuracy, reached 83.2% and 72.3% on DAVIS 2016 and DAVIS 2017, respectively. [ABSTRACT FROM AUTHOR]

We propose an effective approach to make segmentation for objects in videos with an initial input of the object masks in a few frames of the source video. In this method, we cast the segmentation task as a Markov Random Field (MRF) labeling problem. Different from the conventional MRF models, our model uses an additional term of higher-order potential to better propagate the global consistency among frames. The higher-order potential presented in this paper is significant for the proposed method because of its capability to keep the long-range consistency during segmentation. In order to make the MRF energy minimized, we also introduce a smart skill that makes the intractable higher-order potential "invisible" during the optimization so that the problem can be solved simply by applying a standard graph cut algorithm. Besides, the entire process is operated in a bilateral space, where the labeling can be inferred efficiently on the vertices that are sampled regularly from the bilateral grid. The results of a comparison of our method with a number of recently developed methods show that it performs favorably against state-of-the-art algorithms on multiple benchmark data sets in view of accuracy and achieves a much faster runtime performance. [ABSTRACT FROM AUTHOR]

Subtitles are crucial for video content understanding. However, a large amount of videos have only burned-in, hardcoded subtitles that prevent video re-editing, translation, etc. In this paper, we construct a deep-learning-based system for the inverse conversion of a burned-in subtitle video to a subtitle file and an inpainted video, by coupling three deep neural networks (CTPN, CRNN, and EdgeConnect). We evaluated the performance of the proposed method and found that the deep learning method achieved high-precision separation of the subtitles and video frames and significantly improved the video inpainting results compared to the existing methods. This research fills a gap in the application of deep learning to burned-in subtitle video reconstruction and is expected to be widely applied in the reconstruction and re-editing of videos with subtitles, advertisements, logos, and other occlusions. [ABSTRACT FROM AUTHOR]