Search

Arbitrary shape text detection is a challenging task in scene text recognition. Driven by deep learning and large‐scale data sets, the detection method based on connected component (CC) has increasingly gained popularity. However, there are still problems of unclear separation of text instances and incorrect component links. Thus, in this paper, the authors propose a novel component connection method, that is, Fragmented Affinity Reasoning Network of Text Instances (FARNet), for arbitrary shape text detection. The network consists of a Weighted Feature Fusion Pyramid Network (WFFPN), Text Fragments Subgraph (TFS), and Dense Graph Attention Network (DGAT), which can be trained end‐to‐end. The WFFPN is used to generate text fragments, TFS and DGAT jointly construct an affinity reasoning network. Since the neighbouring boundaries between text instances may blend them into a single instance, the core idea is to use the WFFPN to divide the text instance into a series of rectangular fragments, the affinity reasoning network infers the affinity between fragments and then links them to rebuild text instances. Extensive experiments on seven challenging datasets (ICDAR2015, MSRA‐TD500, Totaltext, CTW‐1500, ICDAR 2019MLT, ICDAR2019 ArT, and DAST‐1500) demonstrate that the proposed text detector achieves state‐of‐the‐art performance in both on polygon datasets and quadrilateral datasets. The code is available at https://github.com/giganticpower/FARNet. [ABSTRACT FROM AUTHOR]

Moving object segmentation (MOS) is the process of identifying dynamic objects from video frames, such as moving vehicles or pedestrians, while discarding the background. It plays an essential role in many real-world applications such as autonomous driving, mobile robots, and surveillance systems. With the availability of a huge amount of data and the development of powerful computing infrastructure, deep learning-based methods have shown remarkable improvements in MOS tasks. However, as the dimension of data becomes higher and the network architecture becomes more complicated, deep learning-based MOS models are computationally intensive, which limits their deployment on resource-constrained devices and in delay-sensitive applications. Therefore, more research started to develop fast and lightweight models. This paper aims to provide a comprehensive review of deep learning-based MOS models, with a focus on efficient model design techniques. We summarize a variety of MOS datasets, and conduct a thorough review of segmentation accuracy metrics and model efficiency metrics. Most importantly, we compare the performance of efficient MOS models on popular datasets, identify competitive models and analyze their essential techniques. Finally, we point out existing challenges and present future research directions. [ABSTRACT FROM AUTHOR]

The depth video contains large smooth contents with sharp edges. Since the deep learning-based color video orientated intra prediction methods pay no attention to the characteristics of depth video, they are unsuitable for optimizing the coding efficiency of depth video. In this paper, a multiple resolution prediction method with deep up-sampling is proposed to promote the coding efficiency of depth video. To efficiently encode the depth blocks of different complexity, the depth block is selectively encoded at different resolutions, including $\times 1$ , $\times 1$ /2, and $\times 1$ /4 resolutions. If the block is encoded with a low-resolution (LR), the resolution of reconstructed LR depth block is recovered by an up-sampling network. To constrain the quality of both reconstructed high-resolution depth block and its synthesized view, a view synthesis distortion guidance mechanism is proposed for the up-sampling network. In addition, a distillation-based lightweight up-sampling network is proposed to reduce the computational complexity. Experimental results demonstrate that the proposed multiple resolution prediction method obtains an average of 10.84% BD-rate saving in comparison with 3D-HEVC. [ABSTRACT FROM AUTHOR]

Multiview video coding (MVC) aims to compress the multiview video through the elimination of video redundancies, where the quality of the reference frame directly affects the compression efficiency. In this paper, we propose a deep virtual reference frame generation method based on a disparity-aware reference frame generation network (DAG-Net) to transform the disparity relationship between different viewpoints and generate a more reliable reference frame. The proposed DAG-Net consists of a multi-level receptive field module, a disparity-aware alignment module, and a fusion reconstruction module. First, a multi-level receptive field module is designed to enlarge the receptive field, and extract the multi-scale deep features of the temporal and inter-view reference frames. Then, a disparity-aware alignment module is proposed to learn the disparity relationship, and perform disparity shift on the inter-view reference frame to align it with the temporal reference frame. Finally, a fusion reconstruction module is utilized to fuse the complementary information and generate a more reliable virtual reference frame. Experiments demonstrate that the proposed reference frame generation method achieves superior performance for multiview video coding. [ABSTRACT FROM AUTHOR]

Conventional image classification methods usually require a large number of training samples for the training model. However, in practical scenarios, the amount of available sample data is often insufficient, which easily leads to overfitting in network construction. Few-shot learning provides an effective solution to this problem and has been a hot research topic. This paper provides an intensive survey on the state-of-the-art techniques in image classification based on few-shot learning. According to the different deep learning mechanisms, the existing algorithms are divided into four categories: transfer learning based, meta-learning based, data augmentation based, and multimodal based methods. Transfer learning based methods transfer useful prior knowledge from the source domain to the target domain. Meta-learning based methods employ past prior knowledge to guide the learning of new tasks. Data augmentation based methods expand the amount of sample data with auxiliary information. Multimodal based methods use the information of the auxiliary modal to facilitate the implementation of image classification tasks. This paper also summarizes the few-shot image datasets available in the literature, and experimental results tested by some representative algorithms are provided to compare their performance and analyze their pros and cons. In addition, the application of existing research outcomes on few-shot image classification in different practical fields are discussed. Finally, a few future research directions are identified. [ABSTRACT FROM AUTHOR]

Three-dimensional (3D) video contains plentiful multi-domain correlations, including spatial, temporal, and inter-view correlations. In this paper, a deep multi-domain prediction method is proposed for 3D video coding. Different from previous methods, our proposed method utilizes not only spatial and temporal correlations but also inter-view correlation to obtain a more accurate prediction, and adopts deep convolutional neural networks to effectively fuse multi-domain references. More specifically, a hierarchical prediction mechanism, which includes a spatial-temporal prediction network and a multi-domain prediction network, is designed to overcome the fusion difficulty of multi-domain reference information. Furthermore, a progressive spatial-temporal prediction network and a multi-scale multi-domain prediction network are designed to obtain the spatial-temporal prediction result and multi-domain prediction result respectively. Experimental results show that the proposed method achieves considerable bitrate saving compared with 3D-HEVC. [ABSTRACT FROM AUTHOR]

Hyperspectral image (HSI) clustering is a challenging task due to the complex characteristics in HSI data, such as spatial-spectral structure, high-dimension, and large spectral variability. In this paper, we propose a novel deep spatial-spectral subspace clustering network (DS3C-Net), which explores spatial-spectral information via the multi-scale auto-encoder and collaborative constraint. Considering the structure correlations of HSI, the multi-scale auto-encoder is first designed to extract spatial-spectral features with different-scale pixel blocks which are selected as the inputs. Then, the collaborative constrained self-expressive layers are introduced between the encoder and decoder, to capture the self-expressive subspace structures. By designing a self-expressiveness similarity constraint, the proposed network is trained collaboratively, and the affinity matrices of the feature representation are learned in an end-to-end manner. Based on the affinity matrices, the spectral clustering algorithm is utilized to obtain the final HSI clustering result. Experimental results on three widely used hyperspectral image datasets demonstrate that the proposed method outperforms state-of-the-art methods. [ABSTRACT FROM AUTHOR]

The arbitrary-shape scene text detection faces enormous challenges in accuracy and speed with increasing demand in the industrial sector. Although most studies have introduced segmentation and achieved remarkable performance in text detection tasks, researchers ignored the importance of detection efficiency. This paper proposes a new Joint Multi-Space Perception Network (JMNET) for efficient scene text detection to address this issue. Based on a lightweight feature extraction backbone, we put forward two novel modules, i.e., Scale Spatial Perception Module (SSPM) and Attention Spatial Perception Module (ASPM) to enhance the expression ability of text features with low computational complexity. Moreover, we propose an Unsupervised Embedding Spatial Perception Loss function (UnESP Loss) by introducing the Euclidean distance measurement between the embeddings to overcome the ambiguity of text instance boundary, such as a small line spacing. In this way, text embedding learning is not restricted by specific shapes, and detection robustness can be improved. Extensive experiments on four benchmarks, including the ICDAR2015, MSRA-TD500, CTW1500 and Totaltext, show that the proposed JMNET achieves competitive performance in terms of both accuracy and speed over the state-of-the-art methods. Particularly, our method can reach a remarkable F-measure of 85.5% at 52 FPS on Totaltext. Code is available at: https://github.com/sakura-910/JMNET. [ABSTRACT FROM AUTHOR]

• The current 3D human body reconstruction technology requires a large number of 3D scanning equipment and a complex dynamic capture process, which significantly limits its rapid adoption in these application domains. The purpose of this article is to simplify the process of 3D reconstruction. • By combining deep learning methods, an end-to-end 3D human body reconstruction framework is proposed to achieve the goal of reconstructing a high-precision 3D human body model based on RGB images. [Display omitted] In recent years, with the improvement of artificial intelligence technology, it has become possible to reconstruct high-precision 3D human body models based on ordinary RGB images. The current 3D human body reconstruction technology requires complex external equipment to scan all angles of the human body, which is complicated to be implemented and cannot be popularized. In order to solve this problem, this paper applies deep learning models on reconstructing 3D human body based on monocular images. First of all, this paper uses Stacked Hourglass network to perform convolution operations on monocular images collected from different views. Then Multi-Layer Perceptrons (MLPs) are used to decode the encoded high-level images. The feature codes in the two views(main and side) are fused, and the interior and exterior points are classified by the fusion features, so as to obtain the corresponding 3D occupancy field. At last, the Marching Cube algorithm is used for 3D reconstruction with a specific threshold and then we use Laplace smoothing algorithm to remove artifacts. This paper proposes a dense sampling strategy based on the important joint points of the human body, which has a certain optimization effect on the realization of high-precision 3D reconstruction. The performance of the proposed scheme has been validated on the open source datasets, MGN dataset and the THuman dataset, provided by Tsinghua University. The proposed scheme can reconstruct features such as clothing folds, color textures, and facial details,and has great potential to be applied in different applications. [ABSTRACT FROM AUTHOR]