Search

In this paper, an STN (Spatial Transformer Network)-enhanced message passing module guided by adversarial learning is proposed for human pose estimation. Transformations performed on the predicted heatmaps mean to modify the raw predictions and conduct the message passing among human joints in an elegant way. Firstly, we employ STN submodule to transform the predicted heatmap of one human joint to heatmap of its neighboring joint to remove ambiguity of its neighboring predictions. STN submodule automatically learns the geometric information between adjacent joints and thus builds related neighboring associations among them. Nevertheless, it seems difficult for STN submodule to learn inherent geometric information from a single RGB image alone. Secondly, limb guidance is introduced to assist STN in predicting corresponding correlations. Since quality of limb predictions poses great significance for the guidance, we propose to exploit adversarial learning to improve the quality of limb heatmaps which are easier to learn than precise keypoint location. Hence, the precision of STN transformation improves owing to more precise prior instructions. However, STN submodule might be confused when performing the transformation due to massive noises of the heatmaps. To circumvent this dilemma, at last, we propose to utilize Weighted Mean Square Error (WMSE) loss and convolutional random walk (CRW) which improve the performance further. Our method achieves competitive results on both MPII and LSP benchmarks. [ABSTRACT FROM AUTHOR]

In this paper, we propose to perform mutual learning inside the compositional model for human pose estimation. We conduct inference of the human pose in a compositional model which is composed of parts and subparts networks to better capture intrinsic prior knowledge. A macro–micro mutual learning mechanism is put forward to promote the information interaction between human limbs (parts) and joints (subparts). At first, a macro mutual learning module is proposed to conduct the information interaction macroscopically. Features representing the whole human body are leveraged in this case. Secondly, a micro mutual learning module is proposed to promote the information interaction within each limb triplet group to refine corresponding features microcosmically. Combining the macro and micro mutual learning modules promotes the information interaction across different levels. Besides, a novel Balanced Masked Mean Square Error (BMMSE) loss is put forward to solve the positive–negative samples imbalance problem which is more apparent in compositional model. Effectiveness of the method is evaluated on MPII, LSP and COCO benchmarks. Our algorithm achieves leading positions on the leader board of these three datasets. [ABSTRACT FROM AUTHOR]

Recently, many methods have united low-level and high-level features to generate the desired accurate high-resolution prediction for human parsing. Nevertheless, there exists a semantic-spatial gap between low-level and high-level features in some methods, i.e., high-level features represent more semantics and less spatial details, while low-level ones have less semantics and more spatial details. In this paper, we propose a Semantic-Spatial Fusion Network (SSFNet) for human parsing to shrink the gap, which generates the accurate high-resolution prediction by aggregating multi-resolution features. SSFNet includes two models, a semantic modulation model and a resolution-aware model. The semantic modulation model guides spatial details with semantics and then effectively facilitates the feature fusion, narrowing the gap. The resolution-aware model sufficiently boosts the feature fusion and obtains multi-receptive-fields, which generates reliable and fine-grained high-resolution features for each branch, in bottom-up and top-down processes. Extensive experiments on three public datasets, PASCAL-Person-Part, LIP and PPSS, show that SSFNet achieves significant improvements over state-of-the-art methods. [ABSTRACT FROM AUTHOR]

In recent years, SiamFC-based trackers have received much attention because of their great potentials in balancing tracking accuracy and speed. However, the robustness of most such trackers is greatly affected by the large deformations of targets. We argue that in the cross-correlation operation which is widely used by modern SiamFC-based trackers, the static correlation between the template kernel and the feature maps of test sample is difficult to adapt to the large deformation of the target object. In this paper, we propose a Siamese deformable cross-correlation network (SiamDCN), which introduces the deformable cross-correlation operation into SiamFC in an online self-adaptive way, for robust visual tracking. Compared to the previous SiamFC-based trackers, our SiamDCN is more robust to the large deformations of targets through dynamically and adaptively adjusting the location of correlation calculation for each element of the template kernel in the cross-correlation operation. Moreover, we build a twofold Siamese network, named SiamDCN+, which consists of a SiamDCN branch and a SiamFC branch, for accurate and real-time visual tracking after observing that the features learned in SiamFC are static and discriminative, whereas those in SiamDCN are dynamic and robust, and they complement each other. Extensive experiments on three public benchmarks, OTB2015, VOT2016, and VOT2017, show that the proposed SiamDCN achieves superior localization accuracy than its baseline tracker SiamFC and the proposed SiamDCN+ achieves competitive performance compared to state-of-the-art real-time trackers, while running beyond 40 FPS. [ABSTRACT FROM AUTHOR]

Abstract Reading text in the wild is a challenging task in computer vision. Existing approaches mainly adopt connectionist temporal classification (CTC) or attention models based on recurrent neural network (RNN), and are computationally expensive and hard to train. In this paper, instead of the chain structure of RNN, we propose an end-to-end fully convolutional network with the stacked convolutional layers to effectively capture the long-term dependencies among elements of scene text image. The stacked convolutional layers are much more efficient than bidirectional long short-term memory (BLSTM) in modeling the contextual dependency. In addition, we design a discriminative feature encoder by incorporating the residual attention blocks into a small densely connected network to enhance the foreground text and suppress the background noise. Extensive experiments on seven standard benchmarks, the Street View Text, IIIT5K, ICDAR03, ICDAR13, ICDAR15, COCO-Text and Total-Text, validate that our method not only achieves state-of-the-art or highly competitive recognition performance, but significantly improves the efficiency and reduces the number of parameters as well. [ABSTRACT FROM AUTHOR]