Search

In the era of deep learning, it's important to obtain enough qualified data for model training. For the labor-intensive and time-consuming characteristics of data collection and annotation, date augmentation has been widely employed in deep learning methods to mitigate over-fitting. It has been proven to be one of the most effective ways to ensure the performance of deep models. In the field of computer vision, the principal goal of data augmentation usually is enlarging both the scale and diversity of the dataset, and maintaining the quality of both the image and the annotation data simultaneously. Data augmentation methods at the raw input data level are intuitive to obtain semantic meaningful images. In this paper, we provide a brief survey on some data augmentation studies conducted at image level and briefly categorize the studies into two main classes: those based on image editing and those based on image generation. We also make an analysis of the semantic-preserving considerations within data augmentation techniques. [ABSTRACT FROM AUTHOR]

Video captioning is a joint task of computer vision and natural language processing, which aims to describe the video content using several natural language sentences. Nowadays, most methods cast this task as a mapping problem, which learns a mapping from visual features to natural language and generates captions directly from videos. However, the underlying challenge of video captioning, i.e., sequence to sequence mapping across the different domains, is still not well handled. To address these problems, we introduce the polishing mechanism in an attempt to mimic human polishing process and propose a generate-and-polish framework for video captioning. In this paper, we propose a two-step transformer based polishing network (TSTPN) consisting of two sub-modules: the generation-module is to generate the caption candidate and the polishing-module is to gradually refine the generated candidate. Specifically, the candidate provides a global information of the visual contents in a semantically-meaningful order, where it is firstly considered as a semantic intersnubber to bridge the semantic gap between the text and video, with the cross-modal attention mechanism for better cross-modal modeling; and it secondly provides a global planning ability to maintain the semantic consistency and fluency of the whole sentence for better sequence mapping. In experiments, we present adequate evaluations to show that the proposed TSTPN achieves the comparable and even better performance than the state-of-the-art methods on the benchmark datasets. [ABSTRACT FROM AUTHOR]

In this paper, we propose an algorithm of anomaly detection in crowded scenes by using sparse representation over the normal bases. First, the histogram of maximal optical flow projection (HMOFP) features are extracted from a set of normal training data. Then, the online dictionary learning algorithm is used to train an optimal dictionary with proper redundancy, which is better than the dictionary simply composed by the HMOFP features of the whole training data. In order to detect the normalness of a frame, the l -norm of the sparse reconstruction coefficients is used as the Reconstruction Coefficient Sparsity (RCS). Our algorithm is effective for both global abnormal events (GAE) and local abnormal events (LAE). We evaluate our method on three benchmark datasets-the UMN dataset, the PETS2009 dataset and the UCSD Ped1 dataset. Compared with the most popular methods, experimental results show that our algorithm achieves good results especially for the pixel-level local abnormal event localization. [ABSTRACT FROM AUTHOR]

As a major research field in computer vision, automatic generation of Labanotation has been attracting the attention of many researchers. These researchers use various methods to engage in dance education and protection, but the existing methods do not consider the spatial-temporal dependence modeling of dance movements. Therefore, they do not represent complex dance movements optimally. We propose an automatic generation network of Labanotation based on the long and short spatial-temporal relations. It includes local spatial-temporal feature extraction network, global spatial-temporal feature extraction network, and local and global feature fusion network. It can align the input and output sequences while modeling the skeletal spatial-temporal relation. The local spatial-temporal features among short interval skeleton sequence relationship are obtained through the multi-scale convolution of time and space. The global spatial-temporal features are learned through the transformer network to obtain the relationship among skeleton sequences with long interval. For the output of the two networks, we use the pyramid squeeze attention network to exchange information in long and short spatial-temporal information, achieving complementarity to improve the accuracy of action recognition. Experimental results show that the proposed method outperforms the state-of-the-art methods on laban16 and laban48, which are common datasets for the research of Labanotation. [ABSTRACT FROM AUTHOR]

In this paper, we present a new object recognition model with few instances based on semantic distance. Learning objects with many instances have been studied in computer vision for many years. However, in many cases, not enough positive instances occur, especially for some special categories. We must take full advantage of all instances, including those that do not belong to the category. The main insight is that, given a few positive instances from one category, we can define some other candidate instances as positive instances based on semantic distance to learn this model. Our model responds more strongly to instances with closer semantic distance to positive instances than to instances with farther semantic distance to positive instances. We use a regularized kernel machine algorithm to train the images from the database. The superiority of our method to existing object recognition methods is demonstrated. Experiments using an image database show that our method not only reduces the number of learning instances but also keeps the accurate rate of recognition. [ABSTRACT FROM AUTHOR]

In this paper, we present a new model applied to watershed segmentation optimization. Watershed segmentation is widely used in the domain of computer vision. However, in some cases, there are some unobvious gradient changing between different regions or obvious gradient changing in the same region. So the images are divided into a number of regions either smaller (under-segmentation) or larger (over-segmentation) than the expected numbers of regions. We use a new filter (OLS operator) to pre-process the image. The new filter can smooth the image but keep the edge between different regions sharper based on image decompositions. Compared to traditional watershed segmentation, our method can segment images more accurately. Experiments using an image database show the superiority of our method. Even if we apply our method to complex images with more regions, it is still effective. Furthermore, our model can be applied to other domains of computer vision (e.g., image classification, image recognition, image matting). [ABSTRACT FROM AUTHOR]

Image compositing techniques are primarily utilized to achieve realistic composite results. Some existing image compositing methods, such as gradient domain and alpha matting, are widely used in the field of computer vision, and can typically achieve realistic results, especially for seamless boundaries. However, when the candidate composite images and the target images have obvious differences, such as color, texture and brightness, the composite results are unrealistic and inconsistent. At the same time, traditional compositing methods focus on basic feature matching, ignoring semantic rationality in composition processing. Quite a few compositing methods thus generate composite results without semantic rationality. In this paper, a new multi-scale image composition method has been presented. In the composition process, wavelet pyramid and basic feature handling were used to achieve multi-scale compositions. More importantly, a new criterion was established, based on the semantic rationality of images, which could ensure that the composite images are semantically valid. A large database was created to facilitate experimentation. The experiments showed that the methodology introduced in this paper produced superior results compared to traditional composition methods; the composite results were not only consistent and seamless, but were also semantically valid. [ABSTRACT FROM AUTHOR]