Search

Recent advance on highly realistic AI‐synthesised video approaches makes it hard to distinguish whether a speaker in video is real. Many existing DeepFakes detection approaches can be invalidated by using them as supervision in adversarial training, which in turn improves the performance of DeepFakes. This makes the problem a dilemma. However, human can recognise the identity of familiar persons by observing their motion styles. Inspired by this, the paper proposes a novel method to recognise original speaker's identity clues in DeepFakes videos by learning individual motion styles. As a biological signature, motion styles can neither be used in the training process of DeepFakes nor be modified to deceive detection methods without reducing realistic performance. Also, this paper proposes a novel pipeline to continuously restore the original frames from DeepFakes videos without knowing the DeepFakes approach in advance. Based on above ideas, our scheme makes it possible to restore DeepFakes videos. Through training the cross‐modal transfer module, the appearance embedding can be inferred from the identity code. Extensive experiments are conducted to reveal the effectiveness of the method, it shows for the first time that the original persons can be identified automatically and the original video from DeepFakes videos can be generated. [ABSTRACT FROM AUTHOR]

As one of the most fundamental tasks in computer vision, semantic segmentation assigns per pixel prediction of object categories. Training a robust model for semantic segmentation is challenging since pixel-level annotations are expensive to obtain. To alleviate the burden of annotations, the authors propose a weakly-supervised framework for zero-shot semantic segmentation, which can segment images having target classes without any pixel-level labelled instances. Under the assumption that the accessibility to image-level annotations of target classes does not violate the principle of zero pixel-level label in zero-shot semantic segmentation, we utilised image-level annotations to improve the proposed model's ability to extract pixel-level features. Furthermore, unlike existing zero-shot semantic segmentation methods, which use semantic embeddings as class embeddings to transfer knowledge from source classes to target classes, we use image-level features as their class embeddings to transfer knowledge since the distribution of pixel-level features is more similar to the distribution of image-level features rather than the distribution of semantic embeddings. Experimental results on the PASCAL-VOC data set under different data splits demonstrate that the proposed model achieves promising results. [ABSTRACT FROM AUTHOR]

While promising progress has been achieved in the zero-shot learning (ZSL) task. The existing approaches still suffer from the strong bias problem between the unseen and seen classes. This Letter presents a unified feature generating framework equipped with a boundary decision loss to tackle this issue in ZSL. Specifically, the hybrid semantic and visual classification strategy is proposed, which can effectively align the bidirectional visual-semantic interactions. Furthermore, this Letter introduces a decision loss that optimises the decision boundary of seen and unseen classes to further alleviate the confusion of generated features. Extensive experiments on three popular datasets animals with attributes, Caltech-UCSD-Birds 200-2011, and SUN show that the proposed approach outperforms previous state-of-the-art works under both traditional ZSL and challenging generalised ZSL settings. [ABSTRACT FROM AUTHOR]

Recent advances in convolutional neural networks have achieved prominent results for object counting or locating in images. In these networks, some adopt detection‐based methods which require providing bounding‐box annotations for training and these methods usually cannot deal with the crowded cases well. Other networks adopt density regression‐based methods which require only point annotations for training but these methods can only predict the total count without providing any location information. In this Letter, the authors propose an attention guided convolutional neural network (AGNet) for both crowded object locating and counting using point annotation as supervision without annotated bounding boxes which are far more time‐consuming to label. Accordingly, they define a multi‐task objective loss function to lead the training of AGNet to achieve the goal of crowded object locating. The experimental results on four challenging benchmark datasets for crowded objects counting demonstrate the superiority of the proposed AGNet over recent state‐of‐the‐art approaches. [ABSTRACT FROM AUTHOR]

This Letter presents a new improved zero‐shot learning (ZSL) method based on meta‐learning that automatically adjusts metrics by multi‐layer perceptrons instead of using traditional Euclidean distance, cosine distance and other fixed metrics. In this way, the authors directly train the visual and semantic joint information of the images without adding additional auxiliary, which simply and effectively avoids the error derivation of unseen classes due to improper fixed metrics. Smooth one‐hot encoding rule is used to make the calculation of loss more reasonable. They also discuss the embedding methods to find a suitable low‐noise embedding space and analyse the model performance under the conventional setting and popular generalised ZSL setting. Experimental results show that the proposed automatic metrics method outperforms existing state‐of‐the‐art approaches in some parts of AwA2 and CUB datasets. [ABSTRACT FROM AUTHOR]

A new feature for image retrieval based on edge orientation patterns (EOPs) by classifying image blocks into nine classes according to their EOs. Each input colour image is decomposed into Y, Cb and Cr components, and each component image is divided into non‐overlapping 4 × 4 blocks. For each block, eight EO templates of size 4 × 4 are performed to determine its EO. On the other hand, vector quantisation (VQ) is an efficient clustering and classification technique for high‐dimensional spaces, and thus VQ is also used to extract the texture features. The retrieval simulation results show that the EOP‐histogram‐based features combined with the traditional VQ‐index‐histogram‐based features can largely improve the recall and precision performance. [ABSTRACT FROM AUTHOR]