Search

For image forensics, convolutional neural networks (CNNs) tend to learn image content features rather than subtle manipulation traces, which constrains detection performance. Existing works usually address this issue by following a common pipeline, namely subtracting the original pixel value from the predicted pixel value to enforce CNNs to learn more features from the manipulation traces. However, due to the complicated learning mechanism, they might still have some unnecessary performance losses. In this work, we rethink the advantages of image gradient operator in exposing AI-enabled face forgeries, and design two plug-and-play modules, namely tensor pre-processing (TP) and manipulation trace attention (MTA), by combining the gradient operator with CNNs. Specifically, the TP module refines the feature tensor of each channel in the network by the gradient operator to highlight manipulation traces and improve feature representation. Moreover, the MTA module considers two dimensions, namely channel and manipulation traces, to enforce the network to learn the distribution of the manipulation traces. Both modules can be seamlessly integrated into existing CNNs for end-to-end training. Experiments show that the proposed expert system achieves better results than prior works on five public image datasets. The code is available at: https://github.com/EricGzq/GocNet-pytorch. • We rethink the advantages of gradient operator in exposing face forgeries. • A tensor pre-processing (TP) module is designed for face image forensics. • A manipulation trace attention (MTA) module is designed for feature refinement. • Both TP and MTA are plug-and-play modules, which can be integrated into any CNNs. [ABSTRACT FROM AUTHOR]

For fake face image detection, most existing detectors exploit local artifacts, ignoring the mining of structured forgery clues existed in global images, which greatly constrains detection performance. In this work, we verify the importance of structured forgery clues for fake face image detection, and present a new data augmentation framework called Mining Structured Features (MSF) to promote the convolutional neural network (CNN) based detector. Specifically, MSF generates a position-sensitive artifact map, which is exploited to divide a candidate face image into strong correlation (SC) and weak correlation (WC) regions. By dynamically erasing some positions in SC and WC regions during the training process, MSF can promote the robustness of the detector to the above regions. In essence, MSF expands the attention range of the detector and fully mines the structured features from global images. MSF plays an auxiliary role, which can be seamlessly integrated into existing CNN-based detectors for end-to-end training. Extensive experimental results on four public datasets including HFFD, FF++, DFDC and Celeb-DF show that the detectors trained with the guidance of MSF can mine more useful clues distributed in fake face images to improve detection accuracies, achieving better results than state-of-the-art works. Our code will be available at https://github.com/EricGzq/MSF. [Display omitted] • The first attempt is made to verify the importance of structured forgery clues. • Design a novel data augmentation framework to capture structured forgery clues. • The proposed approach can be integrated with CNN detectors in an end-to-end manner. • Experiments show that our method achieves better results than SOTA works. [ABSTRACT FROM AUTHOR]