Back to Search
Start Over
Interactive Two-Stream Network Across Modalities for Deepfake Detection
- Source :
- IEEE Transactions on Circuits and Systems for Video Technology; November 2023, Vol. 33 Issue: 11 p6418-6430, 13p
- Publication Year :
- 2023
-
Abstract
- As face forgery techniques have become more mature, the proliferation of deepfakes may threaten the security of human society. Although existing deepfake detection methods achieve good performance for in-dataset evaluation, it remains to be improved in the generalization ability, where the representation of the imperceptible artifacts plays a significant role. In this paper, we propose an Interactive Two-Stream Network (ITSNet) to explore the discriminant inconsistency representation from the perspective of cross-modality. In particular, the patch-wise Decomposable Discrete Cosine Transform (DDCT) is adopted to extract fine-grained high-frequency clues, and information from different modalities communicates with each other via a designed interaction module. To perceive the temporal inconsistency, we first develop a Short-term Embedding Module (SEM) to refine subtle local inconsistency representation between adjacent frames, and then a Long-term Embedding Module (LEM) is designed to further refine the erratic temporal inconsistency representation from the long-range perspective. Extensive experimental results conducted on three public datasets show that ITSNet outperforms the state-of-the-art methods both in terms of in-dataset and cross-dataset evaluations.
Details
- Language :
- English
- ISSN :
- 10518215 and 15582205
- Volume :
- 33
- Issue :
- 11
- Database :
- Supplemental Index
- Journal :
- IEEE Transactions on Circuits and Systems for Video Technology
- Publication Type :
- Periodical
- Accession number :
- ejs64405173
- Full Text :
- https://doi.org/10.1109/TCSVT.2023.3269841