Local Region Frequency Guided Dynamic Inconsistency Network for Deepfake Video Detection
In recent years, with the rapid development of deepfake technology, a large number of deepfake videos have emerged on the Internet, which poses a huge threat to national politics, social stability, and personal privacy. Although many existing deepfake detection methods exhibit excellent performance...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Tsinghua University Press
2024-09-01
|
| Series: | Big Data Mining and Analytics |
| Subjects: | |
| Online Access: | https://www.sciopen.com/article/10.26599/BDMA.2024.9020030 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832544386321219584 |
|---|---|
| author | Pengfei Yue Beijing Chen Zhangjie Fu |
| author_facet | Pengfei Yue Beijing Chen Zhangjie Fu |
| author_sort | Pengfei Yue |
| collection | DOAJ |
| description | In recent years, with the rapid development of deepfake technology, a large number of deepfake videos have emerged on the Internet, which poses a huge threat to national politics, social stability, and personal privacy. Although many existing deepfake detection methods exhibit excellent performance for known manipulations, their detection capabilities are not strong when faced with unknown manipulations. Therefore, in order to obtain better generalization ability, this paper analyzes global and local inter-frame dynamic inconsistencies from the perspective of spatial and frequency domains, and proposes a Local region Frequency Guided Dynamic Inconsistency Network (LFGDIN). The network includes two parts: Global SpatioTemporal Network (GSTN) and Local Region Frequency Guided Module (LRFGM). The GSTN is responsible for capturing the dynamic information of the entire face, while the LRFGM focuses on extracting the frequency dynamic information of the eyes and mouth. The LRFGM guides the GTSN to concentrate on dynamic inconsistency in some significant local regions through local region alignment, so as to improve the model’s detection performance. Experiments on the three public datasets (FF++, DFDC, and Celeb-DF) show that compared with many recent advanced methods, the proposed method achieves better detection results when detecting deepfake videos of unknown manipulation types. |
| format | Article |
| id | doaj-art-14c0362758f4487d8f84aa2cf17619f1 |
| institution | Kabale University |
| issn | 2096-0654 |
| language | English |
| publishDate | 2024-09-01 |
| publisher | Tsinghua University Press |
| record_format | Article |
| series | Big Data Mining and Analytics |
| spelling | doaj-art-14c0362758f4487d8f84aa2cf17619f12025-02-03T10:19:58ZengTsinghua University PressBig Data Mining and Analytics2096-06542024-09-017388990410.26599/BDMA.2024.9020030Local Region Frequency Guided Dynamic Inconsistency Network for Deepfake Video DetectionPengfei Yue0Beijing Chen1Zhangjie Fu2Engineering Research Center of Digital Forensics affiliated with Ministry of Education, and also with School of Computer Science, and also with Jiangsu Collaborative Innovation Center of Atmospheric Environment and Equipment Technology (CICAEET), Nanjing University of Information Science and Technology, Nanjing 210044, ChinaEngineering Research Center of Digital Forensics affiliated with Ministry of Education, and also with School of Computer Science, and also with Jiangsu Collaborative Innovation Center of Atmospheric Environment and Equipment Technology (CICAEET), Nanjing University of Information Science and Technology, Nanjing 210044, ChinaEngineering Research Center of Digital Forensics affiliated with Ministry of Education, and also with School of Computer Science, and also with Jiangsu Collaborative Innovation Center of Atmospheric Environment and Equipment Technology (CICAEET), Nanjing University of Information Science and Technology, Nanjing 210044, ChinaIn recent years, with the rapid development of deepfake technology, a large number of deepfake videos have emerged on the Internet, which poses a huge threat to national politics, social stability, and personal privacy. Although many existing deepfake detection methods exhibit excellent performance for known manipulations, their detection capabilities are not strong when faced with unknown manipulations. Therefore, in order to obtain better generalization ability, this paper analyzes global and local inter-frame dynamic inconsistencies from the perspective of spatial and frequency domains, and proposes a Local region Frequency Guided Dynamic Inconsistency Network (LFGDIN). The network includes two parts: Global SpatioTemporal Network (GSTN) and Local Region Frequency Guided Module (LRFGM). The GSTN is responsible for capturing the dynamic information of the entire face, while the LRFGM focuses on extracting the frequency dynamic information of the eyes and mouth. The LRFGM guides the GTSN to concentrate on dynamic inconsistency in some significant local regions through local region alignment, so as to improve the model’s detection performance. Experiments on the three public datasets (FF++, DFDC, and Celeb-DF) show that compared with many recent advanced methods, the proposed method achieves better detection results when detecting deepfake videos of unknown manipulation types.https://www.sciopen.com/article/10.26599/BDMA.2024.9020030deepfake video detectiondynamic inconsistencylocal regionlocal region frequency |
| spellingShingle | Pengfei Yue Beijing Chen Zhangjie Fu Local Region Frequency Guided Dynamic Inconsistency Network for Deepfake Video Detection Big Data Mining and Analytics deepfake video detection dynamic inconsistency local region local region frequency |
| title | Local Region Frequency Guided Dynamic Inconsistency Network for Deepfake Video Detection |
| title_full | Local Region Frequency Guided Dynamic Inconsistency Network for Deepfake Video Detection |
| title_fullStr | Local Region Frequency Guided Dynamic Inconsistency Network for Deepfake Video Detection |
| title_full_unstemmed | Local Region Frequency Guided Dynamic Inconsistency Network for Deepfake Video Detection |
| title_short | Local Region Frequency Guided Dynamic Inconsistency Network for Deepfake Video Detection |
| title_sort | local region frequency guided dynamic inconsistency network for deepfake video detection |
| topic | deepfake video detection dynamic inconsistency local region local region frequency |
| url | https://www.sciopen.com/article/10.26599/BDMA.2024.9020030 |
| work_keys_str_mv | AT pengfeiyue localregionfrequencyguideddynamicinconsistencynetworkfordeepfakevideodetection AT beijingchen localregionfrequencyguideddynamicinconsistencynetworkfordeepfakevideodetection AT zhangjiefu localregionfrequencyguideddynamicinconsistencynetworkfordeepfakevideodetection |