Deepfake Face Detection and Adversarial Attack Defense Method Based on Multi-Feature Decision Fusion
The rapid advancement in deep forgery technology in recent years has created highly deceptive face video content, posing significant security risks. Detecting these fakes is increasingly urgent and challenging. To improve the accuracy of deepfake face detection models and strengthen their resistance...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-06-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/15/12/6588 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849432842356916224 |
|---|---|
| author | Shanzhong Lei Junfang Song Feiyang Feng Zhuyang Yan Aixin Wang |
| author_facet | Shanzhong Lei Junfang Song Feiyang Feng Zhuyang Yan Aixin Wang |
| author_sort | Shanzhong Lei |
| collection | DOAJ |
| description | The rapid advancement in deep forgery technology in recent years has created highly deceptive face video content, posing significant security risks. Detecting these fakes is increasingly urgent and challenging. To improve the accuracy of deepfake face detection models and strengthen their resistance to adversarial attacks, this manuscript introduces a method for detecting forged faces and defending against adversarial attacks based on a multi-feature decision fusion. This approach allows for rapid detection of fake faces while effectively countering adversarial attacks. Firstly, an improved IMTCCN network was employed to precisely extract facial features, complemented by a diffusion model for noise reduction and artifact removal. Subsequently, the FG-TEFusionNet (Facial-geometry and Texture enhancement fusion-Net) model was developed for deepfake face detection and assessment. This model comprises two key modules: one for extracting temporal features between video frames and another for spatial features within frames. Initially, a facial geometry landmark calibration module based on the LRNet baseline framework ensured an accurate representation of facial geometry. A SENet attention mechanism was then integrated into the dual-stream RNN to enhance the model’s capability to extract inter-frame information and derive preliminary assessment results based on inter-frame relationships. Additionally, a Gram image texture feature module was designed and integrated into EfficientNet and the attention maps of WSDAN (Weakly Supervised Data Augmentation Network). This module aims to extract deep-level feature information from the texture structure of image frames, addressing the limitations of purely geometric features. The final decisions from both modules were integrated using a voting method, completing the deepfake face detection process. Ultimately, the model’s robustness was validated by generating adversarial samples using the I-FGSM algorithm and optimizing model performance through adversarial training. Extensive experiments demonstrated the superior performance and effectiveness of the proposed method across four subsets of FaceForensics++ and the Celeb-DF dataset. |
| format | Article |
| id | doaj-art-08307c6e404d43ff90bd77e6f50ed496 |
| institution | Kabale University |
| issn | 2076-3417 |
| language | English |
| publishDate | 2025-06-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj-art-08307c6e404d43ff90bd77e6f50ed4962025-08-20T03:27:15ZengMDPI AGApplied Sciences2076-34172025-06-011512658810.3390/app15126588Deepfake Face Detection and Adversarial Attack Defense Method Based on Multi-Feature Decision FusionShanzhong Lei0Junfang Song1Feiyang Feng2Zhuyang Yan3Aixin Wang4College of Information Engineering, Xizang Minzu University, No. 6, East Section of Wenhui Road, Weicheng District, Xianyang 712082, ChinaCollege of Information Engineering, Xizang Minzu University, No. 6, East Section of Wenhui Road, Weicheng District, Xianyang 712082, ChinaCollege of Information Engineering, Xizang Minzu University, No. 6, East Section of Wenhui Road, Weicheng District, Xianyang 712082, ChinaCollege of Information Engineering, Xizang Minzu University, No. 6, East Section of Wenhui Road, Weicheng District, Xianyang 712082, ChinaCollege of Information Engineering, Xizang Minzu University, No. 6, East Section of Wenhui Road, Weicheng District, Xianyang 712082, ChinaThe rapid advancement in deep forgery technology in recent years has created highly deceptive face video content, posing significant security risks. Detecting these fakes is increasingly urgent and challenging. To improve the accuracy of deepfake face detection models and strengthen their resistance to adversarial attacks, this manuscript introduces a method for detecting forged faces and defending against adversarial attacks based on a multi-feature decision fusion. This approach allows for rapid detection of fake faces while effectively countering adversarial attacks. Firstly, an improved IMTCCN network was employed to precisely extract facial features, complemented by a diffusion model for noise reduction and artifact removal. Subsequently, the FG-TEFusionNet (Facial-geometry and Texture enhancement fusion-Net) model was developed for deepfake face detection and assessment. This model comprises two key modules: one for extracting temporal features between video frames and another for spatial features within frames. Initially, a facial geometry landmark calibration module based on the LRNet baseline framework ensured an accurate representation of facial geometry. A SENet attention mechanism was then integrated into the dual-stream RNN to enhance the model’s capability to extract inter-frame information and derive preliminary assessment results based on inter-frame relationships. Additionally, a Gram image texture feature module was designed and integrated into EfficientNet and the attention maps of WSDAN (Weakly Supervised Data Augmentation Network). This module aims to extract deep-level feature information from the texture structure of image frames, addressing the limitations of purely geometric features. The final decisions from both modules were integrated using a voting method, completing the deepfake face detection process. Ultimately, the model’s robustness was validated by generating adversarial samples using the I-FGSM algorithm and optimizing model performance through adversarial training. Extensive experiments demonstrated the superior performance and effectiveness of the proposed method across four subsets of FaceForensics++ and the Celeb-DF dataset.https://www.mdpi.com/2076-3417/15/12/6588diffusion modelfacial geometrytexture feature enhancementdeepfake face detectionmulti-feature decision fusionadversarial attack and defense |
| spellingShingle | Shanzhong Lei Junfang Song Feiyang Feng Zhuyang Yan Aixin Wang Deepfake Face Detection and Adversarial Attack Defense Method Based on Multi-Feature Decision Fusion Applied Sciences diffusion model facial geometry texture feature enhancement deepfake face detection multi-feature decision fusion adversarial attack and defense |
| title | Deepfake Face Detection and Adversarial Attack Defense Method Based on Multi-Feature Decision Fusion |
| title_full | Deepfake Face Detection and Adversarial Attack Defense Method Based on Multi-Feature Decision Fusion |
| title_fullStr | Deepfake Face Detection and Adversarial Attack Defense Method Based on Multi-Feature Decision Fusion |
| title_full_unstemmed | Deepfake Face Detection and Adversarial Attack Defense Method Based on Multi-Feature Decision Fusion |
| title_short | Deepfake Face Detection and Adversarial Attack Defense Method Based on Multi-Feature Decision Fusion |
| title_sort | deepfake face detection and adversarial attack defense method based on multi feature decision fusion |
| topic | diffusion model facial geometry texture feature enhancement deepfake face detection multi-feature decision fusion adversarial attack and defense |
| url | https://www.mdpi.com/2076-3417/15/12/6588 |
| work_keys_str_mv | AT shanzhonglei deepfakefacedetectionandadversarialattackdefensemethodbasedonmultifeaturedecisionfusion AT junfangsong deepfakefacedetectionandadversarialattackdefensemethodbasedonmultifeaturedecisionfusion AT feiyangfeng deepfakefacedetectionandadversarialattackdefensemethodbasedonmultifeaturedecisionfusion AT zhuyangyan deepfakefacedetectionandadversarialattackdefensemethodbasedonmultifeaturedecisionfusion AT aixinwang deepfakefacedetectionandadversarialattackdefensemethodbasedonmultifeaturedecisionfusion |