Comprehensive Evaluation of Deepfake Detection Models: Accuracy, Generalization, and Resilience to Adversarial Attacks
The rise of deepfakes—synthetic media generated using artificial intelligence—threatens digital content authenticity, facilitating misinformation and manipulation. However, deepfakes can also depict real or entirely fictitious individuals, leveraging state-of-the-art techniques such as generative ad...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-01-01
|
| Series: | Applied Sciences |
| Subjects: | |
| Online Access: | https://www.mdpi.com/2076-3417/15/3/1225 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850199428709744640 |
|---|---|
| author | Maryam Abbasi Paulo Váz José Silva Pedro Martins |
| author_facet | Maryam Abbasi Paulo Váz José Silva Pedro Martins |
| author_sort | Maryam Abbasi |
| collection | DOAJ |
| description | The rise of deepfakes—synthetic media generated using artificial intelligence—threatens digital content authenticity, facilitating misinformation and manipulation. However, deepfakes can also depict real or entirely fictitious individuals, leveraging state-of-the-art techniques such as generative adversarial networks (GANs) and emerging diffusion-based models. Existing detection methods face challenges with generalization across datasets and vulnerability to adversarial attacks. This study focuses on subsets of frames extracted from the DeepFake Detection Challenge (DFDC) and FaceForensics++ videos to evaluate three convolutional neural network architectures—XCeption, ResNet, and VGG16—for deepfake detection. Performance metrics include accuracy, precision, F1-score, AUC-ROC, and Matthews Correlation Coefficient (MCC), combined with an assessment of resilience to adversarial perturbations via the Fast Gradient Sign Method (FGSM). Among the tested models, XCeption achieves the highest accuracy (89.2% on DFDC), strong generalization, and real-time suitability, while VGG16 excels in precision and ResNet provides faster inference. However, all models exhibit reduced performance under adversarial conditions, underscoring the need for enhanced resilience. These findings indicate that robust detection systems must consider advanced generative approaches, adversarial defenses, and cross-dataset adaptation to effectively counter evolving deepfake threats. |
| format | Article |
| id | doaj-art-b7ff80e3195242b8a3a9d792739dffef |
| institution | OA Journals |
| issn | 2076-3417 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Applied Sciences |
| spelling | doaj-art-b7ff80e3195242b8a3a9d792739dffef2025-08-20T02:12:37ZengMDPI AGApplied Sciences2076-34172025-01-01153122510.3390/app15031225Comprehensive Evaluation of Deepfake Detection Models: Accuracy, Generalization, and Resilience to Adversarial AttacksMaryam Abbasi0Paulo Váz1José Silva2Pedro Martins3Applied Research Institute, Polytechnic of Coimbra, 3045-093 Coimbra, PortugalResearch Center in Digital Services (CISeD), Polytechnic of Viseu, 3504-510 Viseu, PortugalResearch Center in Digital Services (CISeD), Polytechnic of Viseu, 3504-510 Viseu, PortugalResearch Center in Digital Services (CISeD), Polytechnic of Viseu, 3504-510 Viseu, PortugalThe rise of deepfakes—synthetic media generated using artificial intelligence—threatens digital content authenticity, facilitating misinformation and manipulation. However, deepfakes can also depict real or entirely fictitious individuals, leveraging state-of-the-art techniques such as generative adversarial networks (GANs) and emerging diffusion-based models. Existing detection methods face challenges with generalization across datasets and vulnerability to adversarial attacks. This study focuses on subsets of frames extracted from the DeepFake Detection Challenge (DFDC) and FaceForensics++ videos to evaluate three convolutional neural network architectures—XCeption, ResNet, and VGG16—for deepfake detection. Performance metrics include accuracy, precision, F1-score, AUC-ROC, and Matthews Correlation Coefficient (MCC), combined with an assessment of resilience to adversarial perturbations via the Fast Gradient Sign Method (FGSM). Among the tested models, XCeption achieves the highest accuracy (89.2% on DFDC), strong generalization, and real-time suitability, while VGG16 excels in precision and ResNet provides faster inference. However, all models exhibit reduced performance under adversarial conditions, underscoring the need for enhanced resilience. These findings indicate that robust detection systems must consider advanced generative approaches, adversarial defenses, and cross-dataset adaptation to effectively counter evolving deepfake threats.https://www.mdpi.com/2076-3417/15/3/1225deepfakesdeep learningXCeptionResNetVGGDFDC |
| spellingShingle | Maryam Abbasi Paulo Váz José Silva Pedro Martins Comprehensive Evaluation of Deepfake Detection Models: Accuracy, Generalization, and Resilience to Adversarial Attacks Applied Sciences deepfakes deep learning XCeption ResNet VGG DFDC |
| title | Comprehensive Evaluation of Deepfake Detection Models: Accuracy, Generalization, and Resilience to Adversarial Attacks |
| title_full | Comprehensive Evaluation of Deepfake Detection Models: Accuracy, Generalization, and Resilience to Adversarial Attacks |
| title_fullStr | Comprehensive Evaluation of Deepfake Detection Models: Accuracy, Generalization, and Resilience to Adversarial Attacks |
| title_full_unstemmed | Comprehensive Evaluation of Deepfake Detection Models: Accuracy, Generalization, and Resilience to Adversarial Attacks |
| title_short | Comprehensive Evaluation of Deepfake Detection Models: Accuracy, Generalization, and Resilience to Adversarial Attacks |
| title_sort | comprehensive evaluation of deepfake detection models accuracy generalization and resilience to adversarial attacks |
| topic | deepfakes deep learning XCeption ResNet VGG DFDC |
| url | https://www.mdpi.com/2076-3417/15/3/1225 |
| work_keys_str_mv | AT maryamabbasi comprehensiveevaluationofdeepfakedetectionmodelsaccuracygeneralizationandresiliencetoadversarialattacks AT paulovaz comprehensiveevaluationofdeepfakedetectionmodelsaccuracygeneralizationandresiliencetoadversarialattacks AT josesilva comprehensiveevaluationofdeepfakedetectionmodelsaccuracygeneralizationandresiliencetoadversarialattacks AT pedromartins comprehensiveevaluationofdeepfakedetectionmodelsaccuracygeneralizationandresiliencetoadversarialattacks |