Deepfake Detection Method Integrating Multiple Parameter-Efficient Fine-Tuning Techniques
In recent years, as deepfake technology matures, face-swapping software and synthesized videos have become widespread. While these techniques offer entertainment, they also provide opportunities for misuse by malicious actors. Consequently, the significance of deepfake detection technology has grown...
Saved in:
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | zho |
| Published: |
Journal of Computer Engineering and Applications Beijing Co., Ltd., Science Press
2024-12-01
|
| Series: | Jisuanji kexue yu tansuo |
| Subjects: | |
| Online Access: | http://fcst.ceaj.org/fileup/1673-9418/PDF/2311053.pdf |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850057369547964416 |
|---|---|
| author | ZHANG Yiwen, CAI Manchun, CHEN Yonghao, ZHU Yi, YAO Lifeng |
| author_facet | ZHANG Yiwen, CAI Manchun, CHEN Yonghao, ZHU Yi, YAO Lifeng |
| author_sort | ZHANG Yiwen, CAI Manchun, CHEN Yonghao, ZHU Yi, YAO Lifeng |
| collection | DOAJ |
| description | In recent years, as deepfake technology matures, face-swapping software and synthesized videos have become widespread. While these techniques offer entertainment, they also provide opportunities for misuse by malicious actors. Consequently, the significance of deepfake detection technology has grown markedly. Existing methods for deepfake detection commonly suffer from issues including poor cross-compression robustness, weak cross-dataset generalization, and high model training overheads. To address these challenges, this paper proposes a deepfake detection approach that combines multiple parameter-efficient fine-tuning techniques. This method utilizes a visual Transformer model pretrained with the masked image modeling self-supervised method as its backbone. Initially, it employs the low-rank adaptation (LoRA) method for fine-tuning the self-attention module parameters of the pretrained model. Concurrently, it introduces a parallel structure incorporating convolutional adapters to capture local texture information, enhancing the model’s adaptability in deepfake detection tasks. Subsequently, a serial structure introduces classical adapters to fine-tune the feed-forward network of the pretrained model, maximizing the utilization of knowledge acquired during the pretraining phase. Ultimately, a multi-layer perceptron replaces the original pretrained model’s classification head for deepfake detection. Experimental results across six datasets demonstrate that this model achieves an average frame-level AUC of approximately 0.996 with only 2×107 trainable parameters. In cross-compression experiments, the average frame-level AUC drop is 0.135. In cross-dataset generalization experiments, the frame-level AUC averages around 0.765. |
| format | Article |
| id | doaj-art-8d514d8a8dc24b9e805183d2f7c2c93d |
| institution | DOAJ |
| issn | 1673-9418 |
| language | zho |
| publishDate | 2024-12-01 |
| publisher | Journal of Computer Engineering and Applications Beijing Co., Ltd., Science Press |
| record_format | Article |
| series | Jisuanji kexue yu tansuo |
| spelling | doaj-art-8d514d8a8dc24b9e805183d2f7c2c93d2025-08-20T02:51:27ZzhoJournal of Computer Engineering and Applications Beijing Co., Ltd., Science PressJisuanji kexue yu tansuo1673-94182024-12-0118123335334710.3778/j.issn.1673-9418.2311053Deepfake Detection Method Integrating Multiple Parameter-Efficient Fine-Tuning TechniquesZHANG Yiwen, CAI Manchun, CHEN Yonghao, ZHU Yi, YAO Lifeng0College of Information and Cyber Security, People’s Public Security University of China, Beijing 100038, ChinaIn recent years, as deepfake technology matures, face-swapping software and synthesized videos have become widespread. While these techniques offer entertainment, they also provide opportunities for misuse by malicious actors. Consequently, the significance of deepfake detection technology has grown markedly. Existing methods for deepfake detection commonly suffer from issues including poor cross-compression robustness, weak cross-dataset generalization, and high model training overheads. To address these challenges, this paper proposes a deepfake detection approach that combines multiple parameter-efficient fine-tuning techniques. This method utilizes a visual Transformer model pretrained with the masked image modeling self-supervised method as its backbone. Initially, it employs the low-rank adaptation (LoRA) method for fine-tuning the self-attention module parameters of the pretrained model. Concurrently, it introduces a parallel structure incorporating convolutional adapters to capture local texture information, enhancing the model’s adaptability in deepfake detection tasks. Subsequently, a serial structure introduces classical adapters to fine-tune the feed-forward network of the pretrained model, maximizing the utilization of knowledge acquired during the pretraining phase. Ultimately, a multi-layer perceptron replaces the original pretrained model’s classification head for deepfake detection. Experimental results across six datasets demonstrate that this model achieves an average frame-level AUC of approximately 0.996 with only 2×107 trainable parameters. In cross-compression experiments, the average frame-level AUC drop is 0.135. In cross-dataset generalization experiments, the frame-level AUC averages around 0.765.http://fcst.ceaj.org/fileup/1673-9418/PDF/2311053.pdfdeepfakes; vision transformer; self-supervised pretrained models; low-rank adaptation (lora); parameter-efficient fine-tuning |
| spellingShingle | ZHANG Yiwen, CAI Manchun, CHEN Yonghao, ZHU Yi, YAO Lifeng Deepfake Detection Method Integrating Multiple Parameter-Efficient Fine-Tuning Techniques Jisuanji kexue yu tansuo deepfakes; vision transformer; self-supervised pretrained models; low-rank adaptation (lora); parameter-efficient fine-tuning |
| title | Deepfake Detection Method Integrating Multiple Parameter-Efficient Fine-Tuning Techniques |
| title_full | Deepfake Detection Method Integrating Multiple Parameter-Efficient Fine-Tuning Techniques |
| title_fullStr | Deepfake Detection Method Integrating Multiple Parameter-Efficient Fine-Tuning Techniques |
| title_full_unstemmed | Deepfake Detection Method Integrating Multiple Parameter-Efficient Fine-Tuning Techniques |
| title_short | Deepfake Detection Method Integrating Multiple Parameter-Efficient Fine-Tuning Techniques |
| title_sort | deepfake detection method integrating multiple parameter efficient fine tuning techniques |
| topic | deepfakes; vision transformer; self-supervised pretrained models; low-rank adaptation (lora); parameter-efficient fine-tuning |
| url | http://fcst.ceaj.org/fileup/1673-9418/PDF/2311053.pdf |
| work_keys_str_mv | AT zhangyiwencaimanchunchenyonghaozhuyiyaolifeng deepfakedetectionmethodintegratingmultipleparameterefficientfinetuningtechniques |