Leveraging Gradient Noise for Detection and Filtering of Byzantine Clients
Distributed Learning enables multiple clients to collaboratively train large models on private, decentralized data. However, this setting faces a significant challenge: real-world datasets are inherently heterogeneous, and the distributed nature of the system makes it vulnerable to Byzantine attacks...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/11129040/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849223095379820544 |
|---|---|
| author | Latifa Errami Vyacheslav Kungurtsev El Houcine Bergou |
| author_facet | Latifa Errami Vyacheslav Kungurtsev El Houcine Bergou |
| author_sort | Latifa Errami |
| collection | DOAJ |
| description | Distributed Learning enables multiple clients to collaboratively train large models on private, decentralized data. However, this setting faces a significant challenge: real-world datasets are inherently heterogeneous, and the distributed nature of the system makes it vulnerable to Byzantine attacks. This combination makes it difficult for standard aggregations to reliably distinguish between honest clients with atypical data and malicious participants attempting to disrupt the training. To address this problem, we propose a novel Byzantine defense that leverages the statistical behavior of stochastic gradient noise (GN) in deep learning to identify malicious clients. Unlike pre-processing techniques that attempt to reduce inter-client gradient variance, our method directly exploits inherent gradient properties to filter malicious updates. Our aggregation can be deployed as a standalone defense or in combination with existing robust aggregation rules. We provide theoretical guarantees on convergence under standard assumptions. Our empirical evaluations on different benchmark datasets further demonstrates that our approach achieves high detection accuracy across a range of attack scenarios, significantly improving robustness without sacrificing model performance. |
| format | Article |
| id | doaj-art-72fcb7282c0c47dca83d81100a93b0ff |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-72fcb7282c0c47dca83d81100a93b0ff2025-08-25T23:12:28ZengIEEEIEEE Access2169-35362025-01-011314549414550610.1109/ACCESS.2025.360018911129040Leveraging Gradient Noise for Detection and Filtering of Byzantine ClientsLatifa Errami0https://orcid.org/0009-0005-0062-6307Vyacheslav Kungurtsev1El Houcine Bergou2https://orcid.org/0000-0001-8685-6974College of Computing, Mohammed VI Polytechnic University, Ben Guerir, MoroccoDepartment of Computer Science, Czech Technical University in Prague, Prague, Czech RepublicCollege of Computing, Mohammed VI Polytechnic University, Ben Guerir, MoroccoDistributed Learning enables multiple clients to collaboratively train large models on private, decentralized data. However, this setting faces a significant challenge: real-world datasets are inherently heterogeneous, and the distributed nature of the system makes it vulnerable to Byzantine attacks. This combination makes it difficult for standard aggregations to reliably distinguish between honest clients with atypical data and malicious participants attempting to disrupt the training. To address this problem, we propose a novel Byzantine defense that leverages the statistical behavior of stochastic gradient noise (GN) in deep learning to identify malicious clients. Unlike pre-processing techniques that attempt to reduce inter-client gradient variance, our method directly exploits inherent gradient properties to filter malicious updates. Our aggregation can be deployed as a standalone defense or in combination with existing robust aggregation rules. We provide theoretical guarantees on convergence under standard assumptions. Our empirical evaluations on different benchmark datasets further demonstrates that our approach achieves high detection accuracy across a range of attack scenarios, significantly improving robustness without sacrificing model performance.https://ieeexplore.ieee.org/document/11129040/Byzantine robustnessdistributed SGDnon-IID data |
| spellingShingle | Latifa Errami Vyacheslav Kungurtsev El Houcine Bergou Leveraging Gradient Noise for Detection and Filtering of Byzantine Clients IEEE Access Byzantine robustness distributed SGD non-IID data |
| title | Leveraging Gradient Noise for Detection and Filtering of Byzantine Clients |
| title_full | Leveraging Gradient Noise for Detection and Filtering of Byzantine Clients |
| title_fullStr | Leveraging Gradient Noise for Detection and Filtering of Byzantine Clients |
| title_full_unstemmed | Leveraging Gradient Noise for Detection and Filtering of Byzantine Clients |
| title_short | Leveraging Gradient Noise for Detection and Filtering of Byzantine Clients |
| title_sort | leveraging gradient noise for detection and filtering of byzantine clients |
| topic | Byzantine robustness distributed SGD non-IID data |
| url | https://ieeexplore.ieee.org/document/11129040/ |
| work_keys_str_mv | AT latifaerrami leveraginggradientnoisefordetectionandfilteringofbyzantineclients AT vyacheslavkungurtsev leveraginggradientnoisefordetectionandfilteringofbyzantineclients AT elhoucinebergou leveraginggradientnoisefordetectionandfilteringofbyzantineclients |