BiPruneFL: Computation and Communication Efficient Federated Learning With Binary Quantization and Pruning
Federated learning (FL) is a decentralized learning framework that allows a central server and multiple devices, referred to as clients, to collaboratively train a shared model without transmitting their private data to a central server. This approach helps to preserve data privacy and reduce the ri...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10909074/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| Summary: | Federated learning (FL) is a decentralized learning framework that allows a central server and multiple devices, referred to as clients, to collaboratively train a shared model without transmitting their private data to a central server. This approach helps to preserve data privacy and reduce the risk of information leakage. However, FL systems often face significant communication and computational overhead due to frequent exchanges of model parameters and the intensive local training required on resource-constrained clients. Existing solutions typically apply compression techniques such as quantization or pruning but only to a limited extent, constrained by the trade-off between model accuracy and compression efficiency. To address these challenges, we propose <monospace>BiPruneFL</monospace>, a communication- and computation-efficient FL framework that combines quantization and pruning while maintaining competitive accuracy. By leveraging recent advances in neural network pruning, <monospace>BiPruneFL</monospace> identifies subnetworks within binary neural networks without significantly compromising accuracy. Additionally, we employ communication compression strategies to enable efficient model updates and computationally lightweight local training. Through experiments, we demonstrate that <monospace>BiPruneFL</monospace> significantly outperforms other baselines, achieving up to <inline-formula> <tex-math notation="LaTeX">$88.1\times $ </tex-math></inline-formula> and <inline-formula> <tex-math notation="LaTeX">$80.8\times $ </tex-math></inline-formula> more efficient communication costs during upstream and downstream phases, respectively, and reducing computation costs by <inline-formula> <tex-math notation="LaTeX">$3.9\times $ </tex-math></inline-formula> to <inline-formula> <tex-math notation="LaTeX">$34.9\times $ </tex-math></inline-formula> depending on the degree of quantization. Despite these efficiency gains, <monospace>BiPruneFL</monospace> achieves accuracy comparable to, and in some cases surpassing, that of uncompressed federated learning models. |
|---|---|
| ISSN: | 2169-3536 |