KL-FedDis: A federated learning approach with distribution information sharing using Kullback-Leibler divergence for non-IID data
Data Heterogeneity or Non-IID (non-independent and identically distributed) data identification is one of the prominent challenges in Federated Learning (FL). In Non-IID data, clients have their own local data, which may not be independently and identically distributed. This arises because clients i...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2025-03-01
|
| Series: | Neuroscience Informatics |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S277252862400027X |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850026469702500352 |
|---|---|
| author | Md. Rahad Ruhan Shabab Mohd. Sultan Ahammad Md. Mahfuz Reza Amit Karmaker Md. Abir Hossain |
| author_facet | Md. Rahad Ruhan Shabab Mohd. Sultan Ahammad Md. Mahfuz Reza Amit Karmaker Md. Abir Hossain |
| author_sort | Md. Rahad |
| collection | DOAJ |
| description | Data Heterogeneity or Non-IID (non-independent and identically distributed) data identification is one of the prominent challenges in Federated Learning (FL). In Non-IID data, clients have their own local data, which may not be independently and identically distributed. This arises because clients involved in federated learning typically have their own unique, local datasets that vary significantly due to factors like geographical location, user behaviors, or specific contexts. Model divergence is another critical challenge where the local models trained on different clients, data may diverge significantly but making it difficult for the global model to converge. To identify the non-IID data, few federated learning models have been introduced as FedDis, FedProx and FedAvg, but their accuracy is too low. To address the clients Non-IID data along with ensuring privacy, federated learning emerged with appropriate distribution mechanism is an effective solution. In this paper, a modified FedDis learning method called KL-FedDis is proposed, which incorporates Kullback-Leibler (KL) divergence as the regularization technique. KL-FedDis improves accuracy and computation time over the FedDis and FedAvg technique by successfully maintaining the distribution information and encouraging improved collaboration among the local models by utilizing KL divergence. |
| format | Article |
| id | doaj-art-3c9d05dae92349cdb978ccbb4265fa9a |
| institution | DOAJ |
| issn | 2772-5286 |
| language | English |
| publishDate | 2025-03-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Neuroscience Informatics |
| spelling | doaj-art-3c9d05dae92349cdb978ccbb4265fa9a2025-08-20T03:00:32ZengElsevierNeuroscience Informatics2772-52862025-03-015110018210.1016/j.neuri.2024.100182KL-FedDis: A federated learning approach with distribution information sharing using Kullback-Leibler divergence for non-IID dataMd. Rahad0Ruhan Shabab1Mohd. Sultan Ahammad2Md. Mahfuz Reza3Amit Karmaker4Md. Abir Hossain5Computer Science & Engineering, Mawlana Bhashani Science and Technology University, Tangail, 1902, BangladeshComputer Science & Engineering, Mawlana Bhashani Science and Technology University, Tangail, 1902, BangladeshComputer Science & Engineering, Mawlana Bhashani Science and Technology University, Tangail, 1902, BangladeshComputer Science & Engineering, Mawlana Bhashani Science and Technology University, Tangail, 1902, BangladeshInstitute of Information & Communication Technology, Bangladesh University of Engineering and Technology, Dhaka, 1205, BangladeshInformation & Communication Technology, Mawlana Bhashani Science and Technology University, Tangail, 1902, Bangladesh; Institute of Information & Communication Technology, Bangladesh University of Engineering and Technology, Dhaka, 1205, Bangladesh; Corresponding author at: Information & Communication Technology, Mawlana Bhashani Science and Technology University, Tangail, 1902, Bangladesh.Data Heterogeneity or Non-IID (non-independent and identically distributed) data identification is one of the prominent challenges in Federated Learning (FL). In Non-IID data, clients have their own local data, which may not be independently and identically distributed. This arises because clients involved in federated learning typically have their own unique, local datasets that vary significantly due to factors like geographical location, user behaviors, or specific contexts. Model divergence is another critical challenge where the local models trained on different clients, data may diverge significantly but making it difficult for the global model to converge. To identify the non-IID data, few federated learning models have been introduced as FedDis, FedProx and FedAvg, but their accuracy is too low. To address the clients Non-IID data along with ensuring privacy, federated learning emerged with appropriate distribution mechanism is an effective solution. In this paper, a modified FedDis learning method called KL-FedDis is proposed, which incorporates Kullback-Leibler (KL) divergence as the regularization technique. KL-FedDis improves accuracy and computation time over the FedDis and FedAvg technique by successfully maintaining the distribution information and encouraging improved collaboration among the local models by utilizing KL divergence.http://www.sciencedirect.com/science/article/pii/S277252862400027XFederated learningFedDisKL divergenceRegularization |
| spellingShingle | Md. Rahad Ruhan Shabab Mohd. Sultan Ahammad Md. Mahfuz Reza Amit Karmaker Md. Abir Hossain KL-FedDis: A federated learning approach with distribution information sharing using Kullback-Leibler divergence for non-IID data Neuroscience Informatics Federated learning FedDis KL divergence Regularization |
| title | KL-FedDis: A federated learning approach with distribution information sharing using Kullback-Leibler divergence for non-IID data |
| title_full | KL-FedDis: A federated learning approach with distribution information sharing using Kullback-Leibler divergence for non-IID data |
| title_fullStr | KL-FedDis: A federated learning approach with distribution information sharing using Kullback-Leibler divergence for non-IID data |
| title_full_unstemmed | KL-FedDis: A federated learning approach with distribution information sharing using Kullback-Leibler divergence for non-IID data |
| title_short | KL-FedDis: A federated learning approach with distribution information sharing using Kullback-Leibler divergence for non-IID data |
| title_sort | kl feddis a federated learning approach with distribution information sharing using kullback leibler divergence for non iid data |
| topic | Federated learning FedDis KL divergence Regularization |
| url | http://www.sciencedirect.com/science/article/pii/S277252862400027X |
| work_keys_str_mv | AT mdrahad klfeddisafederatedlearningapproachwithdistributioninformationsharingusingkullbackleiblerdivergencefornoniiddata AT ruhanshabab klfeddisafederatedlearningapproachwithdistributioninformationsharingusingkullbackleiblerdivergencefornoniiddata AT mohdsultanahammad klfeddisafederatedlearningapproachwithdistributioninformationsharingusingkullbackleiblerdivergencefornoniiddata AT mdmahfuzreza klfeddisafederatedlearningapproachwithdistributioninformationsharingusingkullbackleiblerdivergencefornoniiddata AT amitkarmaker klfeddisafederatedlearningapproachwithdistributioninformationsharingusingkullbackleiblerdivergencefornoniiddata AT mdabirhossain klfeddisafederatedlearningapproachwithdistributioninformationsharingusingkullbackleiblerdivergencefornoniiddata |