Hierarchical Learning: A Hybrid of Federated Learning and Personalization Fine-Tuning
Hierarchical Federated Learning (FL) presents a novel approach that combines global model training with localized personalization fine-tuning to enhance the predictive accuracy of decentralized machine learning systems. Traditional FL methods, which allow multiple clients to collaboratively train a...
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
EDP Sciences
2025-01-01
|
| Series: | ITM Web of Conferences |
| Online Access: | https://www.itm-conferences.org/articles/itmconf/pdf/2025/01/itmconf_dai2024_03029.pdf |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1825206547723059200 |
|---|---|
| author | Li Shuyi Zhang Bairong |
| author_facet | Li Shuyi Zhang Bairong |
| author_sort | Li Shuyi |
| collection | DOAJ |
| description | Hierarchical Federated Learning (FL) presents a novel approach that combines global model training with localized personalization fine-tuning to enhance the predictive accuracy of decentralized machine learning systems. Traditional FL methods, which allow multiple clients to collaboratively train a global model without sharing raw data, are hindered by issues such as non-independent and identically distributed (non-IID) data, communication overhead, and limited generalization across diverse client datasets. This study proposes a hierarchical model that mitigates these challenges by incorporating a global model, trained using the Federated Averaging (FedAvg) algorithm, and applying client-specific fine-tuning to improve local model performance. The experiment conducted on a movie recommendation system demonstrates that this hierarchical approach significantly reduces the global model’s error while offering personalized improvements on client-specific datasets. Results show an average Root Mean Squared Error (RMSE) reduction of 0.0460 following local personalization. This hybrid approach not only enhances model accuracy but also preserves data privacy and increases scalability, making it a promising solution for decentralized recommendation systems. |
| format | Article |
| id | doaj-art-20967a82f7194a76a1f876d3eaf5e871 |
| institution | Kabale University |
| issn | 2271-2097 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | EDP Sciences |
| record_format | Article |
| series | ITM Web of Conferences |
| spelling | doaj-art-20967a82f7194a76a1f876d3eaf5e8712025-02-07T08:21:11ZengEDP SciencesITM Web of Conferences2271-20972025-01-01700302910.1051/itmconf/20257003029itmconf_dai2024_03029Hierarchical Learning: A Hybrid of Federated Learning and Personalization Fine-TuningLi Shuyi0Zhang Bairong1Computer Science, University of Wisconsin-MadisonComputer Science, Rensselaer Polytechnic InstituteHierarchical Federated Learning (FL) presents a novel approach that combines global model training with localized personalization fine-tuning to enhance the predictive accuracy of decentralized machine learning systems. Traditional FL methods, which allow multiple clients to collaboratively train a global model without sharing raw data, are hindered by issues such as non-independent and identically distributed (non-IID) data, communication overhead, and limited generalization across diverse client datasets. This study proposes a hierarchical model that mitigates these challenges by incorporating a global model, trained using the Federated Averaging (FedAvg) algorithm, and applying client-specific fine-tuning to improve local model performance. The experiment conducted on a movie recommendation system demonstrates that this hierarchical approach significantly reduces the global model’s error while offering personalized improvements on client-specific datasets. Results show an average Root Mean Squared Error (RMSE) reduction of 0.0460 following local personalization. This hybrid approach not only enhances model accuracy but also preserves data privacy and increases scalability, making it a promising solution for decentralized recommendation systems.https://www.itm-conferences.org/articles/itmconf/pdf/2025/01/itmconf_dai2024_03029.pdf |
| spellingShingle | Li Shuyi Zhang Bairong Hierarchical Learning: A Hybrid of Federated Learning and Personalization Fine-Tuning ITM Web of Conferences |
| title | Hierarchical Learning: A Hybrid of Federated Learning and Personalization Fine-Tuning |
| title_full | Hierarchical Learning: A Hybrid of Federated Learning and Personalization Fine-Tuning |
| title_fullStr | Hierarchical Learning: A Hybrid of Federated Learning and Personalization Fine-Tuning |
| title_full_unstemmed | Hierarchical Learning: A Hybrid of Federated Learning and Personalization Fine-Tuning |
| title_short | Hierarchical Learning: A Hybrid of Federated Learning and Personalization Fine-Tuning |
| title_sort | hierarchical learning a hybrid of federated learning and personalization fine tuning |
| url | https://www.itm-conferences.org/articles/itmconf/pdf/2025/01/itmconf_dai2024_03029.pdf |
| work_keys_str_mv | AT lishuyi hierarchicallearningahybridoffederatedlearningandpersonalizationfinetuning AT zhangbairong hierarchicallearningahybridoffederatedlearningandpersonalizationfinetuning |