Foundational models and federated learning: survey, taxonomy, challenges and practical insights
Federated learning has the potential to unlock siloed data and distributed resources by enabling collaborative model training without sharing private data. As more complex foundational models gain widespread use, the need to expand training resources and integrate privately owned data grows as well....
Saved in:
| Main Authors: | , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
PeerJ Inc.
2025-07-01
|
| Series: | PeerJ Computer Science |
| Subjects: | |
| Online Access: | https://peerj.com/articles/cs-2993.pdf |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849414174448287744 |
|---|---|
| author | Cosmin-Andrei Hatfaludi Alex Serban |
| author_facet | Cosmin-Andrei Hatfaludi Alex Serban |
| author_sort | Cosmin-Andrei Hatfaludi |
| collection | DOAJ |
| description | Federated learning has the potential to unlock siloed data and distributed resources by enabling collaborative model training without sharing private data. As more complex foundational models gain widespread use, the need to expand training resources and integrate privately owned data grows as well. In this article, we explore the intersection of federated learning and foundational models, aiming to identify, categorize, and characterize technical methods that integrate the two paradigms. As a unified survey is currently unavailable, we present a literature survey structured around a novel taxonomy that follows the development life-cycle stages, along with a technical comparison of available methods. Additionally, we provide practical insights and guidelines for implementing and evolving these methods, with a specific focus on the healthcare domain as a case study, where the potential impact of federated learning and foundational models is considered significant. Our survey covers multiple intersecting topics, including but not limited to federated learning, self-supervised learning, fine-tuning, distillation, and transfer learning. Initially, we retrieved and reviewed a set of over 4,200 articles. This collection was narrowed to more than 250 thoroughly reviewed articles through inclusion criteria, featuring 42 unique methods. The methods were used to construct the taxonomy and enabled their comparison based on complexity, efficiency, and scalability. We present these results as a self-contained overview that not only summarizes the state of the field but also provides insights into the practical aspects of adopting, evolving, and integrating foundational models with federated learning. |
| format | Article |
| id | doaj-art-02407196a5e64b5dbc701c10733fd3ab |
| institution | Kabale University |
| issn | 2376-5992 |
| language | English |
| publishDate | 2025-07-01 |
| publisher | PeerJ Inc. |
| record_format | Article |
| series | PeerJ Computer Science |
| spelling | doaj-art-02407196a5e64b5dbc701c10733fd3ab2025-08-20T03:33:54ZengPeerJ Inc.PeerJ Computer Science2376-59922025-07-0111e299310.7717/peerj-cs.2993Foundational models and federated learning: survey, taxonomy, challenges and practical insightsCosmin-Andrei Hatfaludi0Alex Serban1Foundational Technologies, Siemens SRL, Brasov, RomaniaFoundational Technologies, Siemens SRL, Brasov, RomaniaFederated learning has the potential to unlock siloed data and distributed resources by enabling collaborative model training without sharing private data. As more complex foundational models gain widespread use, the need to expand training resources and integrate privately owned data grows as well. In this article, we explore the intersection of federated learning and foundational models, aiming to identify, categorize, and characterize technical methods that integrate the two paradigms. As a unified survey is currently unavailable, we present a literature survey structured around a novel taxonomy that follows the development life-cycle stages, along with a technical comparison of available methods. Additionally, we provide practical insights and guidelines for implementing and evolving these methods, with a specific focus on the healthcare domain as a case study, where the potential impact of federated learning and foundational models is considered significant. Our survey covers multiple intersecting topics, including but not limited to federated learning, self-supervised learning, fine-tuning, distillation, and transfer learning. Initially, we retrieved and reviewed a set of over 4,200 articles. This collection was narrowed to more than 250 thoroughly reviewed articles through inclusion criteria, featuring 42 unique methods. The methods were used to construct the taxonomy and enabled their comparison based on complexity, efficiency, and scalability. We present these results as a self-contained overview that not only summarizes the state of the field but also provides insights into the practical aspects of adopting, evolving, and integrating foundational models with federated learning.https://peerj.com/articles/cs-2993.pdfFederated learningFoundational modelsMachine learningPrivacy preservingSurveyHealthcare |
| spellingShingle | Cosmin-Andrei Hatfaludi Alex Serban Foundational models and federated learning: survey, taxonomy, challenges and practical insights PeerJ Computer Science Federated learning Foundational models Machine learning Privacy preserving Survey Healthcare |
| title | Foundational models and federated learning: survey, taxonomy, challenges and practical insights |
| title_full | Foundational models and federated learning: survey, taxonomy, challenges and practical insights |
| title_fullStr | Foundational models and federated learning: survey, taxonomy, challenges and practical insights |
| title_full_unstemmed | Foundational models and federated learning: survey, taxonomy, challenges and practical insights |
| title_short | Foundational models and federated learning: survey, taxonomy, challenges and practical insights |
| title_sort | foundational models and federated learning survey taxonomy challenges and practical insights |
| topic | Federated learning Foundational models Machine learning Privacy preserving Survey Healthcare |
| url | https://peerj.com/articles/cs-2993.pdf |
| work_keys_str_mv | AT cosminandreihatfaludi foundationalmodelsandfederatedlearningsurveytaxonomychallengesandpracticalinsights AT alexserban foundationalmodelsandfederatedlearningsurveytaxonomychallengesandpracticalinsights |