Fine-Pruning: A biologically inspired algorithm for personalization of machine learning models
Summary: Neural networks have long strived to emulate the learning capabilities of the human brain. While deep neural networks (DNNs) draw inspiration from the brain in neuron design, their training methods diverge from biological foundations. Backpropagation, the primary training method for DNNs, r...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Elsevier
2025-05-01
|
| Series: | Patterns |
| Subjects: | |
| Online Access: | http://www.sciencedirect.com/science/article/pii/S266638992500090X |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850277971827359744 |
|---|---|
| author | Joseph Bingham Saman Zonouz Dvir Aran |
| author_facet | Joseph Bingham Saman Zonouz Dvir Aran |
| author_sort | Joseph Bingham |
| collection | DOAJ |
| description | Summary: Neural networks have long strived to emulate the learning capabilities of the human brain. While deep neural networks (DNNs) draw inspiration from the brain in neuron design, their training methods diverge from biological foundations. Backpropagation, the primary training method for DNNs, requires substantial computational resources and fully labeled datasets, presenting major bottlenecks in development and application. This work demonstrates that by returning to biomimicry, specifically mimicking how the brain learns through pruning, we can solve various classical machine learning problems while utilizing orders of magnitude fewer computational resources and no labels. Our experiments successfully personalized multiple speech recognition and image classification models, including ResNet50 on ImageNet, resulting in an increased sparsity of approximately 70% while simultaneously improving model accuracy to around 90%, all without the limitations of backpropagation. This biologically inspired approach offers a promising avenue for efficient, personalized machine learning models in resource-constrained environments. The bigger picture: Inspired by the concept of biological neural pruning, the Fine-Pruning approach addresses the need for large labeled datasets and extensive computational resources. It could enable real-time, on-device processing for small devices, including medical devices, without relying on cloud-based systems. Applications like personalized health monitoring, speech recognition, and image classification could become more efficient, accessible, and privacy preserving. This research paves the way for smarter, more adaptable devices that operate effectively in low-resource settings, transforming how we interact with technology in daily life and healthcare. |
| format | Article |
| id | doaj-art-1ebed9484e2a451baa4e673d43753852 |
| institution | OA Journals |
| issn | 2666-3899 |
| language | English |
| publishDate | 2025-05-01 |
| publisher | Elsevier |
| record_format | Article |
| series | Patterns |
| spelling | doaj-art-1ebed9484e2a451baa4e673d437538522025-08-20T01:49:40ZengElsevierPatterns2666-38992025-05-016510124210.1016/j.patter.2025.101242Fine-Pruning: A biologically inspired algorithm for personalization of machine learning modelsJoseph Bingham0Saman Zonouz1Dvir Aran2Faculty of Biology, Technion - Israel Institute of Technology, Haifa, Israel; Corresponding authorCollege of Computing, Georgia Institute of Technology, Atlanta, GA, USAFaculty of Biology, Technion - Israel Institute of Technology, Haifa, Israel; Taub Faculty of Computer Science, Technion - Israel Institute of Technology, Haifa, IsraelSummary: Neural networks have long strived to emulate the learning capabilities of the human brain. While deep neural networks (DNNs) draw inspiration from the brain in neuron design, their training methods diverge from biological foundations. Backpropagation, the primary training method for DNNs, requires substantial computational resources and fully labeled datasets, presenting major bottlenecks in development and application. This work demonstrates that by returning to biomimicry, specifically mimicking how the brain learns through pruning, we can solve various classical machine learning problems while utilizing orders of magnitude fewer computational resources and no labels. Our experiments successfully personalized multiple speech recognition and image classification models, including ResNet50 on ImageNet, resulting in an increased sparsity of approximately 70% while simultaneously improving model accuracy to around 90%, all without the limitations of backpropagation. This biologically inspired approach offers a promising avenue for efficient, personalized machine learning models in resource-constrained environments. The bigger picture: Inspired by the concept of biological neural pruning, the Fine-Pruning approach addresses the need for large labeled datasets and extensive computational resources. It could enable real-time, on-device processing for small devices, including medical devices, without relying on cloud-based systems. Applications like personalized health monitoring, speech recognition, and image classification could become more efficient, accessible, and privacy preserving. This research paves the way for smarter, more adaptable devices that operate effectively in low-resource settings, transforming how we interact with technology in daily life and healthcare.http://www.sciencedirect.com/science/article/pii/S266638992500090Xmachine learningbiomimicryneurosynaptic pruningbiologically feasible learninglot machine learningembedded learning |
| spellingShingle | Joseph Bingham Saman Zonouz Dvir Aran Fine-Pruning: A biologically inspired algorithm for personalization of machine learning models Patterns machine learning biomimicry neurosynaptic pruning biologically feasible learning lot machine learning embedded learning |
| title | Fine-Pruning: A biologically inspired algorithm for personalization of machine learning models |
| title_full | Fine-Pruning: A biologically inspired algorithm for personalization of machine learning models |
| title_fullStr | Fine-Pruning: A biologically inspired algorithm for personalization of machine learning models |
| title_full_unstemmed | Fine-Pruning: A biologically inspired algorithm for personalization of machine learning models |
| title_short | Fine-Pruning: A biologically inspired algorithm for personalization of machine learning models |
| title_sort | fine pruning a biologically inspired algorithm for personalization of machine learning models |
| topic | machine learning biomimicry neurosynaptic pruning biologically feasible learning lot machine learning embedded learning |
| url | http://www.sciencedirect.com/science/article/pii/S266638992500090X |
| work_keys_str_mv | AT josephbingham finepruningabiologicallyinspiredalgorithmforpersonalizationofmachinelearningmodels AT samanzonouz finepruningabiologicallyinspiredalgorithmforpersonalizationofmachinelearningmodels AT dviraran finepruningabiologicallyinspiredalgorithmforpersonalizationofmachinelearningmodels |