Continual Learning With Neuromorphic Computing: Foundations, Methods, and Emerging Applications
The challenging deployment of compute- and memory-intensive methods from Deep Neural Network (DNN)-based Continual Learning (CL), underscores the critical need for a paradigm shift towards more efficient approaches. Neuromorphic Continual Learning (NCL) appears as an emerging solution, by leveraging...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Access |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/11079552/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849245846879600640 |
|---|---|
| author | Mishal Fatima Minhas Rachmad Vidya Wicaksana Putra Falah Awwad Osman Hasan Muhammad Shafique |
| author_facet | Mishal Fatima Minhas Rachmad Vidya Wicaksana Putra Falah Awwad Osman Hasan Muhammad Shafique |
| author_sort | Mishal Fatima Minhas |
| collection | DOAJ |
| description | The challenging deployment of compute- and memory-intensive methods from Deep Neural Network (DNN)-based Continual Learning (CL), underscores the critical need for a paradigm shift towards more efficient approaches. Neuromorphic Continual Learning (NCL) appears as an emerging solution, by leveraging the principles of Spiking Neural Networks (SNNs) and their inherent advantages (e.g., sparse spike-driven operations and bio-plausible learning rules) for improving energy efficiency and performance, thereby enabling efficient CL algorithms (e.g., unsupervised learning approach) executed in dynamically-changed environments with resource-constrained computing systems. Though in its early stages, NCL is already a major research field with an increasing interest in novel SNN-based techniques for different CL methods (e.g., regularization-, replay-, and architecture-based). Motivated by the need for a holistic study of NCL, in this survey, we first provide a detailed background on CL, encompassing the desiderata, settings, metrics, scenario taxonomy, Online Continual Learning (OCL) paradigm, recent DNN-based methods proposed in the literature to address catastrophic forgetting (CF). Then, we analyze these methods based on their achieved CL desiderata, computational and memory costs, as well as network complexity, hence emphasizing the need for energy-efficient CL. After introducing the CL background and the energy efficiency challenges, we provide an extensive background of low-power neuromorphic computing systems including encoding techniques, neuronal dynamics, network architectures, learning rules, neuromorphic hardware processors, software and hardware frameworks, neuromorphic datasets, benchmarks, and evaluation metrics. Then, this survey comprehensively reviews and analyzes state-of-the-art works in the NCL field. The key ideas, implementation frameworks, and performance assessments (including CL, OCL, neuromorphic hardware compatibility aspects) are provided. This survey also covers several hybrid approaches that combine supervised and unsupervised learning paradigms and categorizes them into three main classes. It also covers optimization techniques including SNN operations reduction, weight quantization, and knowledge distillation. Then, this survey covers the progress of real-world NCL applications categorized into adaptive robots and autonomous vehicles with a wide range of use-cases i.e., object recognition, robotic arm control, cars and road lane detection, Simultaneous Localization and Mapping (SLAM), people detection and robotic navigation and provides their specific case-studies with empirical results. Finally, this paper provides a future perspective on the open research challenges for NCL, since the purpose of this study is to be useful for the wider neuromorphic AI research community and to inspire future research in bio-plausible OCL.INDEX TERMS Continual learning (CL), neuromorphic computing, spiking neural networks (SNNs), neuromorphic continual learning (NCL), event-based processing, energy efficiency, online continual learning (OCL), catastrophic forgetting (CF), deep neural networks (DNNs), artificial intelligence (AI), embedded AI systems. |
| format | Article |
| id | doaj-art-75c64c307c184af88e6cb569807f436e |
| institution | Kabale University |
| issn | 2169-3536 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Access |
| spelling | doaj-art-75c64c307c184af88e6cb569807f436e2025-08-20T03:58:40ZengIEEEIEEE Access2169-35362025-01-011312482412487310.1109/ACCESS.2025.358866511079552Continual Learning With Neuromorphic Computing: Foundations, Methods, and Emerging ApplicationsMishal Fatima Minhas0https://orcid.org/0000-0002-2123-5437Rachmad Vidya Wicaksana Putra1https://orcid.org/0000-0001-8597-4530Falah Awwad2https://orcid.org/0000-0001-6154-2143Osman Hasan3Muhammad Shafique4https://orcid.org/0000-0002-2607-8135Electrical and Communication Engineering Department, United Arab Emirates University (UAEU), Al Ain, United Arab EmirateseBRAIN Laboratory, New York University (NYU) Abu Dhabi, Abu Dhabi, United Arab EmiratesElectrical and Communication Engineering Department, United Arab Emirates University (UAEU), Al Ain, United Arab EmiratesSchool of Electrical Engineering and Computer Science (SEECS), National University of Sciences and Technology (NUST), Islamabad, PakistaneBRAIN Laboratory, New York University (NYU) Abu Dhabi, Abu Dhabi, United Arab EmiratesThe challenging deployment of compute- and memory-intensive methods from Deep Neural Network (DNN)-based Continual Learning (CL), underscores the critical need for a paradigm shift towards more efficient approaches. Neuromorphic Continual Learning (NCL) appears as an emerging solution, by leveraging the principles of Spiking Neural Networks (SNNs) and their inherent advantages (e.g., sparse spike-driven operations and bio-plausible learning rules) for improving energy efficiency and performance, thereby enabling efficient CL algorithms (e.g., unsupervised learning approach) executed in dynamically-changed environments with resource-constrained computing systems. Though in its early stages, NCL is already a major research field with an increasing interest in novel SNN-based techniques for different CL methods (e.g., regularization-, replay-, and architecture-based). Motivated by the need for a holistic study of NCL, in this survey, we first provide a detailed background on CL, encompassing the desiderata, settings, metrics, scenario taxonomy, Online Continual Learning (OCL) paradigm, recent DNN-based methods proposed in the literature to address catastrophic forgetting (CF). Then, we analyze these methods based on their achieved CL desiderata, computational and memory costs, as well as network complexity, hence emphasizing the need for energy-efficient CL. After introducing the CL background and the energy efficiency challenges, we provide an extensive background of low-power neuromorphic computing systems including encoding techniques, neuronal dynamics, network architectures, learning rules, neuromorphic hardware processors, software and hardware frameworks, neuromorphic datasets, benchmarks, and evaluation metrics. Then, this survey comprehensively reviews and analyzes state-of-the-art works in the NCL field. The key ideas, implementation frameworks, and performance assessments (including CL, OCL, neuromorphic hardware compatibility aspects) are provided. This survey also covers several hybrid approaches that combine supervised and unsupervised learning paradigms and categorizes them into three main classes. It also covers optimization techniques including SNN operations reduction, weight quantization, and knowledge distillation. Then, this survey covers the progress of real-world NCL applications categorized into adaptive robots and autonomous vehicles with a wide range of use-cases i.e., object recognition, robotic arm control, cars and road lane detection, Simultaneous Localization and Mapping (SLAM), people detection and robotic navigation and provides their specific case-studies with empirical results. Finally, this paper provides a future perspective on the open research challenges for NCL, since the purpose of this study is to be useful for the wider neuromorphic AI research community and to inspire future research in bio-plausible OCL.INDEX TERMS Continual learning (CL), neuromorphic computing, spiking neural networks (SNNs), neuromorphic continual learning (NCL), event-based processing, energy efficiency, online continual learning (OCL), catastrophic forgetting (CF), deep neural networks (DNNs), artificial intelligence (AI), embedded AI systems.https://ieeexplore.ieee.org/document/11079552/Continual Learning (CL)Neuromorphic ComputingSpiking Neural Networks (SNNs)Neuromorphic Continual Learning (NCL)Event-based ProcessingEnergy Efficiency |
| spellingShingle | Mishal Fatima Minhas Rachmad Vidya Wicaksana Putra Falah Awwad Osman Hasan Muhammad Shafique Continual Learning With Neuromorphic Computing: Foundations, Methods, and Emerging Applications IEEE Access Continual Learning (CL) Neuromorphic Computing Spiking Neural Networks (SNNs) Neuromorphic Continual Learning (NCL) Event-based Processing Energy Efficiency |
| title | Continual Learning With Neuromorphic Computing: Foundations, Methods, and Emerging Applications |
| title_full | Continual Learning With Neuromorphic Computing: Foundations, Methods, and Emerging Applications |
| title_fullStr | Continual Learning With Neuromorphic Computing: Foundations, Methods, and Emerging Applications |
| title_full_unstemmed | Continual Learning With Neuromorphic Computing: Foundations, Methods, and Emerging Applications |
| title_short | Continual Learning With Neuromorphic Computing: Foundations, Methods, and Emerging Applications |
| title_sort | continual learning with neuromorphic computing foundations methods and emerging applications |
| topic | Continual Learning (CL) Neuromorphic Computing Spiking Neural Networks (SNNs) Neuromorphic Continual Learning (NCL) Event-based Processing Energy Efficiency |
| url | https://ieeexplore.ieee.org/document/11079552/ |
| work_keys_str_mv | AT mishalfatimaminhas continuallearningwithneuromorphiccomputingfoundationsmethodsandemergingapplications AT rachmadvidyawicaksanaputra continuallearningwithneuromorphiccomputingfoundationsmethodsandemergingapplications AT falahawwad continuallearningwithneuromorphiccomputingfoundationsmethodsandemergingapplications AT osmanhasan continuallearningwithneuromorphiccomputingfoundationsmethodsandemergingapplications AT muhammadshafique continuallearningwithneuromorphiccomputingfoundationsmethodsandemergingapplications |