Continual learning with hebbian plasticity in sparse and predictive coding networks: a survey and perspective
Recently, the use of bio-inspired learning techniques such as Hebbian learning and its closely-related spike-timing-dependent plasticity (STDP) variant have drawn significant attention for the design of compute-efficient AI systems that can continuously learn on-line at the edge. A key differentiati...
Saved in:
| Main Author: | |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IOP Publishing
2024-01-01
|
| Series: | Neuromorphic Computing and Engineering |
| Subjects: | |
| Online Access: | https://doi.org/10.1088/2634-4386/ada08b |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850098626412412928 |
|---|---|
| author | Ali Safa |
| author_facet | Ali Safa |
| author_sort | Ali Safa |
| collection | DOAJ |
| description | Recently, the use of bio-inspired learning techniques such as Hebbian learning and its closely-related spike-timing-dependent plasticity (STDP) variant have drawn significant attention for the design of compute-efficient AI systems that can continuously learn on-line at the edge. A key differentiating factor regarding this emerging class of neuromorphic continual learning system lies in the fact that learning must be carried using a data stream received in its natural order, as opposed to conventional gradient-based offline training, where a static training dataset is assumed available a priori and randomly shuffled to make the training set independent and identically distributed (i.i.d). In contrast, the emerging class of neuromorphic CL systems covered in this survey must learn to integrate new information on the fly in a non-i.i.d manner, which makes these systems subject to catastrophic forgetting. In order to build the next generation of neuromorphic AI systems that can continuously learn at the edge, a growing number of research groups are studying the use of sparse and predictive Coding (PC)-based Hebbian neural network architectures and the related spiking neural networks (SNNs) equipped with STDP learning. However, since this research field is still emerging, there is a need for providing a holistic view of the different approaches proposed in the literature so far. To this end, this survey covers a number of recent works in the field of neuromorphic CL based on state-of-the-art sparse and PC technology; provides background theory to help interested researchers quickly learn the key concepts; and discusses important future research questions in light of the different works covered in this paper. It is hoped that this survey will contribute towards future research in the field of neuromorphic CL. |
| format | Article |
| id | doaj-art-60c9dc8872614372b3b8c7328ce0bd07 |
| institution | DOAJ |
| issn | 2634-4386 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | IOP Publishing |
| record_format | Article |
| series | Neuromorphic Computing and Engineering |
| spelling | doaj-art-60c9dc8872614372b3b8c7328ce0bd072025-08-20T02:40:40ZengIOP PublishingNeuromorphic Computing and Engineering2634-43862024-01-014404200110.1088/2634-4386/ada08bContinual learning with hebbian plasticity in sparse and predictive coding networks: a survey and perspectiveAli Safa0https://orcid.org/0000-0001-5768-8633College of Science and Engineering, Hamad Bin Khalifa University , Doha, QatarRecently, the use of bio-inspired learning techniques such as Hebbian learning and its closely-related spike-timing-dependent plasticity (STDP) variant have drawn significant attention for the design of compute-efficient AI systems that can continuously learn on-line at the edge. A key differentiating factor regarding this emerging class of neuromorphic continual learning system lies in the fact that learning must be carried using a data stream received in its natural order, as opposed to conventional gradient-based offline training, where a static training dataset is assumed available a priori and randomly shuffled to make the training set independent and identically distributed (i.i.d). In contrast, the emerging class of neuromorphic CL systems covered in this survey must learn to integrate new information on the fly in a non-i.i.d manner, which makes these systems subject to catastrophic forgetting. In order to build the next generation of neuromorphic AI systems that can continuously learn at the edge, a growing number of research groups are studying the use of sparse and predictive Coding (PC)-based Hebbian neural network architectures and the related spiking neural networks (SNNs) equipped with STDP learning. However, since this research field is still emerging, there is a need for providing a holistic view of the different approaches proposed in the literature so far. To this end, this survey covers a number of recent works in the field of neuromorphic CL based on state-of-the-art sparse and PC technology; provides background theory to help interested researchers quickly learn the key concepts; and discusses important future research questions in light of the different works covered in this paper. It is hoped that this survey will contribute towards future research in the field of neuromorphic CL.https://doi.org/10.1088/2634-4386/ada08bspiking neural networksnnspike timing dependent plasticitySTDPHebbiancontinual learning |
| spellingShingle | Ali Safa Continual learning with hebbian plasticity in sparse and predictive coding networks: a survey and perspective Neuromorphic Computing and Engineering spiking neural network snn spike timing dependent plasticity STDP Hebbian continual learning |
| title | Continual learning with hebbian plasticity in sparse and predictive coding networks: a survey and perspective |
| title_full | Continual learning with hebbian plasticity in sparse and predictive coding networks: a survey and perspective |
| title_fullStr | Continual learning with hebbian plasticity in sparse and predictive coding networks: a survey and perspective |
| title_full_unstemmed | Continual learning with hebbian plasticity in sparse and predictive coding networks: a survey and perspective |
| title_short | Continual learning with hebbian plasticity in sparse and predictive coding networks: a survey and perspective |
| title_sort | continual learning with hebbian plasticity in sparse and predictive coding networks a survey and perspective |
| topic | spiking neural network snn spike timing dependent plasticity STDP Hebbian continual learning |
| url | https://doi.org/10.1088/2634-4386/ada08b |
| work_keys_str_mv | AT alisafa continuallearningwithhebbianplasticityinsparseandpredictivecodingnetworksasurveyandperspective |