The backpropagation algorithm implemented on spiking neuromorphic hardware
Abstract The capabilities of natural neural systems have inspired both new generations of machine learning algorithms as well as neuromorphic, very large-scale integrated circuits capable of fast, low-power information processing. However, it has been argued that most modern machine learning algorit...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2024-11-01
|
| Series: | Nature Communications |
| Online Access: | https://doi.org/10.1038/s41467-024-53827-9 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850062268913418240 |
|---|---|
| author | Alpha Renner Forrest Sheldon Anatoly Zlotnik Louis Tao Andrew Sornborger |
| author_facet | Alpha Renner Forrest Sheldon Anatoly Zlotnik Louis Tao Andrew Sornborger |
| author_sort | Alpha Renner |
| collection | DOAJ |
| description | Abstract The capabilities of natural neural systems have inspired both new generations of machine learning algorithms as well as neuromorphic, very large-scale integrated circuits capable of fast, low-power information processing. However, it has been argued that most modern machine learning algorithms are not neurophysiologically plausible. In particular, the workhorse of modern deep learning, the backpropagation algorithm, has proven difficult to translate to neuromorphic hardware. This study presents a neuromorphic, spiking backpropagation algorithm based on synfire-gated dynamical information coordination and processing implemented on Intel’s Loihi neuromorphic research processor. We demonstrate a proof-of-principle three-layer circuit that learns to classify digits and clothing items from the MNIST and Fashion MNIST datasets. To our knowledge, this is the first work to show a Spiking Neural Network implementation of the exact backpropagation algorithm that is fully on-chip without a computer in the loop. It is competitive in accuracy with off-chip trained SNNs and achieves an energy-delay product suitable for edge computing. This implementation shows a path for using in-memory, massively parallel neuromorphic processors for low-power, low-latency implementation of modern deep learning applications. |
| format | Article |
| id | doaj-art-1bcbe702fd6249d98dfe0d1d908fbef5 |
| institution | DOAJ |
| issn | 2041-1723 |
| language | English |
| publishDate | 2024-11-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Nature Communications |
| spelling | doaj-art-1bcbe702fd6249d98dfe0d1d908fbef52025-08-20T02:49:59ZengNature PortfolioNature Communications2041-17232024-11-0115111410.1038/s41467-024-53827-9The backpropagation algorithm implemented on spiking neuromorphic hardwareAlpha Renner0Forrest Sheldon1Anatoly Zlotnik2Louis Tao3Andrew Sornborger4Institute of Neuroinformatics, University of Zurich and ETH ZurichPhysics of Condensed Matter & Complex Systems (T-4), Los Alamos National LaboratoryApplied Mathematics & Plasma Physics (T-5), Los Alamos National LaboratoryCenter for Bioinformatics, National Laboratory of Protein Engineering and Plant Genetic Engineering, School of Life Sciences, Peking UniversityInformation Sciences (CCS-3), Los Alamos National LaboratoryAbstract The capabilities of natural neural systems have inspired both new generations of machine learning algorithms as well as neuromorphic, very large-scale integrated circuits capable of fast, low-power information processing. However, it has been argued that most modern machine learning algorithms are not neurophysiologically plausible. In particular, the workhorse of modern deep learning, the backpropagation algorithm, has proven difficult to translate to neuromorphic hardware. This study presents a neuromorphic, spiking backpropagation algorithm based on synfire-gated dynamical information coordination and processing implemented on Intel’s Loihi neuromorphic research processor. We demonstrate a proof-of-principle three-layer circuit that learns to classify digits and clothing items from the MNIST and Fashion MNIST datasets. To our knowledge, this is the first work to show a Spiking Neural Network implementation of the exact backpropagation algorithm that is fully on-chip without a computer in the loop. It is competitive in accuracy with off-chip trained SNNs and achieves an energy-delay product suitable for edge computing. This implementation shows a path for using in-memory, massively parallel neuromorphic processors for low-power, low-latency implementation of modern deep learning applications.https://doi.org/10.1038/s41467-024-53827-9 |
| spellingShingle | Alpha Renner Forrest Sheldon Anatoly Zlotnik Louis Tao Andrew Sornborger The backpropagation algorithm implemented on spiking neuromorphic hardware Nature Communications |
| title | The backpropagation algorithm implemented on spiking neuromorphic hardware |
| title_full | The backpropagation algorithm implemented on spiking neuromorphic hardware |
| title_fullStr | The backpropagation algorithm implemented on spiking neuromorphic hardware |
| title_full_unstemmed | The backpropagation algorithm implemented on spiking neuromorphic hardware |
| title_short | The backpropagation algorithm implemented on spiking neuromorphic hardware |
| title_sort | backpropagation algorithm implemented on spiking neuromorphic hardware |
| url | https://doi.org/10.1038/s41467-024-53827-9 |
| work_keys_str_mv | AT alpharenner thebackpropagationalgorithmimplementedonspikingneuromorphichardware AT forrestsheldon thebackpropagationalgorithmimplementedonspikingneuromorphichardware AT anatolyzlotnik thebackpropagationalgorithmimplementedonspikingneuromorphichardware AT louistao thebackpropagationalgorithmimplementedonspikingneuromorphichardware AT andrewsornborger thebackpropagationalgorithmimplementedonspikingneuromorphichardware AT alpharenner backpropagationalgorithmimplementedonspikingneuromorphichardware AT forrestsheldon backpropagationalgorithmimplementedonspikingneuromorphichardware AT anatolyzlotnik backpropagationalgorithmimplementedonspikingneuromorphichardware AT louistao backpropagationalgorithmimplementedonspikingneuromorphichardware AT andrewsornborger backpropagationalgorithmimplementedonspikingneuromorphichardware |