DS4NN: Direct training of deep spiking neural networks with single spike-based temporal coding
Backpropagation is the foremost prevalent and common algorithm for training conventional neural networks with deep construction. Here we propose DS4NN, temporal backpropagation for deep spiking neural networks with one spike per neuron. We consider a convolutional spiking neural network consisting o...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Amirkabir University of Technology
2023-12-01
|
| Series: | AUT Journal of Electrical Engineering |
| Subjects: | |
| Online Access: | https://eej.aut.ac.ir/article_5080_802cb5a6c14d7e84c5eb6168b526f23a.pdf |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849420190485315584 |
|---|---|
| author | Maryam Mirsadeghi Majid Shalchian Saeed Reza Kheradpisheh |
| author_facet | Maryam Mirsadeghi Majid Shalchian Saeed Reza Kheradpisheh |
| author_sort | Maryam Mirsadeghi |
| collection | DOAJ |
| description | Backpropagation is the foremost prevalent and common algorithm for training conventional neural networks with deep construction. Here we propose DS4NN, temporal backpropagation for deep spiking neural networks with one spike per neuron. We consider a convolutional spiking neural network consisting of simple non-leaky integrate-and-fire (IF) neurons, and a form of coding named time-to-first-spike temporal coding in which, neurons are allowed to fire at most once in a specific time interval, which corresponds to simulation duration here. These features together improve the cost and the speed of network computation. We use a surrogate gradient at firing times to solve the non-differentiability of spike times concerning the membrane potential of spiking neurons, and to prevent the emergence of dead neurons in deep layers, we propose a relative encoding scheme for determining desired firing times. Evaluations on two classification tasks of MNIST and Fashion-MNIST datasets confirm the capability of DS4NN on the deep structure of SNNs. It achieves the accuracy of 99.3% (99.8%) and 91.6% (95.3%) on testing samples (training samples) of respectively MNIST and Fashion-MNIST datasets with the mean required number of 1126 and 1863 spikes in the whole network. This shows that the proposed approach can make fast decisions with low-cost computation and high accuracy. |
| format | Article |
| id | doaj-art-e96ec07fc1824876a80f86ba24008909 |
| institution | Kabale University |
| issn | 2588-2910 2588-2929 |
| language | English |
| publishDate | 2023-12-01 |
| publisher | Amirkabir University of Technology |
| record_format | Article |
| series | AUT Journal of Electrical Engineering |
| spelling | doaj-art-e96ec07fc1824876a80f86ba240089092025-08-20T03:31:49ZengAmirkabir University of TechnologyAUT Journal of Electrical Engineering2588-29102588-29292023-12-0155217919010.22060/eej.2023.21991.55025080DS4NN: Direct training of deep spiking neural networks with single spike-based temporal codingMaryam Mirsadeghi0Majid Shalchian1Saeed Reza Kheradpisheh2Department of Electrical Engineering, Amirkabir University of Technology, Tehran, IranDepartment of Electrical Engineering, Amirkabir University of Technology, Tehran, IranFaculty of Mathematical sciences, Shahid Beheshti University, Tehran, IranBackpropagation is the foremost prevalent and common algorithm for training conventional neural networks with deep construction. Here we propose DS4NN, temporal backpropagation for deep spiking neural networks with one spike per neuron. We consider a convolutional spiking neural network consisting of simple non-leaky integrate-and-fire (IF) neurons, and a form of coding named time-to-first-spike temporal coding in which, neurons are allowed to fire at most once in a specific time interval, which corresponds to simulation duration here. These features together improve the cost and the speed of network computation. We use a surrogate gradient at firing times to solve the non-differentiability of spike times concerning the membrane potential of spiking neurons, and to prevent the emergence of dead neurons in deep layers, we propose a relative encoding scheme for determining desired firing times. Evaluations on two classification tasks of MNIST and Fashion-MNIST datasets confirm the capability of DS4NN on the deep structure of SNNs. It achieves the accuracy of 99.3% (99.8%) and 91.6% (95.3%) on testing samples (training samples) of respectively MNIST and Fashion-MNIST datasets with the mean required number of 1126 and 1863 spikes in the whole network. This shows that the proposed approach can make fast decisions with low-cost computation and high accuracy.https://eej.aut.ac.ir/article_5080_802cb5a6c14d7e84c5eb6168b526f23a.pdfdeep spiking neural networktemporal backpropagationsingle spike-based codingsupervised learningintegrate-and-fire neuron model |
| spellingShingle | Maryam Mirsadeghi Majid Shalchian Saeed Reza Kheradpisheh DS4NN: Direct training of deep spiking neural networks with single spike-based temporal coding AUT Journal of Electrical Engineering deep spiking neural network temporal backpropagation single spike-based coding supervised learning integrate-and-fire neuron model |
| title | DS4NN: Direct training of deep spiking neural networks with single spike-based temporal coding |
| title_full | DS4NN: Direct training of deep spiking neural networks with single spike-based temporal coding |
| title_fullStr | DS4NN: Direct training of deep spiking neural networks with single spike-based temporal coding |
| title_full_unstemmed | DS4NN: Direct training of deep spiking neural networks with single spike-based temporal coding |
| title_short | DS4NN: Direct training of deep spiking neural networks with single spike-based temporal coding |
| title_sort | ds4nn direct training of deep spiking neural networks with single spike based temporal coding |
| topic | deep spiking neural network temporal backpropagation single spike-based coding supervised learning integrate-and-fire neuron model |
| url | https://eej.aut.ac.ir/article_5080_802cb5a6c14d7e84c5eb6168b526f23a.pdf |
| work_keys_str_mv | AT maryammirsadeghi ds4nndirecttrainingofdeepspikingneuralnetworkswithsinglespikebasedtemporalcoding AT majidshalchian ds4nndirecttrainingofdeepspikingneuralnetworkswithsinglespikebasedtemporalcoding AT saeedrezakheradpisheh ds4nndirecttrainingofdeepspikingneuralnetworkswithsinglespikebasedtemporalcoding |