Energy‐Efficient Hardware Implementation of Spiking‐Restricted Boltzmann Machines Using Pseudo‐Synaptic Sampling
Stochastic sampling is performed to reduce hardware energy consumption and prevent overfitting by reducing parameters, because not all data are required for learning. In this study, a new approach, pseudo‐synaptic sampling (PS2) method, which approximates the conventional synaptic sampling machine (...
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2025-05-01
|
| Series: | Advanced Intelligent Systems |
| Subjects: | |
| Online Access: | https://doi.org/10.1002/aisy.202400557 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850261559106863104 |
|---|---|
| author | Hyunwoo Kim Suyeon Jang Uicheol Shin Masatoshi Ishii Atsuya Okazaki Megumi Ito Akiyo Nomura Kohji Hosokawa Sungmin Lee Matthew BrightSky Sangbum Kim |
| author_facet | Hyunwoo Kim Suyeon Jang Uicheol Shin Masatoshi Ishii Atsuya Okazaki Megumi Ito Akiyo Nomura Kohji Hosokawa Sungmin Lee Matthew BrightSky Sangbum Kim |
| author_sort | Hyunwoo Kim |
| collection | DOAJ |
| description | Stochastic sampling is performed to reduce hardware energy consumption and prevent overfitting by reducing parameters, because not all data are required for learning. In this study, a new approach, pseudo‐synaptic sampling (PS2) method, which approximates the conventional synaptic sampling machine (S2M) method through a hardware‐friendly implementation while demonstrating superior efficiency, is introduced. By sampling in front of neurons rather than at each synapse, the PS2 method improves hardware energy efficiency and ensures scalability. Furthermore, it improves energy and area efficiency by eliminating the additional circuit required by other techniques, such as the random walk (RW) method previously used which requires an additional circuit to frequently charge/discharge the membrane potential. Herein, the average firing rate equation for the S2M method is modified to suit the experimental conditions of this study. Through this numerical simulations, it is confirmed that the activation function of the PS2 method aligns with that of the S2M method and verified that the PS2 method can implement stochasticity for restricted Boltzmann machine (RBM) neurons. Experimental validation of the PS2 method, compared to the RW method, for Modified National Institute of Standards and Technology database (MNIST) training and inference on field‐programmable‐gate‐array‐implemented spiking RBM chips reveals promising results. In an MNIST 100‐handwritten digit experiment, the PS2 method exhibits on‐chip training accuracy (92%) comparable to that of the RW method (93%). Furthermore, in the energy consumption analysis, it is shown that the PS2 method reduces power consumption by 94.94% compared to the RW method, highlighting its enhanced power efficiency due to a reduced number of circuit elements. In the investigation into the impact of increasing the frequency at which random bits are generated, it is shown that the RW method experiences accuracy degradation even with slight increases, whereas the PS2 method maintains accuracy over significantly longer periods. This enables further power reduction by allowing for a longer period during random bit generation. In this study, a foundation is laid for maximizing the energy efficiency of spiking neural network processors by optimizing internal noise generation mechanisms. |
| format | Article |
| id | doaj-art-356bc63a5cd24ad891a0f0998239ca0e |
| institution | OA Journals |
| issn | 2640-4567 |
| language | English |
| publishDate | 2025-05-01 |
| publisher | Wiley |
| record_format | Article |
| series | Advanced Intelligent Systems |
| spelling | doaj-art-356bc63a5cd24ad891a0f0998239ca0e2025-08-20T01:55:22ZengWileyAdvanced Intelligent Systems2640-45672025-05-0175n/an/a10.1002/aisy.202400557Energy‐Efficient Hardware Implementation of Spiking‐Restricted Boltzmann Machines Using Pseudo‐Synaptic SamplingHyunwoo Kim0Suyeon Jang1Uicheol Shin2Masatoshi Ishii3Atsuya Okazaki4Megumi Ito5Akiyo Nomura6Kohji Hosokawa7Sungmin Lee8Matthew BrightSky9Sangbum Kim10Department of Material Science & Engineering IC Design Education Center (IDEC) Inter‐University Semiconductor Research Center Research Institute of Advanced Materials Seoul National University Seoul 08826 Republic of KoreaDepartment of Material Science & Engineering IC Design Education Center (IDEC) Inter‐University Semiconductor Research Center Research Institute of Advanced Materials Seoul National University Seoul 08826 Republic of KoreaDepartment of Material Science & Engineering IC Design Education Center (IDEC) Inter‐University Semiconductor Research Center Research Institute of Advanced Materials Seoul National University Seoul 08826 Republic of KoreaIBM Research‐Tokyo 19‐21 Nihonbashi‐Hakozakicho Chuo‐Ku Tokyo 13 103‐8510 JapanIBM Research‐Tokyo 19‐21 Nihonbashi‐Hakozakicho Chuo‐Ku Tokyo 13 103‐8510 JapanIBM Research‐Tokyo 19‐21 Nihonbashi‐Hakozakicho Chuo‐Ku Tokyo 13 103‐8510 JapanIBM Research‐Tokyo 19‐21 Nihonbashi‐Hakozakicho Chuo‐Ku Tokyo 13 103‐8510 JapanIBM Research‐Tokyo 19‐21 Nihonbashi‐Hakozakicho Chuo‐Ku Tokyo 13 103‐8510 JapanDepartment of Material Science & Engineering IC Design Education Center (IDEC) Inter‐University Semiconductor Research Center Research Institute of Advanced Materials Seoul National University Seoul 08826 Republic of KoreaIBM Thomas J. Watson Research Center 1101 Kitchawan Rd Yorktown Heights NY 10598 USADepartment of Material Science & Engineering IC Design Education Center (IDEC) Inter‐University Semiconductor Research Center Research Institute of Advanced Materials Seoul National University Seoul 08826 Republic of KoreaStochastic sampling is performed to reduce hardware energy consumption and prevent overfitting by reducing parameters, because not all data are required for learning. In this study, a new approach, pseudo‐synaptic sampling (PS2) method, which approximates the conventional synaptic sampling machine (S2M) method through a hardware‐friendly implementation while demonstrating superior efficiency, is introduced. By sampling in front of neurons rather than at each synapse, the PS2 method improves hardware energy efficiency and ensures scalability. Furthermore, it improves energy and area efficiency by eliminating the additional circuit required by other techniques, such as the random walk (RW) method previously used which requires an additional circuit to frequently charge/discharge the membrane potential. Herein, the average firing rate equation for the S2M method is modified to suit the experimental conditions of this study. Through this numerical simulations, it is confirmed that the activation function of the PS2 method aligns with that of the S2M method and verified that the PS2 method can implement stochasticity for restricted Boltzmann machine (RBM) neurons. Experimental validation of the PS2 method, compared to the RW method, for Modified National Institute of Standards and Technology database (MNIST) training and inference on field‐programmable‐gate‐array‐implemented spiking RBM chips reveals promising results. In an MNIST 100‐handwritten digit experiment, the PS2 method exhibits on‐chip training accuracy (92%) comparable to that of the RW method (93%). Furthermore, in the energy consumption analysis, it is shown that the PS2 method reduces power consumption by 94.94% compared to the RW method, highlighting its enhanced power efficiency due to a reduced number of circuit elements. In the investigation into the impact of increasing the frequency at which random bits are generated, it is shown that the RW method experiences accuracy degradation even with slight increases, whereas the PS2 method maintains accuracy over significantly longer periods. This enables further power reduction by allowing for a longer period during random bit generation. In this study, a foundation is laid for maximizing the energy efficiency of spiking neural network processors by optimizing internal noise generation mechanisms.https://doi.org/10.1002/aisy.202400557energy‐efficienciesin‐memory computingsneuromorphicspseudo‐synaptic samplingsrestricted Boltzmann machinesspiking neural networks |
| spellingShingle | Hyunwoo Kim Suyeon Jang Uicheol Shin Masatoshi Ishii Atsuya Okazaki Megumi Ito Akiyo Nomura Kohji Hosokawa Sungmin Lee Matthew BrightSky Sangbum Kim Energy‐Efficient Hardware Implementation of Spiking‐Restricted Boltzmann Machines Using Pseudo‐Synaptic Sampling Advanced Intelligent Systems energy‐efficiencies in‐memory computings neuromorphics pseudo‐synaptic samplings restricted Boltzmann machines spiking neural networks |
| title | Energy‐Efficient Hardware Implementation of Spiking‐Restricted Boltzmann Machines Using Pseudo‐Synaptic Sampling |
| title_full | Energy‐Efficient Hardware Implementation of Spiking‐Restricted Boltzmann Machines Using Pseudo‐Synaptic Sampling |
| title_fullStr | Energy‐Efficient Hardware Implementation of Spiking‐Restricted Boltzmann Machines Using Pseudo‐Synaptic Sampling |
| title_full_unstemmed | Energy‐Efficient Hardware Implementation of Spiking‐Restricted Boltzmann Machines Using Pseudo‐Synaptic Sampling |
| title_short | Energy‐Efficient Hardware Implementation of Spiking‐Restricted Boltzmann Machines Using Pseudo‐Synaptic Sampling |
| title_sort | energy efficient hardware implementation of spiking restricted boltzmann machines using pseudo synaptic sampling |
| topic | energy‐efficiencies in‐memory computings neuromorphics pseudo‐synaptic samplings restricted Boltzmann machines spiking neural networks |
| url | https://doi.org/10.1002/aisy.202400557 |
| work_keys_str_mv | AT hyunwookim energyefficienthardwareimplementationofspikingrestrictedboltzmannmachinesusingpseudosynapticsampling AT suyeonjang energyefficienthardwareimplementationofspikingrestrictedboltzmannmachinesusingpseudosynapticsampling AT uicheolshin energyefficienthardwareimplementationofspikingrestrictedboltzmannmachinesusingpseudosynapticsampling AT masatoshiishii energyefficienthardwareimplementationofspikingrestrictedboltzmannmachinesusingpseudosynapticsampling AT atsuyaokazaki energyefficienthardwareimplementationofspikingrestrictedboltzmannmachinesusingpseudosynapticsampling AT megumiito energyefficienthardwareimplementationofspikingrestrictedboltzmannmachinesusingpseudosynapticsampling AT akiyonomura energyefficienthardwareimplementationofspikingrestrictedboltzmannmachinesusingpseudosynapticsampling AT kohjihosokawa energyefficienthardwareimplementationofspikingrestrictedboltzmannmachinesusingpseudosynapticsampling AT sungminlee energyefficienthardwareimplementationofspikingrestrictedboltzmannmachinesusingpseudosynapticsampling AT matthewbrightsky energyefficienthardwareimplementationofspikingrestrictedboltzmannmachinesusingpseudosynapticsampling AT sangbumkim energyefficienthardwareimplementationofspikingrestrictedboltzmannmachinesusingpseudosynapticsampling |