RN‐Net: Reservoir Nodes‐Enabled Neuromorphic Vision Sensing Network
Neuromorphic computing systems promise high energy efficiency and low latency. In particular, when integrated with neuromorphic sensors, they can be used to produce intelligent systems for a broad range of applications. An event‐based camera is such a neuromorphic sensor, inspired by the sparse and...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Wiley
2024-12-01
|
| Series: | Advanced Intelligent Systems |
| Subjects: | |
| Online Access: | https://doi.org/10.1002/aisy.202400265 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850133131303059456 |
|---|---|
| author | Sangmnin Yoo Eric Yeu‐Jer Lee Ziyu Wang Xinxin Wang Wei D. Lu |
| author_facet | Sangmnin Yoo Eric Yeu‐Jer Lee Ziyu Wang Xinxin Wang Wei D. Lu |
| author_sort | Sangmnin Yoo |
| collection | DOAJ |
| description | Neuromorphic computing systems promise high energy efficiency and low latency. In particular, when integrated with neuromorphic sensors, they can be used to produce intelligent systems for a broad range of applications. An event‐based camera is such a neuromorphic sensor, inspired by the sparse and asynchronous spike representation of the biological visual system. However, processing the event data requires either using expensive feature descriptors to transform spikes into frames, or using spiking neural networks (SNNs) that are expensive to train. In this work, a neural network architecture is proposed, reservoir nodes‐enabled neuromorphic vision sensing network (RN‐Net), based on dynamic temporal encoding by on‐sensor reservoirs and simple deep neural network (DNN) blocks. The reservoir nodes enable efficient temporal processing of asynchronous events by leveraging the native dynamics of the node devices, while the DNN blocks enable spatial feature processing. Combining these blocks in a hierarchical structure, the RN‐Net offers efficient processing for both local and global spatiotemporal features. RN‐Net executes dynamic vision tasks created by event‐based cameras at the highest accuracy reported to date at one order of magnitude smaller network size. The use of simple DNN and standard backpropagation‐based training rules further reduces implementation and training costs. |
| format | Article |
| id | doaj-art-b49aa02b9add4e6e8fbe455abadbcccd |
| institution | OA Journals |
| issn | 2640-4567 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | Wiley |
| record_format | Article |
| series | Advanced Intelligent Systems |
| spelling | doaj-art-b49aa02b9add4e6e8fbe455abadbcccd2025-08-20T02:32:04ZengWileyAdvanced Intelligent Systems2640-45672024-12-01612n/an/a10.1002/aisy.202400265RN‐Net: Reservoir Nodes‐Enabled Neuromorphic Vision Sensing NetworkSangmnin Yoo0Eric Yeu‐Jer Lee1Ziyu Wang2Xinxin Wang3Wei D. Lu4Department of Electrical Engineering and Computer Science University of Michigan Ann Arbor 48109 MI USADepartment of Electrical Engineering and Computer Science University of Michigan Ann Arbor 48109 MI USADepartment of Electrical Engineering and Computer Science University of Michigan Ann Arbor 48109 MI USADepartment of Electrical Engineering and Computer Science University of Michigan Ann Arbor 48109 MI USADepartment of Electrical Engineering and Computer Science University of Michigan Ann Arbor 48109 MI USANeuromorphic computing systems promise high energy efficiency and low latency. In particular, when integrated with neuromorphic sensors, they can be used to produce intelligent systems for a broad range of applications. An event‐based camera is such a neuromorphic sensor, inspired by the sparse and asynchronous spike representation of the biological visual system. However, processing the event data requires either using expensive feature descriptors to transform spikes into frames, or using spiking neural networks (SNNs) that are expensive to train. In this work, a neural network architecture is proposed, reservoir nodes‐enabled neuromorphic vision sensing network (RN‐Net), based on dynamic temporal encoding by on‐sensor reservoirs and simple deep neural network (DNN) blocks. The reservoir nodes enable efficient temporal processing of asynchronous events by leveraging the native dynamics of the node devices, while the DNN blocks enable spatial feature processing. Combining these blocks in a hierarchical structure, the RN‐Net offers efficient processing for both local and global spatiotemporal features. RN‐Net executes dynamic vision tasks created by event‐based cameras at the highest accuracy reported to date at one order of magnitude smaller network size. The use of simple DNN and standard backpropagation‐based training rules further reduces implementation and training costs.https://doi.org/10.1002/aisy.202400265event‐based cameramemristorneuromorphicreservoir computingSNN |
| spellingShingle | Sangmnin Yoo Eric Yeu‐Jer Lee Ziyu Wang Xinxin Wang Wei D. Lu RN‐Net: Reservoir Nodes‐Enabled Neuromorphic Vision Sensing Network Advanced Intelligent Systems event‐based camera memristor neuromorphic reservoir computing SNN |
| title | RN‐Net: Reservoir Nodes‐Enabled Neuromorphic Vision Sensing Network |
| title_full | RN‐Net: Reservoir Nodes‐Enabled Neuromorphic Vision Sensing Network |
| title_fullStr | RN‐Net: Reservoir Nodes‐Enabled Neuromorphic Vision Sensing Network |
| title_full_unstemmed | RN‐Net: Reservoir Nodes‐Enabled Neuromorphic Vision Sensing Network |
| title_short | RN‐Net: Reservoir Nodes‐Enabled Neuromorphic Vision Sensing Network |
| title_sort | rn net reservoir nodes enabled neuromorphic vision sensing network |
| topic | event‐based camera memristor neuromorphic reservoir computing SNN |
| url | https://doi.org/10.1002/aisy.202400265 |
| work_keys_str_mv | AT sangmninyoo rnnetreservoirnodesenabledneuromorphicvisionsensingnetwork AT ericyeujerlee rnnetreservoirnodesenabledneuromorphicvisionsensingnetwork AT ziyuwang rnnetreservoirnodesenabledneuromorphicvisionsensingnetwork AT xinxinwang rnnetreservoirnodesenabledneuromorphicvisionsensingnetwork AT weidlu rnnetreservoirnodesenabledneuromorphicvisionsensingnetwork |