Random resistive memory-based deep extreme point learning machine for unified visual processing
Abstract Visual sensors, including 3D light detection and ranging, neuromorphic dynamic vision sensor, and conventional frame cameras, are increasingly integrated into edge-side intelligent machines. However, their data are heterogeneous, causing complexity in system development. Moreover, conventio...
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-01-01
|
| Series: | Nature Communications |
| Online Access: | https://doi.org/10.1038/s41467-025-56079-3 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832585586147328000 |
|---|---|
| author | Shaocong Wang Yizhao Gao Yi Li Woyu Zhang Yifei Yu Bo Wang Ning Lin Hegan Chen Yue Zhang Yang Jiang Dingchen Wang Jia Chen Peng Dai Hao Jiang Peng Lin Xumeng Zhang Xiaojuan Qi Xiaoxin Xu Hayden So Zhongrui Wang Dashan Shang Qi Liu Kwang-Ting Cheng Ming Liu |
| author_facet | Shaocong Wang Yizhao Gao Yi Li Woyu Zhang Yifei Yu Bo Wang Ning Lin Hegan Chen Yue Zhang Yang Jiang Dingchen Wang Jia Chen Peng Dai Hao Jiang Peng Lin Xumeng Zhang Xiaojuan Qi Xiaoxin Xu Hayden So Zhongrui Wang Dashan Shang Qi Liu Kwang-Ting Cheng Ming Liu |
| author_sort | Shaocong Wang |
| collection | DOAJ |
| description | Abstract Visual sensors, including 3D light detection and ranging, neuromorphic dynamic vision sensor, and conventional frame cameras, are increasingly integrated into edge-side intelligent machines. However, their data are heterogeneous, causing complexity in system development. Moreover, conventional digital hardware is constrained by von Neumann bottleneck and the physical limit of transistor scaling. The computational demands of training ever-growing models further exacerbate these challenges. We propose a hardware-software co-designed random resistive memory-based deep extreme point learning machine. Data-wise, the multi-sensory data are unified as point set and processed universally. Software-wise, most weights are exempted from training. Hardware-wise, nanoscale resistive memory enables collocation of memory and processing, and leverages the inherent programming stochasticity for generating random weights. The co-design system is validated on 3D segmentation (ShapeNet), event recognition (DVS128 Gesture), and image classification (Fashion-MNIST) tasks, achieving accuracy comparable to conventional systems while delivering 6.78 × /21.04 × /15.79 × energy efficiency improvements and 70.12%/89.46%/85.61% training cost reductions. |
| format | Article |
| id | doaj-art-ac7782ca38dc482c86da7ff73665bad1 |
| institution | Kabale University |
| issn | 2041-1723 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Nature Communications |
| spelling | doaj-art-ac7782ca38dc482c86da7ff73665bad12025-01-26T12:41:04ZengNature PortfolioNature Communications2041-17232025-01-0116111110.1038/s41467-025-56079-3Random resistive memory-based deep extreme point learning machine for unified visual processingShaocong Wang0Yizhao Gao1Yi Li2Woyu Zhang3Yifei Yu4Bo Wang5Ning Lin6Hegan Chen7Yue Zhang8Yang Jiang9Dingchen Wang10Jia Chen11Peng Dai12Hao Jiang13Peng Lin14Xumeng Zhang15Xiaojuan Qi16Xiaoxin Xu17Hayden So18Zhongrui Wang19Dashan Shang20Qi Liu21Kwang-Ting Cheng22Ming Liu23Department of Electrical and Electronic Engineering, The University of Hong KongDepartment of Electrical and Electronic Engineering, The University of Hong KongDepartment of Electrical and Electronic Engineering, The University of Hong KongKey Lab of Fabrication Technologies for Integrated Circuits Institute of Microelectronics, Chinese Academy of SciencesDepartment of Electrical and Electronic Engineering, The University of Hong KongDepartment of Electrical and Electronic Engineering, The University of Hong KongDepartment of Electrical and Electronic Engineering, The University of Hong KongDepartment of Electrical and Electronic Engineering, The University of Hong KongDepartment of Electrical and Electronic Engineering, The University of Hong KongDepartment of Electrical and Electronic Engineering, The University of Hong KongDepartment of Electrical and Electronic Engineering, The University of Hong KongDepartment of Electrical and Electronic Engineering, The University of Hong KongDepartment of Electrical and Electronic Engineering, The University of Hong KongFrontier Institute of Chip and System, Fudan UniversityCollege of Computer Science and Technology, Zhejiang UniversityFrontier Institute of Chip and System, Fudan UniversityDepartment of Electrical and Electronic Engineering, The University of Hong KongKey Lab of Fabrication Technologies for Integrated Circuits Institute of Microelectronics, Chinese Academy of SciencesDepartment of Electrical and Electronic Engineering, The University of Hong KongSchool of Microelectronics, Southern University of Science and TechnologyKey Lab of Fabrication Technologies for Integrated Circuits Institute of Microelectronics, Chinese Academy of SciencesKey Lab of Fabrication Technologies for Integrated Circuits Institute of Microelectronics, Chinese Academy of SciencesACCESS—AI Chip Center for Emerging Smart Systems, InnoHK Centers, Hong Kong Science ParkKey Lab of Fabrication Technologies for Integrated Circuits Institute of Microelectronics, Chinese Academy of SciencesAbstract Visual sensors, including 3D light detection and ranging, neuromorphic dynamic vision sensor, and conventional frame cameras, are increasingly integrated into edge-side intelligent machines. However, their data are heterogeneous, causing complexity in system development. Moreover, conventional digital hardware is constrained by von Neumann bottleneck and the physical limit of transistor scaling. The computational demands of training ever-growing models further exacerbate these challenges. We propose a hardware-software co-designed random resistive memory-based deep extreme point learning machine. Data-wise, the multi-sensory data are unified as point set and processed universally. Software-wise, most weights are exempted from training. Hardware-wise, nanoscale resistive memory enables collocation of memory and processing, and leverages the inherent programming stochasticity for generating random weights. The co-design system is validated on 3D segmentation (ShapeNet), event recognition (DVS128 Gesture), and image classification (Fashion-MNIST) tasks, achieving accuracy comparable to conventional systems while delivering 6.78 × /21.04 × /15.79 × energy efficiency improvements and 70.12%/89.46%/85.61% training cost reductions.https://doi.org/10.1038/s41467-025-56079-3 |
| spellingShingle | Shaocong Wang Yizhao Gao Yi Li Woyu Zhang Yifei Yu Bo Wang Ning Lin Hegan Chen Yue Zhang Yang Jiang Dingchen Wang Jia Chen Peng Dai Hao Jiang Peng Lin Xumeng Zhang Xiaojuan Qi Xiaoxin Xu Hayden So Zhongrui Wang Dashan Shang Qi Liu Kwang-Ting Cheng Ming Liu Random resistive memory-based deep extreme point learning machine for unified visual processing Nature Communications |
| title | Random resistive memory-based deep extreme point learning machine for unified visual processing |
| title_full | Random resistive memory-based deep extreme point learning machine for unified visual processing |
| title_fullStr | Random resistive memory-based deep extreme point learning machine for unified visual processing |
| title_full_unstemmed | Random resistive memory-based deep extreme point learning machine for unified visual processing |
| title_short | Random resistive memory-based deep extreme point learning machine for unified visual processing |
| title_sort | random resistive memory based deep extreme point learning machine for unified visual processing |
| url | https://doi.org/10.1038/s41467-025-56079-3 |
| work_keys_str_mv | AT shaocongwang randomresistivememorybaseddeepextremepointlearningmachineforunifiedvisualprocessing AT yizhaogao randomresistivememorybaseddeepextremepointlearningmachineforunifiedvisualprocessing AT yili randomresistivememorybaseddeepextremepointlearningmachineforunifiedvisualprocessing AT woyuzhang randomresistivememorybaseddeepextremepointlearningmachineforunifiedvisualprocessing AT yifeiyu randomresistivememorybaseddeepextremepointlearningmachineforunifiedvisualprocessing AT bowang randomresistivememorybaseddeepextremepointlearningmachineforunifiedvisualprocessing AT ninglin randomresistivememorybaseddeepextremepointlearningmachineforunifiedvisualprocessing AT heganchen randomresistivememorybaseddeepextremepointlearningmachineforunifiedvisualprocessing AT yuezhang randomresistivememorybaseddeepextremepointlearningmachineforunifiedvisualprocessing AT yangjiang randomresistivememorybaseddeepextremepointlearningmachineforunifiedvisualprocessing AT dingchenwang randomresistivememorybaseddeepextremepointlearningmachineforunifiedvisualprocessing AT jiachen randomresistivememorybaseddeepextremepointlearningmachineforunifiedvisualprocessing AT pengdai randomresistivememorybaseddeepextremepointlearningmachineforunifiedvisualprocessing AT haojiang randomresistivememorybaseddeepextremepointlearningmachineforunifiedvisualprocessing AT penglin randomresistivememorybaseddeepextremepointlearningmachineforunifiedvisualprocessing AT xumengzhang randomresistivememorybaseddeepextremepointlearningmachineforunifiedvisualprocessing AT xiaojuanqi randomresistivememorybaseddeepextremepointlearningmachineforunifiedvisualprocessing AT xiaoxinxu randomresistivememorybaseddeepextremepointlearningmachineforunifiedvisualprocessing AT haydenso randomresistivememorybaseddeepextremepointlearningmachineforunifiedvisualprocessing AT zhongruiwang randomresistivememorybaseddeepextremepointlearningmachineforunifiedvisualprocessing AT dashanshang randomresistivememorybaseddeepextremepointlearningmachineforunifiedvisualprocessing AT qiliu randomresistivememorybaseddeepextremepointlearningmachineforunifiedvisualprocessing AT kwangtingcheng randomresistivememorybaseddeepextremepointlearningmachineforunifiedvisualprocessing AT mingliu randomresistivememorybaseddeepextremepointlearningmachineforunifiedvisualprocessing |