A bioinspired in-materia analog photoelectronic reservoir computing for human action processing
Abstract Current computer vision is data-intensive and faces bottlenecks in shrinking computational costs. Incorporating physics into a bioinspired visual system is promising to offer unprecedented energy efficiency, while the mismatch between physical dynamics and bioinspired algorithms makes the p...
Saved in:
| Main Authors: | , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-03-01
|
| Series: | Nature Communications |
| Online Access: | https://doi.org/10.1038/s41467-025-56899-3 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850027945800761344 |
|---|---|
| author | Hangyuan Cui Yu Xiao Yang Yang Mengjiao Pei Shuo Ke Xiao Fang Lesheng Qiao Kailu Shi Haotian Long Weigao Xu Pingqiang Cai Peng Lin Yi Shi Qing Wan Changjin Wan |
| author_facet | Hangyuan Cui Yu Xiao Yang Yang Mengjiao Pei Shuo Ke Xiao Fang Lesheng Qiao Kailu Shi Haotian Long Weigao Xu Pingqiang Cai Peng Lin Yi Shi Qing Wan Changjin Wan |
| author_sort | Hangyuan Cui |
| collection | DOAJ |
| description | Abstract Current computer vision is data-intensive and faces bottlenecks in shrinking computational costs. Incorporating physics into a bioinspired visual system is promising to offer unprecedented energy efficiency, while the mismatch between physical dynamics and bioinspired algorithms makes the processing of real-world samples rather challenging. Here, we report a bioinspired in-materia analogue photoelectronic reservoir computing for dynamic vision processing. Such system is built based on InGaZnO photoelectronic synaptic transistors as the reservoir and a TaOX-based memristor array as the output layer. A receptive field inspired encoding scheme is implemented, simplifying the feature extraction process. High recognition accuracies (>90%) on four motion recognition datasets are achieved based on such system. Furthermore, falling behaviors recognition is also verified by our system with low energy consumption for processing per action (~45.78 μJ) which outperforms most previous reports on human action processing. Our results are of profound potential for advancing computer vision based on neuromorphic electronics. |
| format | Article |
| id | doaj-art-35cf7ddf902142c8b7edb2708362a182 |
| institution | DOAJ |
| issn | 2041-1723 |
| language | English |
| publishDate | 2025-03-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Nature Communications |
| spelling | doaj-art-35cf7ddf902142c8b7edb2708362a1822025-08-20T02:59:58ZengNature PortfolioNature Communications2041-17232025-03-0116111110.1038/s41467-025-56899-3A bioinspired in-materia analog photoelectronic reservoir computing for human action processingHangyuan Cui0Yu Xiao1Yang Yang2Mengjiao Pei3Shuo Ke4Xiao Fang5Lesheng Qiao6Kailu Shi7Haotian Long8Weigao Xu9Pingqiang Cai10Peng Lin11Yi Shi12Qing Wan13Changjin Wan14School of Electronic Science and Engineering, National Laboratory of Solid-State Microstructures, Nanjing UniversityCollege of Computer Science and Technology, State Key Laboratory of Brain Machine Intelligence, Zhejiang UniversitySchool of Electronic Science and Engineering, National Laboratory of Solid-State Microstructures, Nanjing UniversitySchool of Electronic Science and Engineering, National Laboratory of Solid-State Microstructures, Nanjing UniversitySchool of Electronic Science and Engineering, National Laboratory of Solid-State Microstructures, Nanjing UniversitySchool of Electronic Science and Engineering, National Laboratory of Solid-State Microstructures, Nanjing UniversitySchool of Electronic Science and Engineering, National Laboratory of Solid-State Microstructures, Nanjing UniversitySchool of Electronic Science and Engineering, National Laboratory of Solid-State Microstructures, Nanjing UniversitySchool of Electronic Science and Engineering, National Laboratory of Solid-State Microstructures, Nanjing UniversityKey Laboratory of Mesoscopic Chemistry, School of Chemistry and Chemical Engineering, Nanjing UniversityJiangsu Key Laboratory of Molecular Medicine, Medical School, Nanjing UniversityCollege of Computer Science and Technology, State Key Laboratory of Brain Machine Intelligence, Zhejiang UniversitySchool of Electronic Science and Engineering, National Laboratory of Solid-State Microstructures, Nanjing UniversityYongjiang Laboratory (Y-LAB)School of Electronic Science and Engineering, National Laboratory of Solid-State Microstructures, Nanjing UniversityAbstract Current computer vision is data-intensive and faces bottlenecks in shrinking computational costs. Incorporating physics into a bioinspired visual system is promising to offer unprecedented energy efficiency, while the mismatch between physical dynamics and bioinspired algorithms makes the processing of real-world samples rather challenging. Here, we report a bioinspired in-materia analogue photoelectronic reservoir computing for dynamic vision processing. Such system is built based on InGaZnO photoelectronic synaptic transistors as the reservoir and a TaOX-based memristor array as the output layer. A receptive field inspired encoding scheme is implemented, simplifying the feature extraction process. High recognition accuracies (>90%) on four motion recognition datasets are achieved based on such system. Furthermore, falling behaviors recognition is also verified by our system with low energy consumption for processing per action (~45.78 μJ) which outperforms most previous reports on human action processing. Our results are of profound potential for advancing computer vision based on neuromorphic electronics.https://doi.org/10.1038/s41467-025-56899-3 |
| spellingShingle | Hangyuan Cui Yu Xiao Yang Yang Mengjiao Pei Shuo Ke Xiao Fang Lesheng Qiao Kailu Shi Haotian Long Weigao Xu Pingqiang Cai Peng Lin Yi Shi Qing Wan Changjin Wan A bioinspired in-materia analog photoelectronic reservoir computing for human action processing Nature Communications |
| title | A bioinspired in-materia analog photoelectronic reservoir computing for human action processing |
| title_full | A bioinspired in-materia analog photoelectronic reservoir computing for human action processing |
| title_fullStr | A bioinspired in-materia analog photoelectronic reservoir computing for human action processing |
| title_full_unstemmed | A bioinspired in-materia analog photoelectronic reservoir computing for human action processing |
| title_short | A bioinspired in-materia analog photoelectronic reservoir computing for human action processing |
| title_sort | bioinspired in materia analog photoelectronic reservoir computing for human action processing |
| url | https://doi.org/10.1038/s41467-025-56899-3 |
| work_keys_str_mv | AT hangyuancui abioinspiredinmateriaanalogphotoelectronicreservoircomputingforhumanactionprocessing AT yuxiao abioinspiredinmateriaanalogphotoelectronicreservoircomputingforhumanactionprocessing AT yangyang abioinspiredinmateriaanalogphotoelectronicreservoircomputingforhumanactionprocessing AT mengjiaopei abioinspiredinmateriaanalogphotoelectronicreservoircomputingforhumanactionprocessing AT shuoke abioinspiredinmateriaanalogphotoelectronicreservoircomputingforhumanactionprocessing AT xiaofang abioinspiredinmateriaanalogphotoelectronicreservoircomputingforhumanactionprocessing AT leshengqiao abioinspiredinmateriaanalogphotoelectronicreservoircomputingforhumanactionprocessing AT kailushi abioinspiredinmateriaanalogphotoelectronicreservoircomputingforhumanactionprocessing AT haotianlong abioinspiredinmateriaanalogphotoelectronicreservoircomputingforhumanactionprocessing AT weigaoxu abioinspiredinmateriaanalogphotoelectronicreservoircomputingforhumanactionprocessing AT pingqiangcai abioinspiredinmateriaanalogphotoelectronicreservoircomputingforhumanactionprocessing AT penglin abioinspiredinmateriaanalogphotoelectronicreservoircomputingforhumanactionprocessing AT yishi abioinspiredinmateriaanalogphotoelectronicreservoircomputingforhumanactionprocessing AT qingwan abioinspiredinmateriaanalogphotoelectronicreservoircomputingforhumanactionprocessing AT changjinwan abioinspiredinmateriaanalogphotoelectronicreservoircomputingforhumanactionprocessing AT hangyuancui bioinspiredinmateriaanalogphotoelectronicreservoircomputingforhumanactionprocessing AT yuxiao bioinspiredinmateriaanalogphotoelectronicreservoircomputingforhumanactionprocessing AT yangyang bioinspiredinmateriaanalogphotoelectronicreservoircomputingforhumanactionprocessing AT mengjiaopei bioinspiredinmateriaanalogphotoelectronicreservoircomputingforhumanactionprocessing AT shuoke bioinspiredinmateriaanalogphotoelectronicreservoircomputingforhumanactionprocessing AT xiaofang bioinspiredinmateriaanalogphotoelectronicreservoircomputingforhumanactionprocessing AT leshengqiao bioinspiredinmateriaanalogphotoelectronicreservoircomputingforhumanactionprocessing AT kailushi bioinspiredinmateriaanalogphotoelectronicreservoircomputingforhumanactionprocessing AT haotianlong bioinspiredinmateriaanalogphotoelectronicreservoircomputingforhumanactionprocessing AT weigaoxu bioinspiredinmateriaanalogphotoelectronicreservoircomputingforhumanactionprocessing AT pingqiangcai bioinspiredinmateriaanalogphotoelectronicreservoircomputingforhumanactionprocessing AT penglin bioinspiredinmateriaanalogphotoelectronicreservoircomputingforhumanactionprocessing AT yishi bioinspiredinmateriaanalogphotoelectronicreservoircomputingforhumanactionprocessing AT qingwan bioinspiredinmateriaanalogphotoelectronicreservoircomputingforhumanactionprocessing AT changjinwan bioinspiredinmateriaanalogphotoelectronicreservoircomputingforhumanactionprocessing |