Memristive Bellman solver for decision-making
Abstract The Bellman equation, with a resource-consuming solving process, plays a fundamental role in formulating and solving dynamic optimization problems. The realization of the Bellman solver with memristive computing-in-memory (MCIM) technology, is significant for implementing efficient dynamic...
Saved in:
| Main Authors: | , , , , , , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-05-01
|
| Series: | Nature Communications |
| Online Access: | https://doi.org/10.1038/s41467-025-60085-w |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849704288270417920 |
|---|---|
| author | Zhe Feng Zuheng Wu Jianxun Zou Lingli Cheng Xiaolong Zhao Xumeng Zhang Jian Lu Cong Wang Yilin Wang Haochen Wang Wenbin Guo Zhibin Qian Yunlai Zhu Zuyu Xu Yuehua Dai Qi Liu |
| author_facet | Zhe Feng Zuheng Wu Jianxun Zou Lingli Cheng Xiaolong Zhao Xumeng Zhang Jian Lu Cong Wang Yilin Wang Haochen Wang Wenbin Guo Zhibin Qian Yunlai Zhu Zuyu Xu Yuehua Dai Qi Liu |
| author_sort | Zhe Feng |
| collection | DOAJ |
| description | Abstract The Bellman equation, with a resource-consuming solving process, plays a fundamental role in formulating and solving dynamic optimization problems. The realization of the Bellman solver with memristive computing-in-memory (MCIM) technology, is significant for implementing efficient dynamic decision-making. However, the iterative nature of the Bellman equation solving process poses a challenge for efficient implementation on MCIM systems, which excel at vector-matrix multiplication (VMM) operations but are less suited for iterative algorithms. In this work, by incorporating the temporal dimension and transforming the solution into recurrent dot product operations, a memristive Bellman solver (MBS) is proposed, facilitating the implementation of the Bellman equation solving process with efficient MCIM technology. The MBS effectively reduces the iteration numbers and which further enhanced by approximated solutions leveraging memristor noise. Finally, the path planning tasks are used to verify the feasibility of the proposed MBS. The theoretical derivation and experimental results demonstrate that the MBS effectively reduces the iteration cycles, facilitating the solving efficiency. This work could be a sound of choice for developing high-efficiency decision-making systems. |
| format | Article |
| id | doaj-art-92e34997b5f04bd9b925d93f8a9d565b |
| institution | DOAJ |
| issn | 2041-1723 |
| language | English |
| publishDate | 2025-05-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Nature Communications |
| spelling | doaj-art-92e34997b5f04bd9b925d93f8a9d565b2025-08-20T03:16:47ZengNature PortfolioNature Communications2041-17232025-05-0116111110.1038/s41467-025-60085-wMemristive Bellman solver for decision-makingZhe Feng0Zuheng Wu1Jianxun Zou2Lingli Cheng3Xiaolong Zhao4Xumeng Zhang5Jian Lu6Cong Wang7Yilin Wang8Haochen Wang9Wenbin Guo10Zhibin Qian11Yunlai Zhu12Zuyu Xu13Yuehua Dai14Qi Liu15School of Integrated Circuits, Anhui UniversitySchool of Integrated Circuits, Anhui UniversitySchool of Integrated Circuits, Anhui UniversityFrontier Institute of Chip and System, Fudan UniversitySchool of Microelectronics, University of Science and Technology of ChinaFrontier Institute of Chip and System, Fudan UniversityResearch Center for Intelligent Computing Hardware, Zhejiang LaboratoryInstitute of Brain-inspired Intelligence, National Laboratory of Solid State Microstructures, School of Physics, Collaborative Innovation Center of Advanced Microstructures, Nanjing UniversitySchool of Microelectronics, University of Science and Technology of ChinaSchool of Integrated Circuits, Anhui UniversitySchool of Integrated Circuits, Anhui UniversitySchool of Integrated Circuits, Anhui UniversitySchool of Integrated Circuits, Anhui UniversitySchool of Integrated Circuits, Anhui UniversitySchool of Integrated Circuits, Anhui UniversityFrontier Institute of Chip and System, Fudan UniversityAbstract The Bellman equation, with a resource-consuming solving process, plays a fundamental role in formulating and solving dynamic optimization problems. The realization of the Bellman solver with memristive computing-in-memory (MCIM) technology, is significant for implementing efficient dynamic decision-making. However, the iterative nature of the Bellman equation solving process poses a challenge for efficient implementation on MCIM systems, which excel at vector-matrix multiplication (VMM) operations but are less suited for iterative algorithms. In this work, by incorporating the temporal dimension and transforming the solution into recurrent dot product operations, a memristive Bellman solver (MBS) is proposed, facilitating the implementation of the Bellman equation solving process with efficient MCIM technology. The MBS effectively reduces the iteration numbers and which further enhanced by approximated solutions leveraging memristor noise. Finally, the path planning tasks are used to verify the feasibility of the proposed MBS. The theoretical derivation and experimental results demonstrate that the MBS effectively reduces the iteration cycles, facilitating the solving efficiency. This work could be a sound of choice for developing high-efficiency decision-making systems.https://doi.org/10.1038/s41467-025-60085-w |
| spellingShingle | Zhe Feng Zuheng Wu Jianxun Zou Lingli Cheng Xiaolong Zhao Xumeng Zhang Jian Lu Cong Wang Yilin Wang Haochen Wang Wenbin Guo Zhibin Qian Yunlai Zhu Zuyu Xu Yuehua Dai Qi Liu Memristive Bellman solver for decision-making Nature Communications |
| title | Memristive Bellman solver for decision-making |
| title_full | Memristive Bellman solver for decision-making |
| title_fullStr | Memristive Bellman solver for decision-making |
| title_full_unstemmed | Memristive Bellman solver for decision-making |
| title_short | Memristive Bellman solver for decision-making |
| title_sort | memristive bellman solver for decision making |
| url | https://doi.org/10.1038/s41467-025-60085-w |
| work_keys_str_mv | AT zhefeng memristivebellmansolverfordecisionmaking AT zuhengwu memristivebellmansolverfordecisionmaking AT jianxunzou memristivebellmansolverfordecisionmaking AT linglicheng memristivebellmansolverfordecisionmaking AT xiaolongzhao memristivebellmansolverfordecisionmaking AT xumengzhang memristivebellmansolverfordecisionmaking AT jianlu memristivebellmansolverfordecisionmaking AT congwang memristivebellmansolverfordecisionmaking AT yilinwang memristivebellmansolverfordecisionmaking AT haochenwang memristivebellmansolverfordecisionmaking AT wenbinguo memristivebellmansolverfordecisionmaking AT zhibinqian memristivebellmansolverfordecisionmaking AT yunlaizhu memristivebellmansolverfordecisionmaking AT zuyuxu memristivebellmansolverfordecisionmaking AT yuehuadai memristivebellmansolverfordecisionmaking AT qiliu memristivebellmansolverfordecisionmaking |