A scalable reinforcement learning framework inspired by hippocampal memory mechanisms for efficient contextual and sequential decision making
Abstract Efficient decision-making in context-dependent, sequential tasks remains a fundamental challenge in reinforcement learning (RL). Inspired by the function of the brain’s hippocampal system, we introduce Hippocampal-Augmented Memory Integration (HAMI), a biologically inspired memory-based RL...
Saved in:
| Main Authors: | , , , , , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Nature Portfolio
2025-07-01
|
| Series: | Scientific Reports |
| Online Access: | https://doi.org/10.1038/s41598-025-10586-x |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849342317405667328 |
|---|---|
| author | Hamed Poursiami Ayana Moshruba Keiland W. Cooper Derek Gobin Md Abdullah-Al Kaiser Ankur Singh Rouhan Noor Babak Shahbaba Akhilesh Jaiswal Norbert J. Fortin Maryam Parsa |
| author_facet | Hamed Poursiami Ayana Moshruba Keiland W. Cooper Derek Gobin Md Abdullah-Al Kaiser Ankur Singh Rouhan Noor Babak Shahbaba Akhilesh Jaiswal Norbert J. Fortin Maryam Parsa |
| author_sort | Hamed Poursiami |
| collection | DOAJ |
| description | Abstract Efficient decision-making in context-dependent, sequential tasks remains a fundamental challenge in reinforcement learning (RL). Inspired by the function of the brain’s hippocampal system, we introduce Hippocampal-Augmented Memory Integration (HAMI), a biologically inspired memory-based RL framework that leverages symbolic indexing, hierarchical memory refinement, and structured episodic retrieval to enhance both learning efficiency and adaptability. We also propose Hierarchical Contextual Sequences (HiCoS), a structured RL environment grounded in neuroscience studies on episodic and sequence memory and context-driven decision-making, which serves as a controlled testbed for evaluating biologically inspired memory-based decision-making systems. Our experimental results demonstrate that HAMI achieves high decision accuracy and improved sample efficiency while maintaining low memory utilization. HAMI’s architecture exhibits significantly lower inference latency than baseline memory-based methods, and its structured retrieval is well-suited for further hardware acceleration with non-volatile memory (NVM)-based content-addressable memory (CAM). By integrating biologically inspired memory mechanisms with structured symbolic representations, HAMI provides a scalable and efficient memory-based RL framework for tackling context-dependent sequential tasks. |
| format | Article |
| id | doaj-art-bf8cc803b0c2402f93c2b190cc22891c |
| institution | Kabale University |
| issn | 2045-2322 |
| language | English |
| publishDate | 2025-07-01 |
| publisher | Nature Portfolio |
| record_format | Article |
| series | Scientific Reports |
| spelling | doaj-art-bf8cc803b0c2402f93c2b190cc22891c2025-08-20T03:43:26ZengNature PortfolioScientific Reports2045-23222025-07-0115111610.1038/s41598-025-10586-xA scalable reinforcement learning framework inspired by hippocampal memory mechanisms for efficient contextual and sequential decision makingHamed Poursiami0Ayana Moshruba1Keiland W. Cooper2Derek Gobin3Md Abdullah-Al Kaiser4Ankur Singh5Rouhan Noor6Babak Shahbaba7Akhilesh Jaiswal8Norbert J. Fortin9Maryam Parsa10Department of Electrical and Computer Engineering, George Mason UniversityDepartment of Electrical and Computer Engineering, George Mason UniversityCenter for the Neurobiology of Learning and Memory, UC IrvineDepartment of Electrical and Computer Engineering, George Mason UniversityDepartment of Electrical and Computer Engineering, University of Wisconsin MadisonDepartment of Electrical and Computer Engineering, University of Wisconsin MadisonDepartment of Electrical and Computer Engineering, University of Wisconsin MadisonDepartment of Statistics, UC IrvineDepartment of Electrical and Computer Engineering, University of Wisconsin MadisonCenter for the Neurobiology of Learning and Memory, UC IrvineDepartment of Electrical and Computer Engineering, George Mason UniversityAbstract Efficient decision-making in context-dependent, sequential tasks remains a fundamental challenge in reinforcement learning (RL). Inspired by the function of the brain’s hippocampal system, we introduce Hippocampal-Augmented Memory Integration (HAMI), a biologically inspired memory-based RL framework that leverages symbolic indexing, hierarchical memory refinement, and structured episodic retrieval to enhance both learning efficiency and adaptability. We also propose Hierarchical Contextual Sequences (HiCoS), a structured RL environment grounded in neuroscience studies on episodic and sequence memory and context-driven decision-making, which serves as a controlled testbed for evaluating biologically inspired memory-based decision-making systems. Our experimental results demonstrate that HAMI achieves high decision accuracy and improved sample efficiency while maintaining low memory utilization. HAMI’s architecture exhibits significantly lower inference latency than baseline memory-based methods, and its structured retrieval is well-suited for further hardware acceleration with non-volatile memory (NVM)-based content-addressable memory (CAM). By integrating biologically inspired memory mechanisms with structured symbolic representations, HAMI provides a scalable and efficient memory-based RL framework for tackling context-dependent sequential tasks.https://doi.org/10.1038/s41598-025-10586-x |
| spellingShingle | Hamed Poursiami Ayana Moshruba Keiland W. Cooper Derek Gobin Md Abdullah-Al Kaiser Ankur Singh Rouhan Noor Babak Shahbaba Akhilesh Jaiswal Norbert J. Fortin Maryam Parsa A scalable reinforcement learning framework inspired by hippocampal memory mechanisms for efficient contextual and sequential decision making Scientific Reports |
| title | A scalable reinforcement learning framework inspired by hippocampal memory mechanisms for efficient contextual and sequential decision making |
| title_full | A scalable reinforcement learning framework inspired by hippocampal memory mechanisms for efficient contextual and sequential decision making |
| title_fullStr | A scalable reinforcement learning framework inspired by hippocampal memory mechanisms for efficient contextual and sequential decision making |
| title_full_unstemmed | A scalable reinforcement learning framework inspired by hippocampal memory mechanisms for efficient contextual and sequential decision making |
| title_short | A scalable reinforcement learning framework inspired by hippocampal memory mechanisms for efficient contextual and sequential decision making |
| title_sort | scalable reinforcement learning framework inspired by hippocampal memory mechanisms for efficient contextual and sequential decision making |
| url | https://doi.org/10.1038/s41598-025-10586-x |
| work_keys_str_mv | AT hamedpoursiami ascalablereinforcementlearningframeworkinspiredbyhippocampalmemorymechanismsforefficientcontextualandsequentialdecisionmaking AT ayanamoshruba ascalablereinforcementlearningframeworkinspiredbyhippocampalmemorymechanismsforefficientcontextualandsequentialdecisionmaking AT keilandwcooper ascalablereinforcementlearningframeworkinspiredbyhippocampalmemorymechanismsforefficientcontextualandsequentialdecisionmaking AT derekgobin ascalablereinforcementlearningframeworkinspiredbyhippocampalmemorymechanismsforefficientcontextualandsequentialdecisionmaking AT mdabdullahalkaiser ascalablereinforcementlearningframeworkinspiredbyhippocampalmemorymechanismsforefficientcontextualandsequentialdecisionmaking AT ankursingh ascalablereinforcementlearningframeworkinspiredbyhippocampalmemorymechanismsforefficientcontextualandsequentialdecisionmaking AT rouhannoor ascalablereinforcementlearningframeworkinspiredbyhippocampalmemorymechanismsforefficientcontextualandsequentialdecisionmaking AT babakshahbaba ascalablereinforcementlearningframeworkinspiredbyhippocampalmemorymechanismsforefficientcontextualandsequentialdecisionmaking AT akhileshjaiswal ascalablereinforcementlearningframeworkinspiredbyhippocampalmemorymechanismsforefficientcontextualandsequentialdecisionmaking AT norbertjfortin ascalablereinforcementlearningframeworkinspiredbyhippocampalmemorymechanismsforefficientcontextualandsequentialdecisionmaking AT maryamparsa ascalablereinforcementlearningframeworkinspiredbyhippocampalmemorymechanismsforefficientcontextualandsequentialdecisionmaking AT hamedpoursiami scalablereinforcementlearningframeworkinspiredbyhippocampalmemorymechanismsforefficientcontextualandsequentialdecisionmaking AT ayanamoshruba scalablereinforcementlearningframeworkinspiredbyhippocampalmemorymechanismsforefficientcontextualandsequentialdecisionmaking AT keilandwcooper scalablereinforcementlearningframeworkinspiredbyhippocampalmemorymechanismsforefficientcontextualandsequentialdecisionmaking AT derekgobin scalablereinforcementlearningframeworkinspiredbyhippocampalmemorymechanismsforefficientcontextualandsequentialdecisionmaking AT mdabdullahalkaiser scalablereinforcementlearningframeworkinspiredbyhippocampalmemorymechanismsforefficientcontextualandsequentialdecisionmaking AT ankursingh scalablereinforcementlearningframeworkinspiredbyhippocampalmemorymechanismsforefficientcontextualandsequentialdecisionmaking AT rouhannoor scalablereinforcementlearningframeworkinspiredbyhippocampalmemorymechanismsforefficientcontextualandsequentialdecisionmaking AT babakshahbaba scalablereinforcementlearningframeworkinspiredbyhippocampalmemorymechanismsforefficientcontextualandsequentialdecisionmaking AT akhileshjaiswal scalablereinforcementlearningframeworkinspiredbyhippocampalmemorymechanismsforefficientcontextualandsequentialdecisionmaking AT norbertjfortin scalablereinforcementlearningframeworkinspiredbyhippocampalmemorymechanismsforefficientcontextualandsequentialdecisionmaking AT maryamparsa scalablereinforcementlearningframeworkinspiredbyhippocampalmemorymechanismsforefficientcontextualandsequentialdecisionmaking |