Multi-Agent Hierarchical Graph Attention Actor–Critic Reinforcement Learning
Multi-agent systems often face challenges such as elevated communication demands, intricate interactions, and difficulties in transferability. To address the issues of complex information interaction and model scalability, we propose an innovative hierarchical graph attention actor–critic reinforcem...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-12-01
|
| Series: | Entropy |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1099-4300/27/1/4 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832588558252113920 |
|---|---|
| author | Tongyue Li Dianxi Shi Songchang Jin Zhen Wang Huanhuan Yang Yang Chen |
| author_facet | Tongyue Li Dianxi Shi Songchang Jin Zhen Wang Huanhuan Yang Yang Chen |
| author_sort | Tongyue Li |
| collection | DOAJ |
| description | Multi-agent systems often face challenges such as elevated communication demands, intricate interactions, and difficulties in transferability. To address the issues of complex information interaction and model scalability, we propose an innovative hierarchical graph attention actor–critic reinforcement learning method. This method naturally models the interactions within a multi-agent system as a graph, employing hierarchical graph attention to capture the complex cooperative and competitive relationships among agents, thereby enhancing their adaptability to dynamic environments. Specifically, graph neural networks encode agent observations as single feature-embedding vectors, maintaining a constant dimensionality irrespective of the number of agents, which improves model scalability. Through the “inter-agent” and “inter-group” attention layers, the embedding vector of each agent is updated into an information-condensed and contextualized state representation, which extracts state-dependent relationships between agents and model interactions at both individual and group levels. We conducted experiments across several multi-agent tasks to assess our proposed method’s effectiveness, stability, and scalability. Furthermore, to enhance the applicability of our method in large-scale tasks, we tested and validated its performance within a curriculum learning training framework, thereby enhancing its transferability. |
| format | Article |
| id | doaj-art-7575164097924466a36d9c3aca6f6120 |
| institution | Kabale University |
| issn | 1099-4300 |
| language | English |
| publishDate | 2024-12-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Entropy |
| spelling | doaj-art-7575164097924466a36d9c3aca6f61202025-01-24T13:31:37ZengMDPI AGEntropy1099-43002024-12-01271410.3390/e27010004Multi-Agent Hierarchical Graph Attention Actor–Critic Reinforcement LearningTongyue Li0Dianxi Shi1Songchang Jin2Zhen Wang3Huanhuan Yang4Yang Chen5Academy of Military Sciences, Beijing 100097, ChinaAcademy of Military Sciences, Beijing 100097, ChinaAcademy of Military Sciences, Beijing 100097, ChinaAcademy of Military Sciences, Beijing 100097, ChinaCollege of Computer, National University of Defense Technology, Changsha 410073, ChinaSchool of Computer Science, Peking University, Beijing 100871, ChinaMulti-agent systems often face challenges such as elevated communication demands, intricate interactions, and difficulties in transferability. To address the issues of complex information interaction and model scalability, we propose an innovative hierarchical graph attention actor–critic reinforcement learning method. This method naturally models the interactions within a multi-agent system as a graph, employing hierarchical graph attention to capture the complex cooperative and competitive relationships among agents, thereby enhancing their adaptability to dynamic environments. Specifically, graph neural networks encode agent observations as single feature-embedding vectors, maintaining a constant dimensionality irrespective of the number of agents, which improves model scalability. Through the “inter-agent” and “inter-group” attention layers, the embedding vector of each agent is updated into an information-condensed and contextualized state representation, which extracts state-dependent relationships between agents and model interactions at both individual and group levels. We conducted experiments across several multi-agent tasks to assess our proposed method’s effectiveness, stability, and scalability. Furthermore, to enhance the applicability of our method in large-scale tasks, we tested and validated its performance within a curriculum learning training framework, thereby enhancing its transferability.https://www.mdpi.com/1099-4300/27/1/4hierarchical graph attentionmulti-agent reinforcement learningcurriculum learning |
| spellingShingle | Tongyue Li Dianxi Shi Songchang Jin Zhen Wang Huanhuan Yang Yang Chen Multi-Agent Hierarchical Graph Attention Actor–Critic Reinforcement Learning Entropy hierarchical graph attention multi-agent reinforcement learning curriculum learning |
| title | Multi-Agent Hierarchical Graph Attention Actor–Critic Reinforcement Learning |
| title_full | Multi-Agent Hierarchical Graph Attention Actor–Critic Reinforcement Learning |
| title_fullStr | Multi-Agent Hierarchical Graph Attention Actor–Critic Reinforcement Learning |
| title_full_unstemmed | Multi-Agent Hierarchical Graph Attention Actor–Critic Reinforcement Learning |
| title_short | Multi-Agent Hierarchical Graph Attention Actor–Critic Reinforcement Learning |
| title_sort | multi agent hierarchical graph attention actor critic reinforcement learning |
| topic | hierarchical graph attention multi-agent reinforcement learning curriculum learning |
| url | https://www.mdpi.com/1099-4300/27/1/4 |
| work_keys_str_mv | AT tongyueli multiagenthierarchicalgraphattentionactorcriticreinforcementlearning AT dianxishi multiagenthierarchicalgraphattentionactorcriticreinforcementlearning AT songchangjin multiagenthierarchicalgraphattentionactorcriticreinforcementlearning AT zhenwang multiagenthierarchicalgraphattentionactorcriticreinforcementlearning AT huanhuanyang multiagenthierarchicalgraphattentionactorcriticreinforcementlearning AT yangchen multiagenthierarchicalgraphattentionactorcriticreinforcementlearning |