A Task Offloading and Resource Allocation Strategy Based on Multi-Agent Reinforcement Learning in Mobile Edge Computing
Task offloading and resource allocation is a research hotspot in cloud-edge collaborative computing. Many existing pieces of research adopted single-agent reinforcement learning to solve this problem, which has some defects such as low robustness, large decision space, and ignoring delayed rewards....
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2024-09-01
|
| Series: | Future Internet |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1999-5903/16/9/333 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850261276612100096 |
|---|---|
| author | Guiwen Jiang Rongxi Huang Zhiming Bao Gaocai Wang |
| author_facet | Guiwen Jiang Rongxi Huang Zhiming Bao Gaocai Wang |
| author_sort | Guiwen Jiang |
| collection | DOAJ |
| description | Task offloading and resource allocation is a research hotspot in cloud-edge collaborative computing. Many existing pieces of research adopted single-agent reinforcement learning to solve this problem, which has some defects such as low robustness, large decision space, and ignoring delayed rewards. In view of the above deficiencies, this paper constructs a cloud-edge collaborative computing model, and related task queue, delay, and energy consumption model, and gives joint optimization problem modeling for task offloading and resource allocation with multiple constraints. Then, in order to solve the joint optimization problem, this paper designs a decentralized offloading and scheduling scheme based on “task-oriented” multi-agent reinforcement learning. In this scheme, we present information synchronization protocols and offloading scheduling rules and use edge servers as agents to construct a multi-agent system based on the Actor–Critic framework. In order to solve delayed rewards, this paper models the offloading and scheduling problem as a “task-oriented” Markov decision process. This process abandons the commonly used equidistant time slot model but uses dynamic and parallel slots in the step of task processing time. Finally, an offloading decision algorithm TOMAC-PPO is proposed. The algorithm applies the proximal policy optimization to the multi-agent system and combines the Transformer neural network model to realize the memory and prediction of network state information. Experimental results show that this algorithm has better convergence speed and can effectively reduce the service cost, energy consumption, and task drop rate under high load and high failure rates. For example, the proposed TOMAC-PPO can reduce the average cost by from 19.4% to 66.6% compared to other offloading schemes under the same network load. In addition, the drop rate of some baseline algorithms with 50 users can achieve 62.5% for critical tasks, while the proposed TOMAC-PPO only has 5.5%. |
| format | Article |
| id | doaj-art-3e6dc21428e043aa90630a85bffdbdb2 |
| institution | OA Journals |
| issn | 1999-5903 |
| language | English |
| publishDate | 2024-09-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Future Internet |
| spelling | doaj-art-3e6dc21428e043aa90630a85bffdbdb22025-08-20T01:55:27ZengMDPI AGFuture Internet1999-59032024-09-0116933310.3390/fi16090333A Task Offloading and Resource Allocation Strategy Based on Multi-Agent Reinforcement Learning in Mobile Edge ComputingGuiwen Jiang0Rongxi Huang1Zhiming Bao2Gaocai Wang3School of Artificial Intelligence Technology, Guangxi Technological College of Machinery and Electricity, Nanning 530007, ChinaSchool of Information Engineering, Guangxi Vocational University of Agriculture, Nanning 530007, ChinaSchool of Artificial Intelligence Technology, Guangxi Technological College of Machinery and Electricity, Nanning 530007, ChinaSchool of Computer and Electronic Information, Guangxi University, Nanning 530004, ChinaTask offloading and resource allocation is a research hotspot in cloud-edge collaborative computing. Many existing pieces of research adopted single-agent reinforcement learning to solve this problem, which has some defects such as low robustness, large decision space, and ignoring delayed rewards. In view of the above deficiencies, this paper constructs a cloud-edge collaborative computing model, and related task queue, delay, and energy consumption model, and gives joint optimization problem modeling for task offloading and resource allocation with multiple constraints. Then, in order to solve the joint optimization problem, this paper designs a decentralized offloading and scheduling scheme based on “task-oriented” multi-agent reinforcement learning. In this scheme, we present information synchronization protocols and offloading scheduling rules and use edge servers as agents to construct a multi-agent system based on the Actor–Critic framework. In order to solve delayed rewards, this paper models the offloading and scheduling problem as a “task-oriented” Markov decision process. This process abandons the commonly used equidistant time slot model but uses dynamic and parallel slots in the step of task processing time. Finally, an offloading decision algorithm TOMAC-PPO is proposed. The algorithm applies the proximal policy optimization to the multi-agent system and combines the Transformer neural network model to realize the memory and prediction of network state information. Experimental results show that this algorithm has better convergence speed and can effectively reduce the service cost, energy consumption, and task drop rate under high load and high failure rates. For example, the proposed TOMAC-PPO can reduce the average cost by from 19.4% to 66.6% compared to other offloading schemes under the same network load. In addition, the drop rate of some baseline algorithms with 50 users can achieve 62.5% for critical tasks, while the proposed TOMAC-PPO only has 5.5%.https://www.mdpi.com/1999-5903/16/9/333mobile edge computingcomputing offloadingresource allocationmulti-agent |
| spellingShingle | Guiwen Jiang Rongxi Huang Zhiming Bao Gaocai Wang A Task Offloading and Resource Allocation Strategy Based on Multi-Agent Reinforcement Learning in Mobile Edge Computing Future Internet mobile edge computing computing offloading resource allocation multi-agent |
| title | A Task Offloading and Resource Allocation Strategy Based on Multi-Agent Reinforcement Learning in Mobile Edge Computing |
| title_full | A Task Offloading and Resource Allocation Strategy Based on Multi-Agent Reinforcement Learning in Mobile Edge Computing |
| title_fullStr | A Task Offloading and Resource Allocation Strategy Based on Multi-Agent Reinforcement Learning in Mobile Edge Computing |
| title_full_unstemmed | A Task Offloading and Resource Allocation Strategy Based on Multi-Agent Reinforcement Learning in Mobile Edge Computing |
| title_short | A Task Offloading and Resource Allocation Strategy Based on Multi-Agent Reinforcement Learning in Mobile Edge Computing |
| title_sort | task offloading and resource allocation strategy based on multi agent reinforcement learning in mobile edge computing |
| topic | mobile edge computing computing offloading resource allocation multi-agent |
| url | https://www.mdpi.com/1999-5903/16/9/333 |
| work_keys_str_mv | AT guiwenjiang ataskoffloadingandresourceallocationstrategybasedonmultiagentreinforcementlearninginmobileedgecomputing AT rongxihuang ataskoffloadingandresourceallocationstrategybasedonmultiagentreinforcementlearninginmobileedgecomputing AT zhimingbao ataskoffloadingandresourceallocationstrategybasedonmultiagentreinforcementlearninginmobileedgecomputing AT gaocaiwang ataskoffloadingandresourceallocationstrategybasedonmultiagentreinforcementlearninginmobileedgecomputing AT guiwenjiang taskoffloadingandresourceallocationstrategybasedonmultiagentreinforcementlearninginmobileedgecomputing AT rongxihuang taskoffloadingandresourceallocationstrategybasedonmultiagentreinforcementlearninginmobileedgecomputing AT zhimingbao taskoffloadingandresourceallocationstrategybasedonmultiagentreinforcementlearninginmobileedgecomputing AT gaocaiwang taskoffloadingandresourceallocationstrategybasedonmultiagentreinforcementlearninginmobileedgecomputing |