Deep Reinforcement Learning-Based Routing Method for Low Earth Orbit Mega-Constellation Satellite Networks with Service Function Constraints
Low-orbit satellite communication networks have gradually become the research focus of fifth-generation (5G) beyond and sixth generation (6G) networks due to their advantages of wide coverage, large communication capacity, and low terrain influence. However, the low earth orbit mega satellite networ...
Saved in:
| Main Authors: | , , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
MDPI AG
2025-02-01
|
| Series: | Sensors |
| Subjects: | |
| Online Access: | https://www.mdpi.com/1424-8220/25/4/1232 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849719643510407168 |
|---|---|
| author | Yan Chen Huan Cao Longhe Wang Daojin Chen Zifan Liu Yiqing Zhou Jinglin Shi |
| author_facet | Yan Chen Huan Cao Longhe Wang Daojin Chen Zifan Liu Yiqing Zhou Jinglin Shi |
| author_sort | Yan Chen |
| collection | DOAJ |
| description | Low-orbit satellite communication networks have gradually become the research focus of fifth-generation (5G) beyond and sixth generation (6G) networks due to their advantages of wide coverage, large communication capacity, and low terrain influence. However, the low earth orbit mega satellite network (LEO-MSN) also has difficulty in constructing stable traffic transmission paths, network load imbalance and congestion due to the large scale of network nodes, a highly complex topology, and uneven distribution of traffic flow in time and space. In the service-based architecture proposed by 3GPP, the introduction of service function chain (SFC) constraints exacerbates these challenges. Therefore, in this paper, we propose GDRL-SFCR, an end-to-end routing decision method based on graph neural network (GNN) and deep reinforcement learning (DRL) which jointly optimize the end-to-end transmission delay and network load balancing under SFC constraints. Specifically, this method constructs the system model based on the latest NTN low-orbit satellite network end-to-end transmission architecture, taking into account the SFC constraints, transmission delays, and network node loads in the end-to-end traffic transmission, uses a GNN to extract node attributes and dynamic topology features, and uses the DRL method to design specific reward functions to train the model to learn routing policies that satisfy the SFC constraints. The simulation results demonstrate that, compared with graph theory-based methods and reinforcement learning-based methods, GDRL-SFCR can reduce the end-to-end traffic transmission delay by more than 11.3%, reduce the average network load by more than 14.1%, and increase the traffic access success rate and network capacity by more than 19.1% and two times, respectively. |
| format | Article |
| id | doaj-art-c6a7f6c5664f44a287e83cb6b78f3148 |
| institution | DOAJ |
| issn | 1424-8220 |
| language | English |
| publishDate | 2025-02-01 |
| publisher | MDPI AG |
| record_format | Article |
| series | Sensors |
| spelling | doaj-art-c6a7f6c5664f44a287e83cb6b78f31482025-08-20T03:12:07ZengMDPI AGSensors1424-82202025-02-01254123210.3390/s25041232Deep Reinforcement Learning-Based Routing Method for Low Earth Orbit Mega-Constellation Satellite Networks with Service Function ConstraintsYan Chen0Huan Cao1Longhe Wang2Daojin Chen3Zifan Liu4Yiqing Zhou5Jinglin Shi6University of Chinese Academy of Sciences, Beijing 100049, ChinaState Key Laboratory of Processors, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, ChinaState Key Laboratory of Processors, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, ChinaState Key Laboratory of Processors, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, ChinaState Key Laboratory of Processors, Institute of Computing Technology, Chinese Academy of Sciences, Beijing 100190, ChinaUniversity of Chinese Academy of Sciences, Beijing 100049, ChinaUniversity of Chinese Academy of Sciences, Beijing 100049, ChinaLow-orbit satellite communication networks have gradually become the research focus of fifth-generation (5G) beyond and sixth generation (6G) networks due to their advantages of wide coverage, large communication capacity, and low terrain influence. However, the low earth orbit mega satellite network (LEO-MSN) also has difficulty in constructing stable traffic transmission paths, network load imbalance and congestion due to the large scale of network nodes, a highly complex topology, and uneven distribution of traffic flow in time and space. In the service-based architecture proposed by 3GPP, the introduction of service function chain (SFC) constraints exacerbates these challenges. Therefore, in this paper, we propose GDRL-SFCR, an end-to-end routing decision method based on graph neural network (GNN) and deep reinforcement learning (DRL) which jointly optimize the end-to-end transmission delay and network load balancing under SFC constraints. Specifically, this method constructs the system model based on the latest NTN low-orbit satellite network end-to-end transmission architecture, taking into account the SFC constraints, transmission delays, and network node loads in the end-to-end traffic transmission, uses a GNN to extract node attributes and dynamic topology features, and uses the DRL method to design specific reward functions to train the model to learn routing policies that satisfy the SFC constraints. The simulation results demonstrate that, compared with graph theory-based methods and reinforcement learning-based methods, GDRL-SFCR can reduce the end-to-end traffic transmission delay by more than 11.3%, reduce the average network load by more than 14.1%, and increase the traffic access success rate and network capacity by more than 19.1% and two times, respectively.https://www.mdpi.com/1424-8220/25/4/1232LEO satellite networkroutingservice function constraintsgraph convolution networkdeep reinforcement learning |
| spellingShingle | Yan Chen Huan Cao Longhe Wang Daojin Chen Zifan Liu Yiqing Zhou Jinglin Shi Deep Reinforcement Learning-Based Routing Method for Low Earth Orbit Mega-Constellation Satellite Networks with Service Function Constraints Sensors LEO satellite network routing service function constraints graph convolution network deep reinforcement learning |
| title | Deep Reinforcement Learning-Based Routing Method for Low Earth Orbit Mega-Constellation Satellite Networks with Service Function Constraints |
| title_full | Deep Reinforcement Learning-Based Routing Method for Low Earth Orbit Mega-Constellation Satellite Networks with Service Function Constraints |
| title_fullStr | Deep Reinforcement Learning-Based Routing Method for Low Earth Orbit Mega-Constellation Satellite Networks with Service Function Constraints |
| title_full_unstemmed | Deep Reinforcement Learning-Based Routing Method for Low Earth Orbit Mega-Constellation Satellite Networks with Service Function Constraints |
| title_short | Deep Reinforcement Learning-Based Routing Method for Low Earth Orbit Mega-Constellation Satellite Networks with Service Function Constraints |
| title_sort | deep reinforcement learning based routing method for low earth orbit mega constellation satellite networks with service function constraints |
| topic | LEO satellite network routing service function constraints graph convolution network deep reinforcement learning |
| url | https://www.mdpi.com/1424-8220/25/4/1232 |
| work_keys_str_mv | AT yanchen deepreinforcementlearningbasedroutingmethodforlowearthorbitmegaconstellationsatellitenetworkswithservicefunctionconstraints AT huancao deepreinforcementlearningbasedroutingmethodforlowearthorbitmegaconstellationsatellitenetworkswithservicefunctionconstraints AT longhewang deepreinforcementlearningbasedroutingmethodforlowearthorbitmegaconstellationsatellitenetworkswithservicefunctionconstraints AT daojinchen deepreinforcementlearningbasedroutingmethodforlowearthorbitmegaconstellationsatellitenetworkswithservicefunctionconstraints AT zifanliu deepreinforcementlearningbasedroutingmethodforlowearthorbitmegaconstellationsatellitenetworkswithservicefunctionconstraints AT yiqingzhou deepreinforcementlearningbasedroutingmethodforlowearthorbitmegaconstellationsatellitenetworkswithservicefunctionconstraints AT jinglinshi deepreinforcementlearningbasedroutingmethodforlowearthorbitmegaconstellationsatellitenetworkswithservicefunctionconstraints |