Graph-based reinforcement learning for software-defined networking traffic engineering
Abstract With the continuous expansion of global Internet infrastructure, wide area networks play a crucial role in transmitting traffic between multiple data centers and users worldwide. However, efficient traffic management has become a core challenge due to the high costs of building and maintain...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
Springer
2025-07-01
|
| Series: | Journal of King Saud University: Computer and Information Sciences |
| Subjects: | |
| Online Access: | https://doi.org/10.1007/s44443-025-00133-z |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1849332192086327296 |
|---|---|
| author | Jingwen Lu Chaowei Tang Wenyu Ma Wenjuan Xing |
| author_facet | Jingwen Lu Chaowei Tang Wenyu Ma Wenjuan Xing |
| author_sort | Jingwen Lu |
| collection | DOAJ |
| description | Abstract With the continuous expansion of global Internet infrastructure, wide area networks play a crucial role in transmitting traffic between multiple data centers and users worldwide. However, efficient traffic management has become a core challenge due to the high costs of building and maintaining these networks. Traditional traffic engineering methods based on linear programming achieve optimal solutions but suffer from exponential computational complexity growth with network size, making them impractical for real-time applications in large-scale networks. Recent machine learning approaches show promise but still face fundamental limitations in handling complex network constraints and maintaining performance across different network scales. This paper proposes GRL-TE (Graph-based Reinforcement Learning for Traffic Engineering), a novel framework that achieves near-optimal performance while maintaining computational efficiency across diverse network scales. GRL-TE introduces three key innovations: (1) TopoFlowNet, a graph neural network architecture that models WANs as bipartite graphs with edge nodes representing physical links and path nodes representing candidate paths, enabling efficient bidirectional information propagation through GINConv layers while MLP modules handle collaborative relationships among paths serving the same demand; (2) A one-step A2C mechanism specifically designed for TE with immediate reward structure, eliminating the need for future state estimation and significantly simplifying training; (3) Integration of ADMM as a post-processing step to iteratively reduce constraint violations while improving solution quality. Extensive experiments on six real-world WAN topologies ranging from 12 to 1,739 nodes demonstrate that GRL-TE achieves an overall average demand satisfaction rate of 89.36%, outperforming state-of-the-art learning-based methods (Teal: 82.04%, Figret: 82.20%) and the clustering-based NCFlow (76.48%), while providing 3-4 orders of magnitude speedup compared to LP solvers on large-scale networks. The framework maintains robust performance under link failures and meets real-time scheduling requirements for production deployment. |
| format | Article |
| id | doaj-art-e2ab725e61a04432ab47378dd6a18fde |
| institution | Kabale University |
| issn | 1319-1578 2213-1248 |
| language | English |
| publishDate | 2025-07-01 |
| publisher | Springer |
| record_format | Article |
| series | Journal of King Saud University: Computer and Information Sciences |
| spelling | doaj-art-e2ab725e61a04432ab47378dd6a18fde2025-08-20T03:46:16ZengSpringerJournal of King Saud University: Computer and Information Sciences1319-15782213-12482025-07-0137612310.1007/s44443-025-00133-zGraph-based reinforcement learning for software-defined networking traffic engineeringJingwen Lu0Chaowei Tang1Wenyu Ma2Wenjuan Xing3School of Microelectronics and Communication Engineering, Chongqing UniversitySchool of Microelectronics and Communication Engineering, Chongqing UniversitySchool of Microelectronics and Communication Engineering, Chongqing UniversitySchool of Microelectronics and Communication Engineering, Chongqing UniversityAbstract With the continuous expansion of global Internet infrastructure, wide area networks play a crucial role in transmitting traffic between multiple data centers and users worldwide. However, efficient traffic management has become a core challenge due to the high costs of building and maintaining these networks. Traditional traffic engineering methods based on linear programming achieve optimal solutions but suffer from exponential computational complexity growth with network size, making them impractical for real-time applications in large-scale networks. Recent machine learning approaches show promise but still face fundamental limitations in handling complex network constraints and maintaining performance across different network scales. This paper proposes GRL-TE (Graph-based Reinforcement Learning for Traffic Engineering), a novel framework that achieves near-optimal performance while maintaining computational efficiency across diverse network scales. GRL-TE introduces three key innovations: (1) TopoFlowNet, a graph neural network architecture that models WANs as bipartite graphs with edge nodes representing physical links and path nodes representing candidate paths, enabling efficient bidirectional information propagation through GINConv layers while MLP modules handle collaborative relationships among paths serving the same demand; (2) A one-step A2C mechanism specifically designed for TE with immediate reward structure, eliminating the need for future state estimation and significantly simplifying training; (3) Integration of ADMM as a post-processing step to iteratively reduce constraint violations while improving solution quality. Extensive experiments on six real-world WAN topologies ranging from 12 to 1,739 nodes demonstrate that GRL-TE achieves an overall average demand satisfaction rate of 89.36%, outperforming state-of-the-art learning-based methods (Teal: 82.04%, Figret: 82.20%) and the clustering-based NCFlow (76.48%), while providing 3-4 orders of magnitude speedup compared to LP solvers on large-scale networks. The framework maintains robust performance under link failures and meets real-time scheduling requirements for production deployment.https://doi.org/10.1007/s44443-025-00133-zTraffic engineeringGraph neural networksDeep reinforcement learningSoftware-defined networks |
| spellingShingle | Jingwen Lu Chaowei Tang Wenyu Ma Wenjuan Xing Graph-based reinforcement learning for software-defined networking traffic engineering Journal of King Saud University: Computer and Information Sciences Traffic engineering Graph neural networks Deep reinforcement learning Software-defined networks |
| title | Graph-based reinforcement learning for software-defined networking traffic engineering |
| title_full | Graph-based reinforcement learning for software-defined networking traffic engineering |
| title_fullStr | Graph-based reinforcement learning for software-defined networking traffic engineering |
| title_full_unstemmed | Graph-based reinforcement learning for software-defined networking traffic engineering |
| title_short | Graph-based reinforcement learning for software-defined networking traffic engineering |
| title_sort | graph based reinforcement learning for software defined networking traffic engineering |
| topic | Traffic engineering Graph neural networks Deep reinforcement learning Software-defined networks |
| url | https://doi.org/10.1007/s44443-025-00133-z |
| work_keys_str_mv | AT jingwenlu graphbasedreinforcementlearningforsoftwaredefinednetworkingtrafficengineering AT chaoweitang graphbasedreinforcementlearningforsoftwaredefinednetworkingtrafficengineering AT wenyuma graphbasedreinforcementlearningforsoftwaredefinednetworkingtrafficengineering AT wenjuanxing graphbasedreinforcementlearningforsoftwaredefinednetworkingtrafficengineering |