Joint Adaptive OFDM and Reinforcement Learning Design for Autonomous Vehicles: Leveraging Age of Updates
Millimeter wave (mmWave)-based orthogonal frequency-division multiplexing (OFDM) stands out as a suitable alternative for high-resolution sensing and high-speed data transmission. To meet communication and sensing requirements, many works propose a static configuration where the wave's hy...
Saved in:
| Main Authors: | , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Open Journal of Vehicular Technology |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10842044/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832088057235374080 |
|---|---|
| author | Mamady Delamou Ahmed Naeem Huseyin Arslan El Mehdi Amhoud |
| author_facet | Mamady Delamou Ahmed Naeem Huseyin Arslan El Mehdi Amhoud |
| author_sort | Mamady Delamou |
| collection | DOAJ |
| description | Millimeter wave (mmWave)-based orthogonal frequency-division multiplexing (OFDM) stands out as a suitable alternative for high-resolution sensing and high-speed data transmission. To meet communication and sensing requirements, many works propose a static configuration where the wave's hyperparameters such as the number of symbols in a frame and the number of frames in a communication slot are already predefined. However, two facts oblige us to redefine the problem, 1) the environment is often dynamic and uncertain, and 2) mmWave is severely impacted by wireless environments. A striking example where this challenge is very prominent is autonomous vehicle (AV). Such a system leverages integrated sensing and communication (ISAC) using mmWave to manage data transmission and the dynamism of the environment. In this work, we consider an autonomous vehicle network where an AV utilizes its queue state information (QSI) and channel state information (CSI) in conjunction with reinforcement learning techniques to manage communication and sensing. This enables the AV to achieve two primary objectives: establishing a stable communication link with other AVs and accurately estimating the velocities of surrounding objects with high resolution. The communication performance is therefore evaluated based on the queue state, the effective data rate, and the discarded packets rate. In contrast, the effectiveness of the sensing is assessed using the velocity resolution. In addition, we exploit adaptive OFDM techniques for dynamic modulation, and we suggest a reward function that leverages the age of updates to handle the communication buffer and improve sensing. The system is validated using advantage actor-critic (A2C) and proximal policy optimization (PPO). Furthermore, we compare our solution with the existing design and demonstrate its superior performance by computer simulations. |
| format | Article |
| id | doaj-art-b123fce502b24ca1ada5d6ce8696b130 |
| institution | Kabale University |
| issn | 2644-1330 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Open Journal of Vehicular Technology |
| spelling | doaj-art-b123fce502b24ca1ada5d6ce8696b1302025-02-06T00:00:57ZengIEEEIEEE Open Journal of Vehicular Technology2644-13302025-01-01645547010.1109/OJVT.2025.353000810842044Joint Adaptive OFDM and Reinforcement Learning Design for Autonomous Vehicles: Leveraging Age of UpdatesMamady Delamou0https://orcid.org/0000-0002-4084-7156Ahmed Naeem1https://orcid.org/0000-0002-1534-5883Huseyin Arslan2https://orcid.org/0000-0001-9474-7372El Mehdi Amhoud3https://orcid.org/0000-0001-6630-5083College of Computing, University Mohammed VI Polytechnic, Benguerir, MoroccoDepartment of Electrical and Electronics Engineering, Istanbul Medipol University, Istanbul, TürkiyeDepartment of Electrical and Electronics Engineering, Istanbul Medipol University, Istanbul, TürkiyeCollege of Computing, University Mohammed VI Polytechnic, Benguerir, MoroccoMillimeter wave (mmWave)-based orthogonal frequency-division multiplexing (OFDM) stands out as a suitable alternative for high-resolution sensing and high-speed data transmission. To meet communication and sensing requirements, many works propose a static configuration where the wave's hyperparameters such as the number of symbols in a frame and the number of frames in a communication slot are already predefined. However, two facts oblige us to redefine the problem, 1) the environment is often dynamic and uncertain, and 2) mmWave is severely impacted by wireless environments. A striking example where this challenge is very prominent is autonomous vehicle (AV). Such a system leverages integrated sensing and communication (ISAC) using mmWave to manage data transmission and the dynamism of the environment. In this work, we consider an autonomous vehicle network where an AV utilizes its queue state information (QSI) and channel state information (CSI) in conjunction with reinforcement learning techniques to manage communication and sensing. This enables the AV to achieve two primary objectives: establishing a stable communication link with other AVs and accurately estimating the velocities of surrounding objects with high resolution. The communication performance is therefore evaluated based on the queue state, the effective data rate, and the discarded packets rate. In contrast, the effectiveness of the sensing is assessed using the velocity resolution. In addition, we exploit adaptive OFDM techniques for dynamic modulation, and we suggest a reward function that leverages the age of updates to handle the communication buffer and improve sensing. The system is validated using advantage actor-critic (A2C) and proximal policy optimization (PPO). Furthermore, we compare our solution with the existing design and demonstrate its superior performance by computer simulations.https://ieeexplore.ieee.org/document/10842044/Age of updatesautonomous vehiclesintegrated sensing and communicationoptimizationreinforcement learningwaveform |
| spellingShingle | Mamady Delamou Ahmed Naeem Huseyin Arslan El Mehdi Amhoud Joint Adaptive OFDM and Reinforcement Learning Design for Autonomous Vehicles: Leveraging Age of Updates IEEE Open Journal of Vehicular Technology Age of updates autonomous vehicles integrated sensing and communication optimization reinforcement learning waveform |
| title | Joint Adaptive OFDM and Reinforcement Learning Design for Autonomous Vehicles: Leveraging Age of Updates |
| title_full | Joint Adaptive OFDM and Reinforcement Learning Design for Autonomous Vehicles: Leveraging Age of Updates |
| title_fullStr | Joint Adaptive OFDM and Reinforcement Learning Design for Autonomous Vehicles: Leveraging Age of Updates |
| title_full_unstemmed | Joint Adaptive OFDM and Reinforcement Learning Design for Autonomous Vehicles: Leveraging Age of Updates |
| title_short | Joint Adaptive OFDM and Reinforcement Learning Design for Autonomous Vehicles: Leveraging Age of Updates |
| title_sort | joint adaptive ofdm and reinforcement learning design for autonomous vehicles leveraging age of updates |
| topic | Age of updates autonomous vehicles integrated sensing and communication optimization reinforcement learning waveform |
| url | https://ieeexplore.ieee.org/document/10842044/ |
| work_keys_str_mv | AT mamadydelamou jointadaptiveofdmandreinforcementlearningdesignforautonomousvehiclesleveragingageofupdates AT ahmednaeem jointadaptiveofdmandreinforcementlearningdesignforautonomousvehiclesleveragingageofupdates AT huseyinarslan jointadaptiveofdmandreinforcementlearningdesignforautonomousvehiclesleveragingageofupdates AT elmehdiamhoud jointadaptiveofdmandreinforcementlearningdesignforautonomousvehiclesleveragingageofupdates |