Causally-Aware Reinforcement Learning for Joint Communication and Sensing
The next-generation wireless network, 6G and beyond, envisions to integrate communication and sensing to overcome interference, improve spectrum efficiency, and reduce hardware and power consumption. Massive Multiple-Input Multiple Output (mMIMO)-based Joint Communication and Sensing (JCAS) systems...
Saved in:
| Main Authors: | , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2025-01-01
|
| Series: | IEEE Transactions on Machine Learning in Communications and Networking |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10971373/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850281377743765504 |
|---|---|
| author | Anik Roy Serene Banerjee Jishnu Sadasivan Arnab Sarkar Soumyajit Dey |
| author_facet | Anik Roy Serene Banerjee Jishnu Sadasivan Arnab Sarkar Soumyajit Dey |
| author_sort | Anik Roy |
| collection | DOAJ |
| description | The next-generation wireless network, 6G and beyond, envisions to integrate communication and sensing to overcome interference, improve spectrum efficiency, and reduce hardware and power consumption. Massive Multiple-Input Multiple Output (mMIMO)-based Joint Communication and Sensing (JCAS) systems realize this integration for 6G applications such as autonomous driving, as it requires accurate environmental sensing and time-critical communication with neighbouring vehicles. Reinforcement Learning (RL) is used for mMIMO antenna beamforming in the existing literature. However, the huge search space for actions associated with antenna beamforming causes the learning process for the RL agent to be inefficient due to high beam training overhead. The learning process does not consider the causal relationship between action space and the reward, and gives all actions equal importance. In this work, we explore a causally-aware RL agent which can intervene and discover causal relationships for mMIMO-based JCAS environments, during the training phase. We use a state dependent action dimension selection strategy to realize causal discovery for RL-based JCAS. Evaluation of the causally-aware RL framework in different JCAS scenarios shows the benefit of our proposed solution over baseline methods in terms of the higher reward. We have shown that in the presence of interfering users and sensing signal clutters, our proposed solution achieves 30% higher data rate in comparison to the communication-only state-of-the-art beam pattern learning method while maintaining sensing performance. |
| format | Article |
| id | doaj-art-aa40b40ed3b34ea9b54176fd0d1341fb |
| institution | OA Journals |
| issn | 2831-316X |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Transactions on Machine Learning in Communications and Networking |
| spelling | doaj-art-aa40b40ed3b34ea9b54176fd0d1341fb2025-08-20T01:48:20ZengIEEEIEEE Transactions on Machine Learning in Communications and Networking2831-316X2025-01-01355256710.1109/TMLCN.2025.356255710971373Causally-Aware Reinforcement Learning for Joint Communication and SensingAnik Roy0https://orcid.org/0009-0003-3942-1830Serene Banerjee1https://orcid.org/0000-0002-5579-4819Jishnu Sadasivan2https://orcid.org/0000-0003-3990-8254Arnab Sarkar3https://orcid.org/0000-0002-5930-2180Soumyajit Dey4https://orcid.org/0000-0001-9329-6389Indian Institute of Technology Kharagpur, Kharagpur, IndiaEricsson Research, Bengaluru, IndiaEricsson Research, Bengaluru, IndiaIndian Institute of Technology Kharagpur, Kharagpur, IndiaIndian Institute of Technology Kharagpur, Kharagpur, IndiaThe next-generation wireless network, 6G and beyond, envisions to integrate communication and sensing to overcome interference, improve spectrum efficiency, and reduce hardware and power consumption. Massive Multiple-Input Multiple Output (mMIMO)-based Joint Communication and Sensing (JCAS) systems realize this integration for 6G applications such as autonomous driving, as it requires accurate environmental sensing and time-critical communication with neighbouring vehicles. Reinforcement Learning (RL) is used for mMIMO antenna beamforming in the existing literature. However, the huge search space for actions associated with antenna beamforming causes the learning process for the RL agent to be inefficient due to high beam training overhead. The learning process does not consider the causal relationship between action space and the reward, and gives all actions equal importance. In this work, we explore a causally-aware RL agent which can intervene and discover causal relationships for mMIMO-based JCAS environments, during the training phase. We use a state dependent action dimension selection strategy to realize causal discovery for RL-based JCAS. Evaluation of the causally-aware RL framework in different JCAS scenarios shows the benefit of our proposed solution over baseline methods in terms of the higher reward. We have shown that in the presence of interfering users and sensing signal clutters, our proposed solution achieves 30% higher data rate in comparison to the communication-only state-of-the-art beam pattern learning method while maintaining sensing performance.https://ieeexplore.ieee.org/document/10971373/Causal discoveryreinforcement learningantenna beamformingjoint communicationsensing |
| spellingShingle | Anik Roy Serene Banerjee Jishnu Sadasivan Arnab Sarkar Soumyajit Dey Causally-Aware Reinforcement Learning for Joint Communication and Sensing IEEE Transactions on Machine Learning in Communications and Networking Causal discovery reinforcement learning antenna beamforming joint communication sensing |
| title | Causally-Aware Reinforcement Learning for Joint Communication and Sensing |
| title_full | Causally-Aware Reinforcement Learning for Joint Communication and Sensing |
| title_fullStr | Causally-Aware Reinforcement Learning for Joint Communication and Sensing |
| title_full_unstemmed | Causally-Aware Reinforcement Learning for Joint Communication and Sensing |
| title_short | Causally-Aware Reinforcement Learning for Joint Communication and Sensing |
| title_sort | causally aware reinforcement learning for joint communication and sensing |
| topic | Causal discovery reinforcement learning antenna beamforming joint communication sensing |
| url | https://ieeexplore.ieee.org/document/10971373/ |
| work_keys_str_mv | AT anikroy causallyawarereinforcementlearningforjointcommunicationandsensing AT serenebanerjee causallyawarereinforcementlearningforjointcommunicationandsensing AT jishnusadasivan causallyawarereinforcementlearningforjointcommunicationandsensing AT arnabsarkar causallyawarereinforcementlearningforjointcommunicationandsensing AT soumyajitdey causallyawarereinforcementlearningforjointcommunicationandsensing |