On Quantum Natural Policy Gradients
This article delves into the role of the quantum Fisher information matrix (FIM) in enhancing the performance of parameterized quantum circuit (PQC)-based reinforcement learning agents. While previous studies have highlighted the effectiveness of PQC-based policies preconditioned with the quantum FI...
Saved in:
| Main Authors: | , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IEEE
2024-01-01
|
| Series: | IEEE Transactions on Quantum Engineering |
| Subjects: | |
| Online Access: | https://ieeexplore.ieee.org/document/10569042/ |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1832584004320100352 |
|---|---|
| author | Andre Sequeira Luis Paulo Santos Luis Soares Barbosa |
| author_facet | Andre Sequeira Luis Paulo Santos Luis Soares Barbosa |
| author_sort | Andre Sequeira |
| collection | DOAJ |
| description | This article delves into the role of the quantum Fisher information matrix (FIM) in enhancing the performance of parameterized quantum circuit (PQC)-based reinforcement learning agents. While previous studies have highlighted the effectiveness of PQC-based policies preconditioned with the quantum FIM in contextual bandits, its impact in broader reinforcement learning contexts, such as Markov decision processes, is less clear. Through a detailed analysis of Löwner inequalities between quantum and classical FIMs, this study uncovers the nuanced distinctions and implications of using each type of FIM. Our results indicate that a PQC-based agent using the quantum FIM without additional insights typically incurs a larger approximation error and does not guarantee improved performance compared to the classical FIM. Empirical evaluations in classic control benchmarks suggest even though quantum FIM preconditioning outperforms standard gradient ascent, in general, it is not superior to classical FIM preconditioning. |
| format | Article |
| id | doaj-art-e5050aba5e83488686c7b85f3d802c6c |
| institution | Kabale University |
| issn | 2689-1808 |
| language | English |
| publishDate | 2024-01-01 |
| publisher | IEEE |
| record_format | Article |
| series | IEEE Transactions on Quantum Engineering |
| spelling | doaj-art-e5050aba5e83488686c7b85f3d802c6c2025-01-28T00:02:23ZengIEEEIEEE Transactions on Quantum Engineering2689-18082024-01-01511110.1109/TQE.2024.341809410569042On Quantum Natural Policy GradientsAndre Sequeira0https://orcid.org/0000-0002-6659-9277Luis Paulo Santos1https://orcid.org/0000-0003-4466-1129Luis Soares Barbosa2Department of Informatics, University of Minho, Braga, PortugalDepartment of Informatics, University of Minho, Braga, PortugalDepartment of Informatics, University of Minho, Braga, PortugalThis article delves into the role of the quantum Fisher information matrix (FIM) in enhancing the performance of parameterized quantum circuit (PQC)-based reinforcement learning agents. While previous studies have highlighted the effectiveness of PQC-based policies preconditioned with the quantum FIM in contextual bandits, its impact in broader reinforcement learning contexts, such as Markov decision processes, is less clear. Through a detailed analysis of Löwner inequalities between quantum and classical FIMs, this study uncovers the nuanced distinctions and implications of using each type of FIM. Our results indicate that a PQC-based agent using the quantum FIM without additional insights typically incurs a larger approximation error and does not guarantee improved performance compared to the classical FIM. Empirical evaluations in classic control benchmarks suggest even though quantum FIM preconditioning outperforms standard gradient ascent, in general, it is not superior to classical FIM preconditioning.https://ieeexplore.ieee.org/document/10569042/Fisher informationnatural policy gradients (NPGs)quantum policy gradientsquantum reinforcement learning |
| spellingShingle | Andre Sequeira Luis Paulo Santos Luis Soares Barbosa On Quantum Natural Policy Gradients IEEE Transactions on Quantum Engineering Fisher information natural policy gradients (NPGs) quantum policy gradients quantum reinforcement learning |
| title | On Quantum Natural Policy Gradients |
| title_full | On Quantum Natural Policy Gradients |
| title_fullStr | On Quantum Natural Policy Gradients |
| title_full_unstemmed | On Quantum Natural Policy Gradients |
| title_short | On Quantum Natural Policy Gradients |
| title_sort | on quantum natural policy gradients |
| topic | Fisher information natural policy gradients (NPGs) quantum policy gradients quantum reinforcement learning |
| url | https://ieeexplore.ieee.org/document/10569042/ |
| work_keys_str_mv | AT andresequeira onquantumnaturalpolicygradients AT luispaulosantos onquantumnaturalpolicygradients AT luissoaresbarbosa onquantumnaturalpolicygradients |