Retentive neural quantum states: efficient ansätze for ab initio quantum chemistry
Neural-network quantum states (NQS) has emerged as a powerful application of quantum-inspired deep learning for variational Monte Carlo methods, offering a competitive alternative to existing techniques for identifying ground states of quantum problems. A significant advancement toward improving the...
Saved in:
| Main Authors: | , , , , , |
|---|---|
| Format: | Article |
| Language: | English |
| Published: |
IOP Publishing
2025-01-01
|
| Series: | Machine Learning: Science and Technology |
| Subjects: | |
| Online Access: | https://doi.org/10.1088/2632-2153/adcb88 |
| Tags: |
Add Tag
No Tags, Be the first to tag this record!
|
| _version_ | 1850178492432384000 |
|---|---|
| author | Oliver Knitter Dan Zhao James Stokes Martin Ganahl Stefan Leichenauer Shravan Veerapaneni |
| author_facet | Oliver Knitter Dan Zhao James Stokes Martin Ganahl Stefan Leichenauer Shravan Veerapaneni |
| author_sort | Oliver Knitter |
| collection | DOAJ |
| description | Neural-network quantum states (NQS) has emerged as a powerful application of quantum-inspired deep learning for variational Monte Carlo methods, offering a competitive alternative to existing techniques for identifying ground states of quantum problems. A significant advancement toward improving the practical scalability of NQS has been the incorporation of autoregressive models, most recently transformers, as variational ansätze. Transformers learn sequence information with greater expressiveness than recurrent models, but at the cost of increased time complexity with respect to sequence length. We explore the use of the retentive network (RetNet), a recurrent alternative to transformers, as an ansatz for solving electronic ground state problems in ab initio quantum chemistry. Unlike transformers, RetNets overcome this time complexity bottleneck by processing data in parallel during training, and recurrently during inference. We give a simple computational cost estimate of the RetNet and directly compare it with similar estimates for transformers, establishing a clear threshold ratio of problem-to-model size past which the RetNet’s time complexity outperforms that of the transformer. Though this efficiency comes at the expense of decreased expressiveness relative to the transformer, we overcome this gap through training strategies that leverage the autoregressive structure of the model—namely, variational neural annealing. Our findings support the RetNet as a means of improving the time complexity of NQS without sacrificing accuracy. We provide further evidence that the ablative improvements of neural annealing extend beyond the RetNet architecture, suggesting it would serve as an effective general training strategy for autoregressive NQS. |
| format | Article |
| id | doaj-art-2e425b81ff98450cb21c331fec80d496 |
| institution | OA Journals |
| issn | 2632-2153 |
| language | English |
| publishDate | 2025-01-01 |
| publisher | IOP Publishing |
| record_format | Article |
| series | Machine Learning: Science and Technology |
| spelling | doaj-art-2e425b81ff98450cb21c331fec80d4962025-08-20T02:18:43ZengIOP PublishingMachine Learning: Science and Technology2632-21532025-01-016202502210.1088/2632-2153/adcb88Retentive neural quantum states: efficient ansätze for ab initio quantum chemistryOliver Knitter0https://orcid.org/0000-0001-9163-943XDan Zhao1James Stokes2Martin Ganahl3Stefan Leichenauer4https://orcid.org/0009-0004-9701-1968Shravan Veerapaneni5https://orcid.org/0000-0002-2294-7233University of Michigan , Ann Arbor, MI, United States of America; SandboxAQ , Palo Alto, CA, United States of AmericaSandboxAQ , Palo Alto, CA, United States of America; New York University , New York, NY, United States of AmericaUniversity of Michigan , Ann Arbor, MI, United States of AmericaSandboxAQ , Palo Alto, CA, United States of AmericaSandboxAQ , Palo Alto, CA, United States of AmericaUniversity of Michigan , Ann Arbor, MI, United States of AmericaNeural-network quantum states (NQS) has emerged as a powerful application of quantum-inspired deep learning for variational Monte Carlo methods, offering a competitive alternative to existing techniques for identifying ground states of quantum problems. A significant advancement toward improving the practical scalability of NQS has been the incorporation of autoregressive models, most recently transformers, as variational ansätze. Transformers learn sequence information with greater expressiveness than recurrent models, but at the cost of increased time complexity with respect to sequence length. We explore the use of the retentive network (RetNet), a recurrent alternative to transformers, as an ansatz for solving electronic ground state problems in ab initio quantum chemistry. Unlike transformers, RetNets overcome this time complexity bottleneck by processing data in parallel during training, and recurrently during inference. We give a simple computational cost estimate of the RetNet and directly compare it with similar estimates for transformers, establishing a clear threshold ratio of problem-to-model size past which the RetNet’s time complexity outperforms that of the transformer. Though this efficiency comes at the expense of decreased expressiveness relative to the transformer, we overcome this gap through training strategies that leverage the autoregressive structure of the model—namely, variational neural annealing. Our findings support the RetNet as a means of improving the time complexity of NQS without sacrificing accuracy. We provide further evidence that the ablative improvements of neural annealing extend beyond the RetNet architecture, suggesting it would serve as an effective general training strategy for autoregressive NQS.https://doi.org/10.1088/2632-2153/adcb88machine learningquantum chemistryneural quantum statesAI for scienceRetNetsvariational neural annealing |
| spellingShingle | Oliver Knitter Dan Zhao James Stokes Martin Ganahl Stefan Leichenauer Shravan Veerapaneni Retentive neural quantum states: efficient ansätze for ab initio quantum chemistry Machine Learning: Science and Technology machine learning quantum chemistry neural quantum states AI for science RetNets variational neural annealing |
| title | Retentive neural quantum states: efficient ansätze for ab initio quantum chemistry |
| title_full | Retentive neural quantum states: efficient ansätze for ab initio quantum chemistry |
| title_fullStr | Retentive neural quantum states: efficient ansätze for ab initio quantum chemistry |
| title_full_unstemmed | Retentive neural quantum states: efficient ansätze for ab initio quantum chemistry |
| title_short | Retentive neural quantum states: efficient ansätze for ab initio quantum chemistry |
| title_sort | retentive neural quantum states efficient ansatze for ab initio quantum chemistry |
| topic | machine learning quantum chemistry neural quantum states AI for science RetNets variational neural annealing |
| url | https://doi.org/10.1088/2632-2153/adcb88 |
| work_keys_str_mv | AT oliverknitter retentiveneuralquantumstatesefficientansatzeforabinitioquantumchemistry AT danzhao retentiveneuralquantumstatesefficientansatzeforabinitioquantumchemistry AT jamesstokes retentiveneuralquantumstatesefficientansatzeforabinitioquantumchemistry AT martinganahl retentiveneuralquantumstatesefficientansatzeforabinitioquantumchemistry AT stefanleichenauer retentiveneuralquantumstatesefficientansatzeforabinitioquantumchemistry AT shravanveerapaneni retentiveneuralquantumstatesefficientansatzeforabinitioquantumchemistry |